Complementary Medicine Has Reached the Centre of Scientific Interest: Some Thoughts on Ioannidis’ Citation DatabaseWalach H.
Complement Med Res 2019;26:373–375
Free full text:
https://www.karger.com/Article/FullText/504467Excerpts:
"Not long ago, John P.A. Ioannidis, probably one of the most frequently cited and visible medical methodologists and public health researchers, published a database of the 100,000 most frequently cited scientists worldwide [1]. This database ranks researchers and authors according to their impact in terms of citations, i.e., the extent to which other researchers and authors are making use of their work. This has become a standard way of measuring scientific impact, because it provides us with standardized metrics across disciplines."
"Now, obviously, there are a couple of pitfalls. If researchers get away with citing their own work a lot and at the same time publish a lot, they artificially enhance their h-index, because perhaps 40 or 50% of the citations are not made by other authors but appear in work by the researchers themselves. In addition, areas where there is a lot of active research, such as clinical medicine or cell biology, will automatically produce more citations of important papers than smaller areas like philosophy or social sciences. Ioannidis tried to avoid such pitfalls by various means, e.g., purging the h-index of self-citations and adjusting the ranking to the relative importance in a field. The ranking in the database is therefore the relative ranking of each author with potential distortions having been eliminated. However, there are 2 generic distortions germane to the science metric. First, heavy-handed monographs, that might be central to a field, are not normally taken into account...The other distortion is, of course, the content. No majority vote can define the value of content, and no numerical index can reflect this value. The h-index is a formal measure. How often something is cited is not necessarily indicative of quality."
"The original publication, available online [1], offers a link to supplementary files that contain the original datasets that can be filtered and searched. It is instructive to do this for the field of complementary and alternative medicine (CAM), which is in fact its own professional discipline within the database. If we do this, we find that altogether 44 authors of CAM articles are among the 100,000 most influential authors worldwide...I was quite surprised by this finding, considering the comparatively small field of CAM research within the larger field of medical and scientific research. Estimating that perhaps 5,000 authors are active in CAM research worldwide, 1% of these are among these 100,000 most influential writers in this database."
"It could be helpful to use this database to gauge the 'scientific impact' of CAM and CAM researchers against those who would publicly denounce CAM research as unscientific. Anyone can download the original table and search for the names of these people. The prominent skeptics often don’t even show up among the 100,000 most important researchers and, if they do, they are perhaps less prominent than they would like others to believe. Daniel Dennett, for example, prominent skeptic and self-declared solver of the mystery of consciousness, ranks 73,779th. Richard Dawkins does not even appear in the Table. Of the 12 members on the scientific board of the association of German skeptics frequently called upon by the press to issue statements regarding the scientific viability of CAM treatments and other supposedly unscientific areas, only 3 appear in the table. This will surely not change the public discourse as this is more about politics than about data and evidence, but it may help to keep it centered."
"Another thought: performing CAM research was an uphill battle for those who started it. The funding streams are far scarcer and narrower than for conventional research. It was difficult, initially, to publish CAM research in mainstream journals, and it took a while for dedicated journals such as Complementary Medicine Research, one of the oldest in the field, to be established. It is a success worth celebrating that CAM research has not stayed in the niche of the small community citing its own findings, otherwise the researchers in this field would have never made it to rank within the 100,000 most frequently cited worldwide. For most of the time, our research findings have not been to the liking of the mainstream community, and they are somewhat begrudgingly accepted and published. Some of the authors who fed into the mainstream narrative of CAM being useless and ineffective were of course welcomed more warmly and quoted more widely, but, by and large, this is not the reason for the success of CAM research.
"Some of these researchers have been active for 20 years or more. We show here, in Table 1, that CAM has indeed become mainstream, and it would be unscientific to deny or overlook this fact. It would be wise for everyone claiming that CAM research is unscientific to first take a potentially sobering look into this database to gauge their own standing before they speak on behalf of a community who has possibly not even taken much notice of their own research so far."
>>>>>>>>>>>>>>>>>>>>>>
Comment: Walach's comment that "The prominent skeptics often don’t even show up among the 100,000 most important researchers" is amusing. If you look at Table 1, you can see that Edzard Ernst far surpasses the others on the list.
