|
15 Feb 2010: The general public is often lead to believe that the much-vaunted peer review process is THE "Golden Standard" in research. In speaking with members of the American public, the general impression appears to be something like this: Peer reviewed papers are subjected to such careful and thorough examination that all errors can be identified before a paper is accepted for publication, which implies that peer-reviewed research is without error.
A key point at which to start is to note that reviewers are generally volunteers; they receive no compensation and often little acknowledgement. They are also [supposed to be] anonymous to the author(s) of any paper they review. At the same time, the editorial staff and reviewers are supposed to keep submissions confidential. Because of these things, the rigor of a review is limited by the willingness or resources of the review to dig deep. Further, a reviewer may know the author(s) of the paper they are reviewing, and thus be biased one way or an other. Former professors and former students, former colleagues, competitors at other research institutions. Reviewers also often do not have access to the supporting data, computer code or other basic. That is, they often only have the paper documenting the research. If something is not present in the paper, the reviewer will be unlikely to know of its existence. Finally, reviewers may be given limits of what they can request of the author.
Remember too that more publications in a researcher's curriculum vita or resume, can enhance his or her reputation, leading to offers of tenure (which may bring better pay). Tenure is usually considered the equivalent of "job-for-life."
Of course, nothing stops anyone from citing literature that is not peer-reviewed while implying that it has been, or for even summarizing the research document correctly.
Unfortunately, we have much evidence that peer review is no longer so good (if it ever was), and the evidence continues to mount. It's evident in many fields, from genetics, to environment to medicine. Consider, for example, the case of Hwang Woo-Suk, the South Korean veterinarian who had claimed to have cloned embryonic human stem cells. His results had been published in the major scientific journal Science.
The recent Climategate incident, which made public selected emails, computer code and other documents (some of an advocacy nature) from the Climactic Reserch Unit at the University of East Anglia provides the means to see how, in the matter of climate science, the system can be twisted to benefit some, and erect barriers to others. That is, the system's value is overstated, and that the process in use by too many "professional journals" has become corrupted. Some may assert that there was concern that "bad science" might be published, but the perception of "good" and "bad" are subjective terms. What should matter most is if the work is based on good scientific observations and reproducible, and that it adds positively to the understanding of some phenomenon.
Highly respected researchers David H. Douglass, Professor of Physics, University of Rochester, New York, and John R. Christy, Distinguished Professor, Atmospheric Science, University of Alabama at Huntsville, provide a discussion here and here about how a paper they co-authored with others was delayed from print publication for nearly a year by the Journal while a challenge could be mounted by another group. The challenging paper (the review, acceptance and publication of which was accelerated by Journal staff) was published as a separate paper. The normal route would have a civil exchange of comments and responses to the original paper.Why? Douglass, et al., would have probably been afforded the closing remarks in the exchange. An addendum submitted to the Journal by Douglass was apparently "lost" by editors at the Journal of Climatology system.
Transparency? Ethics? What are those???
Read More:
Climategate documents archive: http://www.eastangliaemails.com/index.php">http://www.eastangliaemails.com/index.php
Christy & Douglass:
- http://wattsupwiththat.com/2009/12/21/gaming-the-peer-review-system-ipcc-scientists-behaving-badly/
- http://www.americanthinker.com/2009/12/a_climatology_conspiracy.html
On Scientific Misconduct, more generally:
- http://en.wikipedia.org/wiki/Scientific_misconduct
- http://en.wikipedia.org/wiki/Hwang_Woo-Suk (Hwang Woo-Suk)
|
