Peer Review: It’s Not What You Think It Is

| May 29, 2015

DNA_Structure+Key+Labelled.pn_NoBBThe good news is that the peer review process is not at the heart of the scientific process. Peer review refers to the review of an article by an author’s peers before it is submitted to a journal. The reviewers do not duplicate the work or corroborate it. They evaluate it on the basis of proper methodology and its interest to the readership of the journal. The most important criteria in the peer review is that the article is comprehensive enough so someone who reads the article after it is published can duplicate the work or use the results in their own work.

This creates the basis for an ongoing process of other scientists who read the paper and attempt to duplicate or critique the paper’s findings or conclusions. The results of the independent verifications or successful (or not successful) application of the original work will appear in the new papers submitted by those investigators and they will cite the original paper in their own paper. The professional journals and other citation serviices will keep track of the citations, such that if you were coming to the subject a while after the original article was published and needed to know or use the results, you would pull that paper and all subsequent papers that cite that one.  Those papers will be cited as well so you can study the every growing web of independent investigators who successfully or unsuccessfully corroborated the original work or used it in their own work successfully or unsuccessfully. The growing web of citations will also help you evaluate the work of the evaluators as they will be using each other’s work as well.

When does this process end? Never. All published articles will remain under this kind of scrutiny forever until no one is interested in any of it down the line. This is how real science is done and tested over and over again for decades and even for hundreds of years in some cases. The peer review process only serves to ensure that the original article is complete enough so the process I described above can move along properly.

It is disappointing when a fraudulent article gets through the peer review process, but remember that the peer reviewers are not duplicating the work. The reason why fraudulent articles are eventually exposed is because of the relentless process of post publication evaluation I described above.

I sometimes get the feeling that interested laymen think that the pre-publication peer review is how science is established. As if when it is published it becomes “received knowledge.” But no, in science if it gets published it is considered an untested hypothesis, which will then require independent verification by many different independent investigators before it becomes an established theory. This is also why non-professional sources like YouTube videos and religious blogs like AIG are not reliable regardless of their content. There are no mechanisms for providing an audit trail of all the subsequent findings by different people that corroborate the original findings.

Go to any professional scientific journal website, look up an article and you will see the citations “tab” what will list all the papers that cite this one to date. And that includes papers published in other professional journals, not just teh original one. For example, here is an article in a publically accessible professional journal. Click on the Metrics tab and then scroll down the metrics page and click on Cited panel if it is not already open. Then pick one of the citation lists such as ISI Web of Science. A list of links will come up to all the articles published by other scientists who duplicated or used the work in this article and cited it as the source.

Tags: , ,

Category: Science and Evidence

About the Author ()

Comments are closed.