Quality control in science journals

Like every human endeavour, the scientific review process is fraught with human frailties and can be hijacked in a variety of ways. As a result, it is the subject of much experimentation. Quality control in science journals is evolving, and a code of ethics is in hot pursuithttp://econ.st/1DIT6Ud



Oath market
THE process by which academics check the work of their colleagues before it goes to print—peer review, in the argot—is nearly as old as scientific publishing...
ECON.ST

THE process by which academics check the work of their colleagues before it goes to print—peer review, in the argot—is nearly as old as scientific publishing itself. But like every human endeavour, it is fraught with human frailties and the process can be hijacked in a variety of ways. As a result, and as with many other aspects of publishing, peer review is the subject of much experimentation. One upstart publisher is trying to codify good behaviour.

Peer review's current incarnation took shape in the middle of the 20th century: authors submit a manuscript to a publisher, who then seeks out academics suitable to comment on it; they then submit critiques anonymously to the authors, who amend the work to reflect the critiques. The system nearly works. The reasons for anonymity are manifold, but that information asymmetry often causes trouble, with reviewers shooting down rivals' work, pinching ideas, or just plain dragging their feet (overwhelmingly, reviewing is unpaid).

There are a few green shoots of innovation in the field, though. One idea is to remove the veil and carry out peer review publicly: reviewers' identities and their reports are published online for all to see. Proponents reckon this provides incentives for both honesty and courtesy. Faculty of 1000, an online biology and medicine publisher, has taken this tack with F1000 Research, its flagship journal.

Indeed it is taking the idea further. Michael Markie, an associate publisher for F1000 Research, believes that a commitment to change must also come from authors and reviewers, not just journal editors and publishers. Mr Markie was a co-author of a paper—itself the subject of fervent open peer-review—which proposed a kind of oath and a set of guidelines to encourage even-handed and helpful behaviours for reviewers. The oath reads

Principle 1: I will sign my name to my review

Principle 2: I will review with integrity

Principle 3: I will treat the review as a discourse with you; in particular, I will provide constructive criticism

Principle 4: I will be an ambassador for the practice of open science

Faculty of 1000 has begun to encourage reviewers to cite the oath in their reports, in the hope that other publishers will adopt the practice as well. Already, Pensoft Publishers and the Journal of Open Research Software are following suit. 

All of this may sound a bit twee. But the truth is that there is no peer-review training. Marcia McNutt, editor-in-chief of Science, a high-profile journal, is concerned that some publishers implicitly assume that reviewers are aware of an appropriate code of conduct. That is not the case, which is why the simple-sounding oath is better than no guidance at all. Dr McNutt suggests that each journal could tailor it to suit the circumstances of different fields: agreeing to respect the author’s intellectual property, for example, or declaring all conflicts of interest.

But bringing everything out in the open for a civil discussion does not solve all of peer review's problems. One criticism of open peer review is that, under the gaze of all and sundry, reviewers tend to be too gentle, and Dr McNutt fears that inadequate criticism might do more of a disservice to science than the "bruised sensibilities from closed reviews".

As a result, several journals, including some from the prestigious Nature stable, have moved in the opposite direction. They have introduced a double-blind system that is already in use in much of the humanities: both reviewer and author remain anonymous. This does not always work—some fields are so small, and some academics' proclivities so particular, that identities can be easily guessed.

Ulrich Pöschl, editor of the European Geosciences Union (EGU) journal Atmospheric Chemistry and Physics, suggests having open peer review but offering reviewers the option of anonymity—the way nearly every EGU journal is run. Dr Pöschl believes that may strike the perfect balance between the two extremes, and says that the overall quality of output has gone up as the journals have shifted to interactive, open review.

All of this is experimentation that will, in time, be good for science. Publishing will continue its relentless move to online and publishers will work to increase the services and added value that authors demand. A nudge toward more professional dealings along the way certainly cannot hurt.