Let us not skip so lightly past censorship effects of bad peer review, Orac.

Aug 12 2010 Published by under Careerism, Peer Review, Scientific Publication

Orac has a discussion up in which he dissects allegations of problems with the peer review process and solutions that have been proposed. In this he is responding to a piece in The Scientist catchily titled "I Hate Your Paper". One part of the discussion hinged on this quote from the bit in The Scientist:

since top journals get too many submissions and it’s easier to just reject a paper than spend the time to improve it. Regardless of the motivation, the result is the same, and it’s a “problem,” Kaplan says, “that can very quickly become censorship.”

Orac thinks this is overblown:

Most recently, it took me multiple submissions to four different journals to get a manuscript published. It took nearly a year and a half and more hours of writing and rewriting and doing more experiments than I can remember. But "censorship"? I'm half tempted ot respond to Dr. Kaplan: Censorship. You keep using that word. I do not think it means what you think it means. In fact, I just did.

No, incompetent or biased peer review is not "censorship." It's incompetent or biased peer review, and it's a problem that needs to be dealt with wherever and whenever possible.

I'm not so sure I'm willing to back away from the word "censorship" quite so readily. Orac has the perspective of a relatively established, independent investigator who furthermore, if push really came to shove, could abandon his research laboratory for some interval and just work as a surgeon. (I'm not exactly sure what his balance is but he makes it pretty clear that being a surgeon is a substantial part of his work life.)

Even if we only consider established investigators, well, these are the folks that have the endurance to put up with submission to four journals and can sustain the effort to keep conducting new experiments and revising the paper. The more junior scientists do not have this luxury. Grad students and postdocs may need to simply cut bait and move on..if nobody in the lab takes up the reins, the publication may never see the light of day. Or the data may come out but the original person's contribution has been downgraded from first author to middle author.

Even junior faculty are at risk. They need publication output to get their first grants and ultimately to sustain their tenure bid. They might be working on a number of different lines of work and if one pub is held up, and another sails through, well, this may dictate the next 10-15 years of their career focus. Papers lead to research grants which lead to more papers. Etc.

I have several lines of research that I've pursued at various times in the past. Many of them I no longer pursue. There are still interesting questions there. In some cases other people have jumped into the topic or even gotten a grant awarded that is more or less what I've proposed in the past and never got funded. It happens, this is the natural process of things and I'm not complaining. I make my choices based on a constellation of personal interest, lab interest/skills, promising research findings and what I can manage to get funded.

Nevertheless, it may be the case for some people that an unfairly antagonistic peer review process curtails the publication of data, prevents the securing of funding and ultimately cuts off that research tendril from the laboratory's future.

I understand why Orac is leery of calling this censorship...because it is not the intent of the peer review system to censor. Unfortunately, bad peer review can have the effect of censorship. This is a valid reason to want to improve the process by which manuscripts are vetted for publication.

7 responses so far

  • [...] This post was mentioned on Twitter by drdr A and mavericksmusing, ScientopiaBlogs. ScientopiaBlogs said: Let us not skip so lightly past censorship effects of bad peer review, Orac. http://dlvr.it/3jjYJ [...]

  • physioprof says:

  • Bioorganic chemist says:

    Excellent discussion. The effects on grad students (first to non-first authorship) and on research directions are particularly salient. I had a project go from "exceptional work, let's see it published" to "one of several ways to do this" in study section (over a couple years and grant variants) due to extreme delays in publishing two major papers, both held up (in multiple rounds, in multiple journals) by one PITA reviewer (clearly a competitor) outweighing 3-4 strong reviews (and an editor that was impervious to my pleas). Publication delay -> non-funding of promising project -> no longer a major project in the lab (despite it being my favorite project).

    In the end, I think that the problem is really the editors, and one that can be solved by editors taking a more proactive role in weeding out bullshit reasons not to publish.

  • DrugMonkey says:

    Editors who are active scientists themselves are more likely to recognize and cut through conflicted reviewer BS than are the professional editors. Just another crime to chalk up to GlamourMagification of science.

  • physioprof says:

    They might be working on a number of different lines of work and if one pub is held up, and another sails through, well, this may dictate the next 10-15 years of their career focus. Papers lead to research grants which lead to more papers. Etc.

    This is an exaggeration. Once you get your grant, you can do what you want (within reason), and if you're productive, no one is gonna go through your original specific aims with a fine-tooth comb.

    And I'm with Orac on the "censorship" thing. It is the authors' choice to submit to high-impact journals with low acceptance rates. It ain't censorship because you spent two years going back and forth with Cell only to end up rejected, when you could have submitted to some fucken IF = 3 journal no one reads and sailed into print.

  • Especially if the conclusions on the paper run counter to what the journal readily publishes (in my field such an issue is the question of whether pesticides are problematic or not in all -- or just some -- watersheds). I've seen papers rejected without review, only to be accepted elsewhere with minimal if any changes, where the conclusions ran counter to the journals usual mantra. Censorship? you betcha.

  • [...] There is a recent piece in the The Scientist titled “I Hate Your Paper” that outlines the problems with the current peer review system and describes some of the alternatives that have been suggested. This has been discussed byOrac, which was followed up on by DrugMonkey. [...]

Leave a Reply