We had a previous extended discussion on the ways in which scientists might react to the rejection of a submitted manuscript which followed a post over at double-doc's place. The discussions following all of these posts touched on one of the annoyances of manuscript peer review, namely requests for the authors to provide extensive additional experimental work to justify acceptance for publication.
It seems a trio of senior chaps have been reading blogs.
One problem with the current publication process arises from the overwhelming importance given to papers published in high-impact journals such as Science. Sadly, career advancement can depend more on where you publish than what you publish. Consequently, authors are so keen to publish in these select journals that they are willing to carry out extra, time-consuming experiments suggested by referees, even when the results could strengthen the conclusions only marginally. All too often, young scientists spend many months doing such "referees'experiments." Their time and effort would frequently be better spent trying to move their project forward rather than sideways. [emphasis added-DM]
It is surprising that so many referees make unnecessary demands, as they are authors themselves and know how it feels when the situation is reversed. Such demands are discouraging for young scientists and, cumulatively, slow the progress of science. Of course, peer review is critical for making sure that the authors' conclusions are sound, and some referees' experiments would substantially advance the story. But frequently, these would justify an additional paper. Science advances in stages, and no story is complete.
Wait, are they plagiarizing Your Humble Narrator? Damn. Or is it just possible that these thoughts just might occur to a whole bunch of scientists....?
Ok smart guys (because the evidence of your CVs suggests you are most assuredly smart guys), got any solutions?
Editors should insist that reviewers rigorously justify each new experiment that they request. They should also ask reviewers to estimate how much time and effort the experiment might require. With this information in hand, editors can more easily override referees'excessive demands. This requires confident, knowledgeable, and experienced editors, and it risks alienating referees, who are often hard to come by. Nonetheless, editors should be encouraged and empowered to perform this crucial task.
Indeed. Wonder how they feel about the whole professional editors / *working-scientist editors thing? Would more senior or more experience editors be more likely to smack down excessive reviewer demands? More likely to send back the decision letter with "and oh, btw, feel free to ignore reviewer #1's demands for new data" which I will note that I see frequently in the society-journal-level playing field.
*I hate the term "working scientist". Really I do. I think it is elitist and disrespects many people who think scientifically, indeed even more dispassionately scientifically than many so-called working scientists. (I am a lot more conflicted by my "barely hacking it as postdocs" crack than it might seem, btw)
UPDATE 07/08/08: I just noticed a post on the RabidReviewer over on the Professor Chaos blog.