We're approximately one year into the NIH policy that encouraged deposition of manuscripts into preprint servers. My perception is that the number of labs taking the time to do so is steadily increasing.
It is rather slow compared to what I would have expected, going by the grant applications I have reviewed in the past year.
Nevertheless, preprint deposition is getting popular enough that the secondary questions are worth discussing.
How many preprints are too many?
Meaning, is there a ratio of preprints to now-published-in-Journalofology preprints that is of concern?
It is sort of like the way I once viewed listing conference abstracts on a CV. It's all good if you can see a natural progression leading up to eventual publication of a paper. If there are a lot of conference presentations that never led to papers then this seems....worrisome?
So I've been thinking about how preprints may be similar. If one has a ton of preprints that never ever seen to get published, this may be an indication of certain traits. Bad traits. Inability to close type of traits.
So I have been thinking that one of the things guiding my preprint behavior is how many my lab has at a given time that have not advanced to publication yet. And maybe there are times when waiting to upload more preprints is advisable.
Thoughts, Dear Reader?
almost tenured PI raised some interesting questions in a comment:
So you want me to submit a preprint so I can get comments that I have to spend time responding to? No thanks. I spend enough time responding to the comments of reviewers from real journals. I can't imagine how much time I'd have to spend responding to comments that are public and immortalized online. No paper is perfect. How many comments does a paper need before it's acceptable for publication? Where does it end? I do not need more work when it already feels like the bar to publishing a paper keeps rising and rising.
Ok, so first off we need to recognize that the null hypothesis right now has to be that there will not be extensive substantial commentary on the preprints. PubMed Central shut down its commenting scheme for lack of use. Journals have been trying out various systems for years (a decade or more?) without general success. The fear that each posted pre-print will attract a host of journal-club style comments is probably not well supported.
But lets suppose your preprint does get some critical comment*. Are you obliged to respond?
This ties into the uncertainty and disagreement over when to submit the preprint. At what stage of the process do you post it? Looking at the offerings on bioRxiv, I think many folks are very close to my position. Namely, we are waiting to submit a preprint until it is ready to go out for peer review at a journal.
So any comments it gets are being made in parallel with review and I can choose to address or not when I get the original decision back and need to revise the manuscript. Would the comments then somehow contaminate the primary review? Would the reviewers of a revision see the comments on the pre-print and demand you address those as well as the regular peer comments? Would an Editor? For now I seriously doubt this is a problem.
So, while I think there may be many reasons for people not to want to post their manuscripts as pre-prints, I don't think the fear that this will be an extra dose of Reviewer #3 is well supported.
*I had one get a comment and we ended up including something in a revision to address it so, win-win.