Preprint Ratio

Jun 30 2018 Published by under Preprints

We're approximately one year into the NIH policy that encouraged deposition of manuscripts into preprint servers. My perception is that the number of labs taking the time to do so is steadily increasing.

It is rather slow compared to what I would have expected, going by the grant applications I have reviewed in the past year.

Nevertheless, preprint deposition is getting popular enough that the secondary questions are worth discussing.

How many preprints are too many?

Meaning, is there a ratio of preprints to now-published-in-Journalofology preprints that is of concern?

It is sort of like the way I once viewed listing conference abstracts on a CV. It's all good if you can see a natural progression leading up to eventual publication of a paper. If there are a lot of conference presentations that never led to papers then this seems....worrisome?

So I've been thinking about how preprints may be similar. If one has a ton of preprints that never ever seen to get published, this may be an indication of certain traits. Bad traits. Inability to close type of traits.

So I have been thinking that one of the things guiding my preprint behavior is how many my lab has at a given time that have not advanced to publication yet. And maybe there are times when waiting to upload more preprints is advisable.

Thoughts, Dear Reader?

16 responses so far

  • Almost tenured PI says:

    If other PIs are like me, I'm not sure the preprint trend will continue. We submitted our first preprint a few months ago the same day we submitted to a standard journal. I quickly got an email from the editor of a respected society journal asking me to submit my preprint to their journal. At first I was flattered, and then realized that they send this to everyone who submits a preprint in that topic area. Our preprint didn't get any comments, though it had a lot of hits in terms of views and downloads. The article was accepted at a journal with one round of reviews and published the same day it was accepted. In all, it took 2 and a half months to get it published. We really didn't gain anything from the preprint, except more work to do in uploading the preprint and the revised preprint. And furthermore, the preprint screws up my google scholar citation counts because the papers I self-cited are counted twice (from the preprint and from the real article), and I feel like that's cheating.

  • Gob of Goo says:

    In terms of conference abstracts, at least in my field (I'm not in biomedical science, your mileage may vary) and at my institution (not a R1, a minority serving institution) a lot of times this is the goal for undergrads or first-semester Master's students (with no prior background in research) taking on their first short-term projects to "get their feet wet" and "learn how to do science"- starting out with baby steps of training. Often, these are projects, though with legitimate enough motivations and aims, to aim for having an abstract and a presentation at a conference, but not with the depth, motivation (or funding) to hope to turn into a full-fledged journal publication. Thus, is conference-abstract-to-publication ratio really a worthy metric to be considering?

  • Jonathan Badger says:

    Well, do you cite both the preprint and the published article in the same article? If it is just the preprint and then in later articles the journal version, that's totally fair and not double dipping. Likewise for people citing you.

  • DrugMonkey says:

    For those who are preprinting to establish priority, I suspect citing the preprint will be important.

  • Rheophile says:

    I suspect that the equilibrium with preprints will be preprint = paper either submitted or nearly so, i.e. when you would have circulated a draft for comment. So if a lab is submitting a large number of papers that *never* end up in print, either they have many unreliable papers or they only publish their data if it ends up in a glam story. I think that's probably worth knowing about. It could also show the long review lag pre-glam, of course. But I suspect those labs will also not preprint.

  • Almost tenured PI says:

    Jonathan Badger, let me explain a little better. In our article we cite several of our own past papers. Let's say it's 5 self-citations. Google scholar thus gives me 5 citations from the preprint and then another 5 now that the article is published in a journal. This does seem like double dipping, and could be a way to game the h-index calculation.

  • Almost tenured PI says:

    While I'm focusing on myself in the comment above, I should also mention that every other author we cite is also getting two citations from us in their google scholar profiles (one from the preprint and one from our published article).

  • drugmonkey says:

    This is another reason why Google Scholar is bs for citation metrics.

  • Grumpy says:

    Almost tenured,
    In my experience, Google scholar always manages to eventually fix the double dipping issue and recognize that the arxiv and published versions are the same. One can even merge the two directly in Google scholar I think (but I haven't done that in a while).

    I'd bet that uploading to arxiv will prove, in the longrun, to not have been a waste of your time. Now there is always a quick standard way for ppl to get open access to your paper.

    DM,
    Every single paper I've posted on arxiv was published within 2 yrs. I would guess the norm in experimental physical sciences is >90% of all papers that are posted to arxiv are eventually published. There are some theory fields where that fraction is much lower, but those folks operate on a different system of recognition.

    Actually there are several manuscripts I wish I had posted to arxiv that were never published. These include my PhD thesis and a write-up I did back in grad school that I lost interest in publishing but had some useful info in it. Years later, my thesis has been cited a few dozen times..I wonder how many more cites it would have if I had posted it on arxiv?

    Now that biomed folks are finally joining the preprint standard, I wonder if new ideas/usages for the arxiv will become standard. annotated data or code dumps? or maybe methods and protocols essays?

    It's funny to see you all struggle with this preprint business since it is such a standard, well-liked, and not-at-all controversial thing to do in physics.

  • Ola says:

    Agree this could be a problem on the biosketch if the ratio was extreme, and of course the early career investigators are gonna be far more vulnerable to the StockCritique "does not have a strong track record of turning pre-prints into actual papers". But hey, you know how you fix that? By turning pre-prints into actual papers! Once you've done it a few times, you get to say "we have a strong track record of turning pre-prints into actual papers!"

    The trick (which I suspect most people do) is to not submit a pre-print until the story is ready for regular journal submission. If you're sending half-finished stuff onto BioRxiv that you would not consider ready for a regular journal, that's just asking for trouble.

    And while we're being honest, I've found that "well the preprint is already out there so get the hell on with it!" can be a useful motivational tool for the first author to do the requested experiments and corrections ASAP when the paper comes back from review. Once it's out there, the clock is ticking for scoop-ville, which tends to light a fire under the first author's ass better than anything else I've tried - not that I have a whole arsenal of such tools 😉

  • Drugmonkey says:

    it is such a standard, well-liked, and not-at-all controversial thing to do in physics.

    and was it always thus, right from the initial invention?

  • drugmonkey says:

    Once you've done it a few times, you get to say "we have a strong track record of turning pre-prints into actual papers!"

    Good point.

    The trick (which I suspect most people do) is to not submit a pre-print until the story is ready for regular journal submission. If you're sending half-finished stuff onto BioRxiv that you would not consider ready for a regular journal, that's just asking for trouble.

    This is the approach I have been taking so far. The drawback is that it sort of undercuts one of the allegedly more positive, forward-thinking reasons for preprints, i.e. getting feedback as you go along. Not sure how I feel about that yet.

    the clock is ticking for scoop-ville
    The Glam hounds pushing for this seem to feel as though the establishment of pre-print priority will help them continue their endless-revisions-and-more-experiments game they play amongst themselves while advantaging their postdocs against those that won't preep.

  • Curiosity says:

    What actually is the preprint policy (sorry if I've been asleep and the NIH instructions can be opaque)? For a renewal app, do you report them in the Publication List, the Biosketch, the Progress Report? All or a subset of these? If they are not part of the previous project but inform the renewal?

  • biochembelle says:

    This has me wondering now... What are applicants counting as "interim research products" with regard to preprints?

    Some preprints in biorxiv have no new data but are rather re-analyses/critiques to refute published claims (in essence, a more rapid & user-controlled version of a letter to the editor). There may be no hope or intent of publishing in a peer-reviewed journal. But it might supplement rationale for ongoing/proposed work. It's not clear in my reading of NIH guidance on interim research products whether that's something that would "count".

  • AcademicLurker says:

    and was it always thus, right from the initial invention?

    From the beginning. When the arxiv started, it was mainly used by people working HEP theory. In this community there was already a longstanding tradition of circulating preprints by mail, so the arxiv just made it easier for people to do what they were doing already.

Leave a Reply