Citing Preprints

In my career I have cited many non-peer-reviewed sources within my academic papers. Off the top of my head this has included:

  1. Government reports
  2. NGO reports
  3. Longitudinal studies
  4. Newspaper items
  5. Magazine articles
  6. Television programs
  7. Personal communications

I am aware of at least one journal that suggests that "personal communications" should be formatted in the reference list just like any other reference, instead of the usual parenthetical comment.

It is much, much less common now but it was not that long ago that I would run into a citation of a meeting abstract with some frequency.

The entire point of citation in a scientific paper is to guide the reader to an item from which they can draw their own conclusions and satisfy their own curiosity. One expects, without having to spell it out each and every time, that a citation of a show on ABC has a certain quality to it that is readily interpreted by the reader. Interpreted as different from a primary research report or a news item in the Washington Post.

Many fellow scientists also make a big deal out of their ability to suss out the quality of primary research reports merely by the place in which it was published. Maybe even by the lab that published it.

And yet.

Despite all of this, I have seen more than one reviewer objection to citing a preprint item that has been published in bioRxiv.

As if it is somehow misleading the reader.

How can all these above mentioned things be true, be such an expectation of reader engagement that we barely even mention it but whooooOOOAAAA!

All of a sudden the citation of a preprint is somehow unbelievably confusing to the reader and shouldn't be allowed.

I really love* the illogical minds of scientists at times.

26 responses so far

  • J. Brian Byrd says:

    DrugMonkey is correct. It is _scholarly_ to cite the course of ideas. There's not too much more to it.

    And yet...

    Not only do some reviewers object (apparently), but some journals do not permit citation of preprints in the manner other items are cited.

    An example observed in the wild:
    "We do not allow formal citation of preprints in the reference list, but they can be cited in the main text, for example: (BioRxiv: https://doi.org/10.1101/xxxxxx)."

  • Jonathan Badger says:

    Even if some of the things you mention seem a bit unusual (TV programs? Newspaper articles?), surely everyone has cited a book at some point. They generally aren't peer reviewed in the normal sense of the term.

  • Same with some reviews and commentary which appear to just be ordinary journal articles in a citation list.

  • qaz says:

    I think the real question is whether BioArxiv is seen as equivalent to citing a TV news show, equivalent to "personal communication", to a scientific book or book chapter (which tends to be edited, but for which peer review is often less), or to a "peer reviewed journal paper".

    I think everyone knows that BioArxiv is more than a personal communication because you *can* go back and read the article, as compared to "personal communication". On the other hand, it seems to be more than a TV show and yet less than full peer reviewed journal paper. I think JonathanBadger is on the right track and that it is like citing a book or a book chapter, both of which have a limited level of peer review. I mean we used to cite (I still do) SFN abstracts.

    I'm personally fine with citing BioArxiv just as I am with a book or a book chapter (and have done all three), but I don't think its so illogical for people to be concerned about it.

  • A. Tasso says:

    One time I got into a war with the JAMA editors because they wouldn't let me cite an NBER Working Paper (economics)

  • Ola says:

    Add NCT database entries to that list. Lawd only knows who's policing those things? But I cite then all the time - "there's a clinical trial looking at this, so someone's paying attention to the underlying basic science"

  • Eli Rabett says:

    arXiv papers are citer beware. Lots of nonsense so the issue is are you vouching for what you are citing or just leading the reader down the garden path

  • drugmonkey says:

    qaz- what does it matter what an item is “seen as”? Different readers will have their own view on anything that is cited. From media type to journal cachet to the lab head. Why are we supposed to accommodate the reader (impossible anyway, given diverse attitudes about quality) solely for bioRxiv?

  • drugmonkey says:

    JBB- I think I am considerably less bothered by demand to cite in text only compared with the implication it can’t be mentioned at all.

  • drugmonkey says:

    ER- that applies to everything that one cites.

  • qaz says:

    Every paper is a communication. This means that you need to understand your audience. (A popular science book should be written differently than a research journal article - both are hard to do well, but they are very different.) All I'm saying is that we as a culture need to figure out where preprints fit into the puzzle.

    Remember, for example, how neuroscience used to consider very clear differences between SFN abstracts and peer-reviewed journal articles, but both of which were regularly cited in many classic articles. Generally, I think people understood the difference because abstracts were called "abstracts".

    Certainly, I feel preprints should be cited. The question is what conclusions you should draw from them. I have cited many things, including the Rolling Stones and Shakespeare, in my publications. But it's very clear what I was drawing from them and how it was different from what was drawn from a peer-reviewed rat experiment.

  • drugmonkey says:

    What on earth is so confusing to you about a preprint?

  • Grumpy says:

    As a physicist this never comes up. But seems like an easy response: "We chose not to follow this suggestion. We believe citing archived documents that informed this work is the correct course of action."

  • smn says:

    I wonder if there are already publications citing comments from PubMedCommons or PubPeer. Even blog posts can sometimes be really worthy of citing in a scientific paper.

  • drugmonkey says:

    Didn’t they take down pubmed commons?

  • Ola says:

    @SMN - I've cited PubPeer in a grant proposal, as in "these ideas may at first glance seem contrary to pubished wisdom, but such wisdom has recently been the subject of intense online debate regarding reproducibility and data integrity (www.pubpeer.blah...) suggesting more work is needed in this area"

  • dushanbe says:

    I cite BioRXiv papers as sort of placeholder citations, knowing that, by the time peer-review of my manuscript is completed, those preprints will likely have post-print versions that can replace them in my citations. Not always, but usually.

  • drugmonkey says:

    Interesting strategy. Is the peer review process predictable enough in your field that most of the time you make it to acceptance before the citing manuscript goes to proof stage?

  • bacillus says:

    In recent years I've been taught that if it wasn't published in C/N/S then all other citations are like the proverbial tree in the empty forest

  • Morgan Price says:

    Although some wierdos like DM and myself think that the purpose of citations is to inform the reader, many biologists believe that the purpose of citations is to allocate prestige.

  • A Salty Scientist says:

    Certainly, I feel preprints should be cited. The question is what conclusions you should draw from them. I have cited many things, including the Rolling Stones and Shakespeare, in my publications. But it's very clear what I was drawing from them and how it was different from what was drawn from a peer-reviewed rat experiment.

    Not to pick on you personally, because I have seen this many times, but why should the stamp of peer review suddenly change how a manuscript is cited? Yes, the preprint may be complete and utter bullshit, but so are many a peer-reviewed paper. We should be critically evaluating the science of every manuscript we cite (if we do this for a cited pre-print, well hell, now it's been peer reviewed). I originally felt that NIH was being somewhat condescending on rigor and reproducibility, but I understand their point on codifying discussion of "the general strengths and weaknesses of the prior research being cited."

  • Jon Tennant says:

    I feel that the discussion here should be useful to readers of this blog: http://fossilsandshit.com/should-we-cite-preprints/

    Key point: Responsibility of the citation lies with the citer, and the source does not absolve one of the ability to think critically.

  • jmz4 says:

    Is the pushback more that they shouldn't be cited at all, or that they shouldn't be considered strong evidence to cite as support for a particular claim being made in the manuscript?

  • drugmonkey says:

    The position motivating my post is that preprints should not ever be cited.

  • jmz4 says:

    That seems ridiculous. In fact, if you're aware of evidence, or an assertion of theory, that is similar to what you are presenting and you *don't* cite it, I'd consider that to be unseemly; since you could easily be seen as trying to overstate the originality of your work.

  • drugmonkey says:

    Yes. Fighting the publication priority fight is definitely an issue when it comes to how people view pre-prints.

Leave a Reply