Why would you want to leave points on the table when your NIH Grant is reviewed[1]?

Jan 24 2012 Published by under Grant Review, Grantsmanship, NIH Careerism

It is well established that approximately 85.473% of the battle when it comes to NIH Grantsmithing is making it easy for the reviewers to grasp your point[2].

Really easy.

Many bloggers have described the fact that you need to think about your audience. The grant reviewer in my mind's eye is an overworked, grant-stressed, paper-decision-major-revision, lab disaster supervising PI who has finally cracked your grant open at 10 pm after putting his kids to bed, throwing some laundry in, running the dishes and making lunches for tomorrow's schoolday. . Plus, she may have an infant squirming and latching on poorly at the same time. Still licking wounds from the last disappointing summary statement from his or her own grant application.That's if you are lucky.

If you are unlucky she is getting serious about reviewing your grant while crammed in coach on the way to Bethesda for the actual meeting. And is planning on submitting the written critique the moment she hits the free wifi at the airport....

Then, should your application be so lucky as to be discussed, realize this, Dear Reader. The three assigned reviewers may have had the time, if they had the inclination, to pour over your application at leisure.

The rest of the panel? Not so much.

It is the rare reviewer who has read through the entire panel's worth of applications in detail. And if there appears to be a disagreement between the three assigned reviewers during discussion, these other reviewers have something on the order of 10-15 minutes to scan through your application to attempt to resolve matters in their own mind.

You want their job to be as easy as possible. This is why I hammer away at the creation of blank space, headings, clear figures and direct writing. You know this. What many people don't seem to realize is that your selection of citation style matters as well.

A lot.

Some folks seem to think that it doesn't matter[3] or that the NIH insists on numbered citation styles[4].

[As a bit of a sidebar, before I go too deep, there is absolutely nothing about grant review that is fixed in stone. So if you know people who always use the numbered citations and get funded[5], bully for them. But realize they got the award despite a lapse in grantsmithing[6], not because it is unimportant.]

Back to our story, it appears some of this confusion may be attributable to Endnote, as one authority claims that the Endnote style for "NIH" is numbered[7,8]. I have recently tested this on my Endnote X4 for Windows and I find this assertion to be untrue. My style for NIH (default installation) is Author, Date style, not numbered.

but still. If you have some version of Endnote that has a particular style for "NIH" don't you think you'd better check to see what the rules say? I mean, you could have a template grant from some idiot that uses Georgia font but you know to put it in the Arial, no? Or if your sample page is in 12pt font, you go and see what the rules actually say, right? So why would you take what some non-NIH entity says about the style for citation? I mean, you could ask the Twitts[8a]...but why not check the source [9].

Provide a bibliography of any references cited in the Project Narrative. Each reference must include the names of all authors (in the same sequence in which they appear in the publication), the article and journal title, book title, volume number, page numbers, and year of publication.

emphasis added- no reshuffling of "co-contribution" authors permitted!

also,

When citing articles that fall under the Public Access Policy, were authored or co-authored by the applicant and arose from NIH support, provide the NIH Manuscript Submission reference number (e.g., NIHMS97531) or the PubMed Central (PMC) reference number (e.g., PMCID234567) for each article. If the PMCID is not yet available because the Journal submits articles directly to PMC on behalf of their authors, indicate “PMC Journal – In Process.”

That's it though. No mention of a required citation style or of a bibliographic style, so long as the pertinent information is included.

Now, I know why you are tempted to use the gawdawful numbered citations. You think you are saving space and can squeeze yet more blabbedy-blah text in there to overhwhelm the reviewers.

Resist the urge.

It is hard to read academic text with numbered citations if you are actually using the citations to create the argument in your mind. Now maybe I am weird. But for me, especially in my subfields of interest, Name, Date for a paper citation is sufficient cueing to dredge up any number of details about the paper in question. With a mental image of the findings, interpretations, etc. Call me crazy, but your random schema for numbering the references doesn't do the job.

Even if I flip back and forth between my place in the text and the reference list, I've probably forgotten what the hell [254] refers to a paragraph later. There is a prayer that (Schmelmitz et al, 1968) will trigger me to remember the paper without any need to flip to the citation list at all. And I'll likely remember which one you mean, if I've flipped back to check the full citation, for longer than a few sentences.

Oh, and speaking of flipping to the Bibliography. Much easier to do with hardcopy grant applications. Much harder to do when reading the PDF on a computer. Maybe there are some fancy double pane views but I sure as heck don't use them...are you gambling your average reviewers knows some way to do this?

So numbered citations are a clearly inferior way to cite an academic work. In fact, when I am the boss of science, whomever came up with this or perpetuates it as a good way to cite papers is going to be in the second group up against the wall[10]. They make it hard for the reader to figure out what work you are citing. Let's face it, you aren't wasting the space to explain the prior finding, either, are you? You are using the mere citation itself in hopes that it makes the point for you! Why would you want to make it hard on the reviewers? It's only going to annoy them.

__
[1] Drugmonkey, 2012

[2]DrugMonkey, 2012

[3]Dr Zen, 2012

[4]Luminescer, 2012

[5]Luminescer, 2012

[6]Drugmonkey, 2010

[7]ericsuh, 2012

[8]ericsuh, 2012

[8a]Namnezia, 2012

[9]Application Guide for NIH and Other PHS Agencies

[10]Unless they are also a GlamourMag editor, then they go with their regular group[11].

[11]"First"

59 responses so far

  • odyssey says:

    This reviewers enthusiasm for the proposal was significantly dampened by the PI's incorrect citation of [11] and failure to cite [10]. [fixed-- DM]In addition, the seminal work of Schmelmitz et al. was published in 1986, not 1968.

  • pinus says:

    cPP and you brought this up before. I had previously used numbered style...I have changed to author name and date style. it is working well for me.

  • Zen Faulkes says:

    I agree with you that the (Author, year) references are better than numbered references IF the reader knows the literature as well as you do. In large, highly active research fields, this is a good bet. I agree that from what I have heard second hand (mostly from you, really) NIH grant review panels fit this bill.

    In cases where a reader does not have an intimate familiarity with the literature, I don’t think the advantage of one format over the other is as clear.

    http://neurodojo.blogspot.com/2011/01/names-versus-numbers-great-referencing.html

  • drugmonkey says:

    For your next trick, pinus, try to get all the BSD's to switch from numbered on your next mega-collaborative-MaxiGrant! I tried that and was summarily shot down....

  • drugmonkey says:

    Disagree Zen. If a citation is repeated more than once, you get some savings on the subsequent instances of the citation.

  • anon says:

    I prefer the numbered format. I find the long lists of names and dates used in references that interrupt the narrative to be distracting and at least as equally difficult to follow in the reference list on a computer screen. Yes, references are important, but I need to know the gist of a proposal more than I need to know whether the applicant is capable of citing key names in the field, or themselves.

    And this:

    "What many people don't seem to realize is that your selection of citation style matters as well. A lot"

    Where's the data? Your personal experience or do you have real success rates of applications that use the numbered format vs names & dates? Without this, I don't believe you. Sorry.

  • Dr Becca says:

    Horrors!! I used numbered style on my R21 proposal. Space reasons, yes, but also reading flow--I find it harder to follow papers in which half a paragraph is taken up with citation text. What were they saying again?

    Like you say, reviewers are likely skimming at least a little. Shouldn't you make it easy as possible to just read so they can get a sense of your argument, and if there's a questionable citation or something the reviewer feels the need to follow up on, then they can check the bibliography?

  • Anon2 says:

    I have to completely disagree. I've done a fair amount of reviewing (I'm reviewing for a Feb panel right now) and only once or twice have I had a grant so close to my own area that I would have recognized a publication from an author name and year alone. On the contrary (this might be field specific, in my field all citations in publications are numbered not named) I HATE reading through the publication list past Schmelmitz 1954, Schmelmitz 1960 and Schmelmitz 1963 to find Schmelmitz 1968 (which may or may not have been mis-cited). Fortunately almost all of the PIs whose grants I'm reviewing did it "right". Fittingly, one that I'm sitting on but just can't bring myself to write the review did it the other way. Hopefully I'm not too grumpy about the annoying named references when I do write the review!

  • drugmonkey says:

    Yes, references are important, but I need to know the gist of a proposal more than I need to know whether the applicant is capable of citing key names in the field, or themselves.

    You missed an important caveat in my post, namely the one about making and apprehending an evidence based argument. If you base your opinion on lyrical narrative flow that's fine, but you are doing humanities, not science.

    It is not about whether the applicant is "capable of citing key names". Not at all. It is about whether the rationale, significance, importance, approach, etc are based on the ongoing body of evidence or not.

  • drugmonkey says:

    but also reading flow--I find it harder to follow papers in which half a paragraph is taken up with citation text.

    The point I was trying to make over at Neurodojo before giving up in frustration with the stupid ass commenting system is simple. Attentional psych tells us that the brain is wonderfully good at paying selective attention to things and to adjusting to pay attention to different categories of things at need. Personally, I find that if I am reading for "flow" and essentially ignoring the citations, this is an automatic process. Those elements never even reach conscious awareness. Conversely, it is easy for me to operate in a mode that reads "with citations".

  • drugmonkey says:

    only once or twice have I had a grant so close to my own area that I would have recognized a publication from an author name and year alone.

    Another point I wanted to make over at Neurodojo, before being defeated by the fucked up, repugnant commenting system that Zen has in place, is that even under this scenario there are reader savings to be gained if any articles are cited more than once in the entire app. Not to mention, a savings if the reviewer finds herself re-reading* the application later.

    __
    *and I almost always find myself re-reading significant parts of applications multiple times.

  • anon says:

    "Personally, I find that if I am reading for "flow" and essentially ignoring the citations, this is an automatic process. Those elements never even reach conscious awareness. Conversely, it is easy for me to operate in a mode that reads "with citations"."

    Just because the citations are numbered within the text does not mean that they are being ignored. To me, it seems we are arguing about which is more important to follow and read: the text or the citations. I seem to think (as does Anon2) that text is easier to follow with numbered citations - cited basis for evidence included. It's not that hard to look up a number.

    Still, come up with success rate comparisons and I'll shut up.

  • drugmonkey says:

    It's not that hard to look up a number.

    wrong.

    Still, come up with success rate comparisons and I'll shut up.

    hahaaah, who suggested anyone should "shut up"?

  • Comrade PhysioProf says:

    Eschew Georgia at your peril! And you are 100 percent right about citation format. Name-date citations save a lot of back-and-forth with the reference list. And this "flow" notion is fucken ridiculous. Only rubes and n00bs "read" grant applications in complete sentences and paragraphs in the order they are laid down on the page. Experienced reviewers skim and scan around a grant in a highly non-linear fashion, looking for specific pieces of information and justification, like a detective at a crime scene.

  • drugmonkey says:

    Eschew Georgia at your peril!

    I'm sort of surprised you caught that, given your highly non-linear skimming and all...

  • proflikesubstance says:

    The main reasons I prefer the (name, year) approach are 1) it is immediately clear to the reader what body of lit you are citing and how recent the work is. 2) It is MUCH easier to highlight the work my group has done if I can cite our own papers clearly and obviously and (PLS et al. 2011, PLS Trainee et al. 2012) is a good way to send that message.

    Plus, I fucking hate having to look into the ref section to find what paper the author is going on about.

  • Yaakov says:

    When you are Boss of Science, DM, please consider mandating that citations be hyperlinked author-year style, with one of those hover-over pop-up windows to the title and abstract of the cited paper.

  • Namnezia says:

    @Yaakov, now you're talking!

  • Heavy says:

    Minor point but I actually prefer (Name Date) much better. The extra comma is superfluous.

    I've come around to Georgia actually based on previous posts. It is a solid font.

  • Comrade PhysioProf says:

    Dude, I'm skimming and scanning to identify strengths and weaknesses, not randomly.

  • Dr. Zeek says:

    Wait, I just went and changed the font in the grant we are writing from Georgia to Arial, but now, after reading the comments I am getting all confused. Which one is better?

  • Dr. Zeek says:

    Oh, and we are using the numbered reference style. The last time we wrote a NIH renewal (when the page limit was 25 pages, mind you) we (I) changed all the (Name date) references to numbers on the PI's orders and gained 1.25 pages of space. Grant was successful (and written in Arial).

  • drugmonkey says:

    It is well established that more grants are funded after being written in Arial than inGeorgia.

  • Namnezia says:

    Dr. Zeek, if you want to gain some space, as well as make your text more pleasing, turn hyphenation on. Also use 11 pt arial with line spacing set exactly to 12 pt.

  • Dr. Zeek says:

    Weird, but well, Arial it is. And Namnezia, the line spacing "trick" just gave me some more room (and looks a lot nicer). Thanks for the tips.

  • drugmonkey says:

    Dr. Zeek,

    to be serious for a minute, Comrade PhysioProf, resident font geek, insists that Georgia is the best of the available fonts. some blah-de-blah about readability.

    Me, I've never really had any opinion as a grant writer or as a reviewer. Just never noticed until CPP brought it up. My comments on the topic should be read as taking a swipe at font nerdery, not as having any real opinion as to the best font.

  • Namnezia says:

    Dr. Zeek - What about hyphenation, did that gain you space?

  • While it is probably true that more grants are funded with Arial than with Georgia, this is confounded by the fact that is almost certainly true that vastly more grants are submitted with Arial than with Georgia. What we need are the success rates for Georgia grants and Arial grants.

  • Dr. Zeek says:

    Sorry, DM, for completely derailing the comments on this post (although it has been a great procrastination tool this afternoon). In all seriousness, I was more curious than anything. Really, I like the "look" of Georgia. A lot. It seems quirky to me. But the last thing I want to do is piss any reviewer off. I cannot believe that I am seriously pondering which font to use. I think the late nights are finally getting to me.

    The way that I look at it is (and correct me if I am wrong, since I am really new at all of this) is that the most fundamental thing needed for a good grant is solid, thought-provoking science. That being said, if you irritate the reviewers with bad English, shitty layouts, fuzzy figures and all that assorted crap, then you will never get them to see the science. Unfair? Meh. Chalk it up to human nature. It would be like me showing up for my job interview last week wearing ripped jeans, a Led Zeppelin shirt and all my tattoos showing. I don't want to give the reviewers anything to bitch about besides the science.

    Namnezia- had the hyphenation on before I changed the line-spacing, but yeah, flipping back and forth between it being on and off I can see it frees up space. I am going fully justified as well. I like the clean line look. Again, all aesthetics, but still....

  • drugmonkey says:

    Since I don't see much difference one way or another, I was game for CPP's assertion about the superiority of Georgia. I have submitted grant applications in that font face. At least one has been funded so.....meh. seems perfectly valid as an option.

  • I'm curious, do the people that prefer author date citations work in fields where the PI is listed as the first author?

    My field always lists the PI as the last author, so even though I know what different labs are working on, I almost never know the name of the student/postdoc that did the work and therefore do not find author date citations useful at all.

  • I am going fully justified as well. I like the clean line look.

    Do not full-justify. It fuckes uppe the word spacing, and makes your text harder to read. It also makes it much harder for the reader to scan to the next line, because all the lines are the exact same length.

  • Namnezia says:

    Do not full-justify. It fuckes uppe the word spacing, and makes your text harder to read. It also makes it much harder for the reader to scan to the next line, because all the lines are the exact same length.

    Not if you hyphenate, then the spacing mostly fixes itself. I agree with your second assertion though.

  • Comrade PhysioProf says:

    If you're using a proper typesetting program, then the hyphenating helps the wordspacing. But it's still better to not justify. If you're usind MS Word, hyphenation helps much less.

  • eli rabett says:

    IT'S THE FUCKING TITLE that tells you what you need to know!!! ACS/APS/AIP journal reference styles leave you up the creek w/o a paddle

  • drugmonkey says:

    EP-

    Most of the papers I'm citing are Senior-Author-Last. Again, this is not about generalities like what lab did what. It is about serving as a reminder cue for specific individual results.

  • drugmonkey says:

    Er-
    I probably couldn't even hazard a guess at the titles for any of my most cited papers...including my own.

  • Dr. Zeek says:

    Ah shit. I don't want to do this anymore.

  • MediumPriority4Life says:

    I just grabbed a fist full of NSF cash with numbered citations and Palantino Linotype font. Perhaps this font type offsets the use of numbered references.

  • CoR says:

    If I could submit in Georgia, I would. It is strangely easy to read, doesn't look as machinist as Arial, and less stuffy than TNR.

  • drugmonkey says:

    "stuffy"??

  • Hermitage says:

    BLEH, you all have me opening mfing Word to 'quick sly fox jumped over the lazy brown dog' all these goddamn fonts.

    Fuck, Georgia looks sexy.

    Goddamnit I'm agreeing with CPP. FML.

  • Arno says:

    In the pdf, the citation reference should be a link to the bibliography entry anyway, so that you can simply click it, look it up, and use the "back"-button to return to the line you're currently reading. Of course, this can still be disruptive, but does allow to access the information quickly.

    The same goes for references like "in Section 4", etc.

  • Eli Rabett says:

    What Arno said, and Eli doesn't give fuck what the fuck you fucking remember about your fucking shit papers, HE wants to know the titles so he can have a fucking clue about what the fuck you are babbling about.

  • arrzey says:

    I just got a boatload of grants (NIH) to review. The commitment is always to reviewing quality, etc. However, these piss me off (roughly in order of pissed-ness):

    Bad writing - esp. passive voice. Let Hemingway be your guide.
    Typos
    Cheating on margins, no space at all btwn para. White space is your friend.
    Tiny pictures that I can't read
    Numbered citations. its also about assessing what cites are missing.
    Full-justify (spacing is weird)

    Font doesn't matter (to me).

    But people who worry about "getting more room" miss the point - the limits are there not just for me, the reviewer, but to give YOU the writer a sense of what is needed. If you need to push for more room, you are saying too much - don't edit by word, take out sentences or paragraphs.

  • In the pdf, the citation reference should be a link to the bibliography entry anyway, so that you can simply click it, look it up, and use the "back"-button to return to the line you're currently reading.

    The fact that it "should" be this way doesn't mean jacke fucken diddly shitte, because you have to construct your application with the research plan and bibliography as separate PDFs. Whether you could make the research plan PDF such that a mouseover or click of the citation would give you a pop-up or whatever of the bibliographic information is a different question. I do know that grant reviewers are urged not to click on any links in application PDFs, because that could allow their IP addresses to possibly be gleaned by an applicant.

    Fuck, Georgia looks sexy.

    The reason why you should use Georgia rather than Arial for a grant application has fucke all to do with how aesthetically pleasing Georgia is (although it is a nice typeface; Matthew Carter is a very talented type designer). The reason is that it is has serifs and is therefore much more legible and readable when set in sentences and paragraphs of long stretches of body text than a sans serif typeface like Arial. You *should* use Arial (or another allowable sans serif typeface) in your applications in the figure legends as a means of setting them off nicely from the body text.

    If we were allowed to choose any typefaces whatsoever in NIH grant applications, I sure as fucke wouldn't be using either Georgia or Arial.

  • proflikesubstance says:

    What font would you use, CPP? Oh wait, we probably haven't heard of it.
    /CPPfonthipster

  • drugmonkey says:

    Exactly, arrzey, exactly.

  • Neurosearcher says:

    The Grant Reviewers Formatting Friend

    I have listened to this debate from every career angle I could have imagined. From those tenured to undergraduates, to Prof. Grey Fox to reviewers, and from those with R01s and those praying their startup funds will last until one comes through.

    @arrzey: I almost removed the next paragraph for you, but I hope your boatload of grants is less Old Man and the Sea, and more a Farewell to Arms.

    As a 36 year old undergraduate, I was especially elated to stumble across this post. I have had amazing fortune with grant funding. While I wholeheartedly support the concept that someone actually took the time to read all twelve pages of my rambling diatribe of a proposal, however improbable…I am much more inclined to assert that aesthetics played an equal, if not a greater role in the multiple fellowships that the work has garnered (They say merit, but what do reviewers know anyway?).

    Thus, I would like to thank you all for helping me choose my final project for Matlab as a parting gift to the language, so to speak. I am thinking of calling the app "The Grant Reviewers Formatting Friend". Please feel free to assist with the naming. I feel that all 47 comments were just perfect ideas. . After all, it would be unwise to hand over my best ideas for stimuli to a tenured professor right?

    @DM, I will include a Drugmonkey Style for your convenience DM, and may request permission to include some of these great quotes as random commands.

    @CPP, I’ll make it open source by fall for kicks, and I’ll integrate it with better languages for independence and security (I’ll make certain to place a warning in my headers with the hyperlink on my proposals just for you).

    Lastly, @odyssey, I hope that Wolf Schimelmitz and his attorney are most fondly remembered for their patent application which secured Wolf’s invention of the Self-Closing Wallet in 1939 (Schmelmitz et al, 1941).

    Thanks Again!

  • becca says:

    Even with Endnote, the numbered style is more work to write. I like writing in (author, date) much better. Although maybe that's just from reading enough Drugmonkey that I intentionally created the mental framework of (author, date) to remember papers by?

    That said, if you are going to cite 30 sources for a particular fact, (author, date) sucks donkey balls. Though I'm not sure if citing 30 sources has any actual purpose other than to wave your "scholarship" around.

    I suppose the advantage of numbered is that if the *person* who said it is particularly worth emphasizing, you can put "As Drugmonkey et al found...". In contrast, for many statements it is enough to know that there are data to support the assertion (e.g. "the role of the blangersplort was originally identified in controlling cell growth in tumors [11!whothefuckcares]. Recently, Drugmonkey et al have identified a role for the blangerplort in the prefrontal cortext [39]"). There's nothing stopping you who insist (or have PIs that insist) on using numbered format to use names for emphasis. And you should.

  • Dave Bridges says:

    but i need numbered citations. What i have to say is far too important for brevity, whitespace, legible figures or wide margins!

  • drugmonkey says:

    If you are citing 30 sources per point you are doing it wrong. More than 3 in a grant application better have a damn good reason for the rare exception.

  • If you are citing 30 sources per point you are doing it wrong. More than 3 in a grant application better have a damn good reason for the rare exception.

    This is sort of true, but it is also important to make sure that you cite *everyfuckenthing* in your field that is relevant. Because you can't know the exact review panel composition when you submit your application, and all it takes is one ad hoc fuckebagge who gets all pissed off because you didn't cite his fucken turd in the Journal of Left Big Toe Vascular Cell And Molecular Biochemistry, thus demonstrating that the PI "is not aware of key published findings in the field".

  • drugmonkey says:

    I agree that in the course of the total application it is wise to try to include papers from as many of the potentially related laboratories as possible....somewhere. but doing this by having 10 citations at the end of every sentence is overkill to the point of making your text ridiculous. Yes, even with numbered citations!

    If there are that many players involved and that many directly related papers, maybe you need to streamline your grant proposal a little bit. focus more.

  • drugmonkey says:

    I like aarzey's comment

    But people who worry about "getting more room" miss the point - the limits are there not just for me, the reviewer, but to give YOU the writer a sense of what is needed. If you need to push for more room, you are saying too much - don't edit by word, take out sentences or paragraphs.

    more and more as I think about it.

    I wish more grantwriters would realize that less is more.

    There's nothing stopping you who insist (or have PIs that insist) on using numbered format to use names for emphasis.

    This seems a decent compromise position, particularly when you are working with the dumkopfs who insist on numbered citations

  • Gerty-Z says:

    WHY would anyone full-justify?? It makes your text look like a giant wall of words. blech. Hyphenation can be good but you have to be careful. Word will do some fucked up things with hyphens if you aren't careful.

  • Dr. Zeek says:

    WHY would anyone full-justify?? It makes your text look like a giant wall of words. blech.

    I know, but the lines! The margins have a straight line...I like things with crisp, clean lines. Straight lines. None of this wiggly stuff. It just...I don't know, maybe it is my OCD tendencies.

    I like boxes. I like putting things in boxes.

  • Isis the Scientist says:

    I think the NIH should allow cocksure as an acceptable font.

    Go ahead and google it. It is a bit of typographical genius.

  • Dr. Zeek says:

    I am so tempted to send the rough draft of the proposal to my cohorts in cocksure font. True stroke of genius.

Leave a Reply