Writedit and the Bergster publish a book on NIH grant strategy

Nov 25 2013 Published by under Careerism, NIH, NIH Careerism, NIH funding

I cannot wait until my copy of this book arrives.

How the NIH Can Help You Get Funded An Insider's Guide to Grant Strategy
Michelle L. Kienholz and Jeremy M. Berg
Oxford University Press
ISBN: 9780199989645

Kienholz is, of course, our longstanding blog friend writedit

Michelle Kienholz has partnered with scientists, clinicians, and public health researchers from all disciplines at dozens of universities to develop grant applications for almost every federal agency, including most grant mechanisms for each of the institutes and centers at the NIH. She volunteers her knowledge and experience on her popular blog, Medical Writing, Editing and Grantsmanship (as writedit), through which she has learned the most common and vexing concerns of researchers who interact with the NIH and how best to foster a partnership between investigators and NIH personnel.

and Jeremy Berg, PhD who

joined the University of Pittsburgh in June 2011 as the associate senior vice chancellor for science strategy and planning in the health sciences and a faculty member in the Department of Computational and Systems Biology. Prior, Dr. Berg became director of the National Institute of General Medical Sciences (NIGMS) at the National Institutes of Health (NIH) in November 2003.

is, well, familiar to our readership as the prior head of NIGMS, blogger and provider of much grant-funding data.

Berg recently twitted a teaser graph from the book which finally coughs up a comparison of funding policy for several ICs. According to the Twitter comment it refers to FY 2012 trends.
KienholzBerg-Funding Curves-2012

Nine ICs were willing to cough up data on the percentage of grants funded by the percentile they achieved at study section review. Lower is better, in NIH parlance so you can see that almost everything in the top 7-10% is getting funded across the ICs. Once you get to the top 35th percentile, your chances of funding are almost (but not quite) nil.

What is of best interest here is that we can finally see contrasting IC styles. There are 28 total ICs so this is just a subset but the NCI is huge and the NIMH is no slouch either. The topic domains range from cancer to the brain to metabolic to infectious disease to basic science so there is some breadth there too. I like this as a representative picture although we must always remain suspicious that those who chose not to send the authors their data might have done so for.....reasons.

Anyhow, what jumps out at me first is that NINDS has the sharpest dropoff past their apparent payline. If I am not mistaken, this is precisely the IC that is rumoured to assert their strictness with respect to payline. Strictness involves two choice points of the Program Staff. Whether to skip over grants that fall below (better than) the payline and whether to pick up grants that fall above the payline. Although I do seem to spot some skips under the payline for NINDS, NIA and NIAMS do not appear to have a similar skips. All the other graphs do appear to show skipping behavior. On the other side of "strict payline" behavior, clearly NINDS has funded some grants above their readily apparent payline. It's just that the distribution drops off much more steeply for them.

I note that NIGMS, NIDA and NIAID seem to have the smoothest curves of pickups away from the apparent payline. The reason I say "apparent" payline is that some institutes, of which NIDA and NIMH are two iirc, insist they do not have a payline. What I have asserted since I noticed Berg's posting of NIGMS' funding decisions is that published payline or not, ICs follow roughly the same behavior. These charts demonstrate that. All that differs is the slope of the curve defining above-apparent-payline pickups.

I'm hoping I'll have more to discuss once my copy of the book arrives.

13 responses so far

  • Grumble says:

    I love how NIA and NINDS have a little dip right after the apparent payline, and then there's a slight bump in funding at scores slightly after (worse than) the dip. It's almost like it's spite: "you just missed the payline, so we aren't going to even consider you. Now, this other grant that scored 10 percentage points worse than the payline? We don't feel as much as if we're just awarding that one out of sympathy, so we'll give it to them instead."

    Yet more evidence for the random bullshit nature of NIH funding.

  • drugmonkey says:

    This could also be an ESI bump? maybe?

    there is a hint of how this sort of effect could happen in the NCI FY11 data. although it would depend on hard-lining the established investigators to produce a discernible bump in the overall trend.

  • sop scientist says:

    Thanks for the heads up. Just ordered a copy!

  • Ola says:

    Are those graphs for all grants, or only R01s? I would guess the former. If they include ESI/NI, as well as R21s, P01s, RO3s, and a bunch of other mechanisms, then it's not had to imagine a curve like that.

    I'm more troubled by the curviness at the bottom end than at the top. At the top, there are both good and bad reasons why a top scoring grant might get cut. At the bottom, I can only think of bad reasons why a crappy grant would get funded.

  • eeke says:

    Ola, what is depicted as the "bottom" of these charts is not the worst of all grants - just the ones that scored < 35. Given the low chance of funding to begin with, it is unlikely that any grant that scores between 20-25 percentile is a crappy grant.

  • Jeremy Berg says:

    Ola: The graphs are just for R01s.
    At least for NIGMS, the large majority of the grants at the "bottom" of the funding curve are from ESIs, primarily A1s, as DM suggested. Furthermore, I agree with eeke, these are not "crappy" applications. In fact, they are probably not different statistically from the grants in the middle of the curves.

  • dsks says:

    eeke "Given the low chance of funding to begin with, it is unlikely that any grant that scores between 20-25 percentile is a crappy grant."

    Right. I'm reminded of that graph DM reblogged (from Berg I think) showing a pretty weak/non existent correlation between a grant's percentile placement (in the range <30%ile range) and the later impact of the proposed work in terms of number and quality of publications.

    From the objective perspective of Teh Science, this is good news, because it means that there's a lot of high quality proposals coming in and that thus no shortage of great ideas to spend money on... but from the subjective point of view of the PIs competing for that money it sucks a$$.

  • leopold says:

    I've always been curious about writedit having followed the blog for several years. And admired her stamina during the shutdown, as she answered numerous questions, while the NIH went dark. I will happily buy the book. If the book is even a fraction as useful as the blog, then the book will be worthwhile.

  • meshugena313 says:

    my copy arrived from amazon today, will start reading soon. Although how this will help with grants that are reviewed at 26% with the only critique being that the proposal "is too highly focused" and that aim 3 "might be difficult", i don't know. This is a the second time a PO told me basically to rewrite an A1 to a new A0 as the same proposal but different words, and that he's only had CSR flag such a rewrite a couple of times. WTF?

    Too many cats chasing too few mice...

  • drugmonkey says:

    This is a the second time a PO told me basically to rewrite an A1 to a new A0 as the same proposal but different words, and that he's only had CSR flag such a rewrite a couple of times. WTF?

    This is exactly why I said that the deletion of the A2 opportunity was going to do absolutely NOTHING, and the funded A0s they were brandishing were going to be highly contaminated with what are really A2, A3, A4 submissions.

    Francis Collins and Sally Rockey get to brandish a shorter time-to-funding from initial proposal but this is not what is happening in reality.

  • meshugena313 says:

    DM - well my "A2" will certainly take a lot more effort to draft than a simple revision. So a big freakin waste of time, and will get in the way of the stack of 4 manuscripts I'm trying to get out the door, plus the grad course I run. But I'll diligently read writedit and Berg's book after the kids are asleep and I take out the garbage, perhaps there is some magic fairy dust in there.

  • meshugena313 says:

    Seriously, there might be an argument to eliminating resubmissions of A1s that were triaged on both submissions, but beyond that it really is ridiculous, with the lack of real difference in the top 35% or so (as Berg had shown and in this book, I think0.

    I was pretty shitty at musical chairs when I was a kid, but I'm still standing now... or are we all tribute, like in the hunger games?

    Time for more scotch.

  • […] this cat is partly out of the bag thanks to DrugMonkey, I am pleased to announce that I finally committed to page some of what I have learned in working […]

Leave a Reply