The NIGMS Director Jeremy Berg is da CandyMan!

Sep 14 2010 Published by under Grant Review, NIH, NIH Budgets and Economics

He just keeps that data crack flowing for the grant geeks in the audience. Today's analysis breaks down the NIGMS funded and unfunded applications by Investigator status. Early Stage Investigator and New Investigator (collectively never had a major R01 award before, distinguished by ESI's having to be no more than 10 yrs out from terminal degree) applications are identified as are Type 1 (new submissions) and Type 2 (competing continuation applications for established investigators.

A plot of the overall impact score versus the percentile for 655 NIGMS R01 applications reviewed during the January 2010 Council round. Solid symbols show applications for which awards have been made and open symbols show applications for which awards have not been made. Red circles indicate early stage investigators, blue squares indicate new investigators who are not early stage investigators and black diamonds indicate established investigators.

Yowsa, data galore!

Go over to his post, he has a cumulative percentile breakdown that is pretty fascinating too.

Director Berg is once again to be congratulated for really leading the way at the NIH. Transparency of this type cannot help but accrue to the benefit of Institutes that are making funding decisions in a reasonable way. Sure, there are going to be cases to explain such as the poor sucker with the fantastic score that didn't get funded. But these data point to the relative consistency with which their apparent exceptional funding decisions (aka "pickups") are made. These types of plots (especially the cumulative one he included but I've not poached here) also help us to assess the degree to which study sections are disadvantaging ESI or NI apps in their scoring and how so. Are the truly excellent top 10% getting their due but then things break down in the grey zone? Are study sections assigning good ESI/PI apps to the scored but decidedly not-good ranges of 25-35th%ile land? Do these relationships change over time as the NIH commitment to ESI/NI investigators takes hold?

17 responses so far

  • becca says:

    I met Berg a couple of years back. Now imagining him as Gene Wilder/Willy Wonka (http://vimeo.com/4571649). Can't. Stop. Giggling.

    Also, feel really bad for poor 3% square.

  • Physician Scientst says:

    Re: 3% new investigator. I was on study section with a grant from an elite senior investigator (foreign who came to the states) and checked the "new investigator" box. While technically true, this was not the intended recipient. I suspect something like that was happening here. (or their mouse/human sections were woefully inadequate and lacking protections).

  • drugmonkey says:

    What amazes me is that it allegedly took so long for NIH to catch on that senior, established folks were sucking up all that schweet NI money...

    Creation of the ESI was ok but punished a class of folks they didn't need to punish. Why didn't they just make NI status depend on less than 4-6 years in faculty appt? Would've been fairer IMO.

  • Why didn’t they just make NI status depend on less than 4-6 years in faculty appt?

    Because that wouldn't have been as effective at reversing the trend of increasing average age of first R01. Of course, it does this by helping to ensure that some poor old fuckers *never* get an R01, and thus don't contribute to the rising average.

  • drugmonkey says:

    I know, I know. When are they going to start bragging on that trendline finally turning down while giving a big ol' face kick to the unlucky NI's too old for ESI...

  • Sean says:

    Off-topic: In case you do not already know, your site move makes a huge number of your links to older posts inoperable.

  • DrugMonkey says:

    If you mean links to scientopia.org, Sean, yes we are aware of this problem. Sorry about that.

    Something is going wrong with the redirects ever since we switched hosts recently. We're trying to hold tight in hopes this problem can be resolved rather than telling people to change their feeds and re-editing all of our posts.

    in the mean time if you substitute scientopia.org where the link is for scientopia.org you can find the older posts.

  • Odyssey says:

    I feel bad for the dude out at the 70th percentile. If he sees that plot he'll know he's the worst of the worst.

    Maybe that's not a bad thing...

  • drugmonkey says:

    actually you raise a question, Odyssey. Should be about 50-60% of these suckers triaged. Not clear if Berg is including all discussed (voted Overall Impact Score) with the not-discussed (would have to just be the average preliminary Impact Score from the assigned reviewers) or just the discussed applications. If the latter, why in the heck are so many applications getting discussed that wind up with such bad scores?

  • The triaged grants are not shown, as per Jeremy's response to my query at his post about the proportions of triaged grants for the four classes of applicantions. The reason that the percentiles go up to the level they do is because study sections tend to review a *minimum* of 50% of applications in rank order and frequently up to near 60%, since you have to review *all* of the applications with the same preliminary average impact score, and then because you also add in a few that are "pulled up" at the discretion of any single panel member. I only count 13 applications above 60 %ile, which is less than 2% of the total. So this looks unremarkable.

  • drugmonkey says:

    where the heck is whimple to insist there must be shenanigans of people pulling up their buddy's clearly triage-ready app for discussion?

  • Proteus says:

    What bugs me the most is to see unprepared members of the study section voting *blindly* for proposals submitted by recognizable names from recognizable institutions. :-(

  • drugmonkey says:

    why is it "blindly" if they have just listened to the three assigned reviewers give their spiel?

  • Dude, obviously Proteus is a mind-reader and knows exactly what scores all the reviewers write on their secret score sheets and what they are thinking when they write them.

  • almudena says:

    Thank you Dr Berg for the candies. Keep the good work and congratulations for the Award.

  • [...] only IC that, to my knowledge, made their funding data available was the NIGMS. We grant geeks were big fans, even those of us who don't seek funding from that particular Institute of the [...]

  • […] well, familiar to our readership as the prior head of NIGMS, blogger and provider of much grant-funding […]

Leave a Reply


five − 4 =