Repost: Radical changes in NIH funded research careers

I'm dragging up a post from the archives of the old DrugMonkey blog today. It was originally published July 6, 2007 and outlined a few thoughts I had on changes that might improve NIH funded careers in the biomedical sciences. Enjoy.

A recent post at Galactic Interactions proposes what seem to be substantial changes in the current NSF- and NASA-funded astronomy research career. I had a few observations in the comments trying to understand just what was being proposed and how this might apply to the NIH-funded biomedical research career. In some exasperation with my not-gettin'-it comments, Rob Knop suggests that I'm a hidebound traditionalist. I don't think so. I'm becoming increasingly convinced that the way astronomy careers work and the way biomed research careers work these days are sufficiently different that the specific problems and solutions may not look very similar. However, Rob raises an excellent challenge which I might view as "okay complainers, don't like the systems determining your career? how would you change it?".
Fascinatingly, while thinking about my answer and creating the above text, the NIH launched a similar query ! (as usual, Agent Writedit is on the case.)
So, how would YHN Go Big?
[ open thread ]
I discussed elsewhere the proposition that the MERIT extension (a 5 yr noncompeting turned into a 10 yr noncompeting interval) should be applied to those who need it, namely that rare New Investigator who does manage to get a grant. I still think this is a great idea.
Create career awards (not fellowships because of the way institutions use this to screw fellows out of the usual employment benefits) for that category of doctoral research scientist who is happy to labor away in someone else's lab without being a PI. These people already exist, in great numbers and often work through to retirement in nebulous job categories. Let's recognize that these people are an essential fuel for the NIH engine. It can be on the 5 yr cycle so that productivity is assessed and individuals are accountable to produce. This will create a great deal of independence in these individuals so that they are not beholden to one PI. Think of the side bennies on scientific fraud!
Ditch ALL F32 fellowships in favor of the K99/R00 transition mechanisms. Remember, the "/" translates to "get a job offer" so this does not equal "free jobs for any lame postdoc".
Cap the number of awards or absolute grant dollars of any given PI. Let's get serious. There is no miracle of ever-expanding capability of one person, no matter how brilliant, to really do justice to six R01 equivalent projects with <10% effort each because of all the other obligations. The only way the BigWigs pull this off is because there is a team of highly competent scientists under them doing the actual work of running the project. Not to mention coming up with many (most?) of the really brilliant ideas. Why not let some of those people be PI to get appropriate credit for their work? What would be lost? Not to mention cost/benefit gains. These bigwigs are over NIH salary cap which is some $180,000 per year. Are you telling me two lowly assistant/associate professors for this cost isn't a better value? The big project and all the alleged "synergistic value" and "economy of scale" values could still be easily obtained, the BigWig would just have to share the credit (and maybe change to a more collaborative style, another GoodThing).
Update 07/10/07:
One of the major problems with NIH grant review is the churning of revised applications. There is a huge bias for revised grants, check CRISP for new grants in any given year, gate by your favorite study section and look for the proportion of -01, -01A1 and -01A2 to verify this. This becomes more intense as the funding lines shrink and the revision process becomes a holding pattern for highly meritorious grants waiting to land on the runway. This costs a lot of money and opportunity as scientists prepare these applications and other scientists review them. And I should be clear that in a very large number of cases, the critique/revise process results in very little change to the conduct of the eventual science (even if the PI is induced to say the "right" thing in the application).
To address the specific case of low funding lines, I think the NIH ICs should continue to "intend to fund" up to their historical funding line (pick a number but it will be something 20-25%ile). Then, in a given round they should fund some fraction of the current funding line from the new grants and fund some additional fraction from the "holdovers" from the previous round up to the historical funding line. For argument let's say split the difference. 5%ile for new proposals and 5%ile for holdovers from the prior round in a 10%ile round. Now the key is that there is going to be considerable overlap between the just-missed from one round and the just-fundable two rounds down the road (hard to turn a grant around in one round). Over the course of a year, there is going to be a big bulk of proposals funded that would have been funded anyway at the savings of one less revised application. I'd argue that there will be little loss of the "best of best new proposals" on a per round basis because most of the brand new ones have to get in line behind revised apps anyway.
The more radical solutions I have in mind would be independent of specific funding climate. The first approach would be to significantly alter the instructions to reviewers. CSR provides specific instructions, instructions general enough to be useless and also explicitly avoids instructing on many issues (some of which I think could use more instruction). I'm not really privy to the whys of all this, the generic poles of the debate are NIH interests versus maintaining the independence of peer review. The point is, they can and do issue instructions, like the 5 review criteria, when they want to. For this purpose, there could be a series of instructions stating that the unrevised app is to be prioritized, in fact some of the changes toward significance and reduced application length will move in this direction anyway. I'd like to see it firmed up. Second, ICs could start looking at score percentiles by revision status and simply allocate a majority of their funds in a given round to unrevised applications. Going by the abovementioned CRISP technique, most of the CSR study sections close to my area end up with about 10% of the funded grants getting there unrevised. I'd like to see this up around 60%.

In re-reading this post from last year, I still agree with most of my ideas. Interestingly we've seen some of these issues raised in the ongoing NIH self-examination effort which I take to mean that I am not alone in these thoughts!
Dear Readers, chime in with your ideas. It is no accident that I am reposting this one today. As I mentioned in the comments, yesterday's little joke drew considerable attention from the domain. Transparent as I thought it was, a few of my agents confirm the impression that there was a little email viral infection running through the NIH and not all of it was "lookit this April Fool's funny"! Whatever the source of the hits, our humble blog came to the attention of a major category of people we would like to influence. Who knows, maybe a few of them will hang around for awhile. Maybe an idea or two will stick. Maybe they will realize that there is a whole slice of their scientist population that they are not reaching with their efforts to drum up opinion from the BlueHaired and GrayBearded.

So DearReaders, since I know you all have your own ideas, throw 'em out. No time like the present.

10 responses so far

  • phd says:

    did you consider that POs are totally invested in status quo? And their MERIT-orious friends? That one is a fantasy DM!

  • juniorprof says:

    I'm gonna copy a suggestion I made in one of your other posts on the topic at the old site. I also sent this comment (or something like it) to NIH in their open comment period.
    The comment is about my concern with the lack of data to establish that more senior reviewers actually do a better job reviewing than asst profs or senior postdoc rank reviewrs.
    "Why not have NIH run an experiment. Establish a study section of asst profs and postdocs to run at the same time as an established study section but keep them completely independent (with the established one not even knowing the other exists). Leave all the actual scoring and summary statements to the established group but compare the summaries for the grants and scores between the groups over a few cycles and see what happens. To make it even more interesting, track the progress of the proposed projects over a couple years (without attention to what is funded or not) and see which groups were able to best identify projects that would make a contribution. A seperate independent panel could judge this comprised of senior and junior researchers in the same field. Would something of that sort give us a better picture of what is actually important in peer review of grants rather than just the same old dogma? I

  • DrugMonkey says:

    I should also note that juniorprof sent me a relevant article awhile ago that I was supposed to be blogging. I never got to it.
    Mark A. Kliewer, Kelly S. Freed, David M. DeLong, Perry J. Pickhardt and James M. Provenzale
    Reviewing the Reviewers: Comparison of Review Quality and Reviewer Characteristics at the American Journal of Roentgenology
    AJR 2005; 184:1731-1735
    it can be found (i think for free to anyone) here:
    Money quote from the Abstract:
    The highest-rated AJR reviewers tended to be young and from academic institutions. The quality of peer review did not correlate with the sex, academic rank, or subspecialty of the reviewer.

  • drdrA says:

    So much to comment on I barely know where to start. So lets start with capping the # of $$ give to any individual PI...
    Perhaps you saw the article in Nature (Revamp for NIH Grants, Nature 451, Feb 22, 2008, since this is access by paid subscription I haven't figured out how to link this to a blog yet)- there are 783 grantees who have 4 OR MORE R01s... What's this,... that has to STOP, especially when one of those R01s is being used to pay the PIs salary in 100% soft-money places.

  • drdrA says:

    Well, 4 or more GRANTS it says,... not 4 or more R01s.. sorry I mis-quoted..
    It's still nuts.

  • DrugMonkey says:

    drdrA, we had something on the copiously-fundedPI issue here.
    I should also clarify something for my new readers for openness sake. I am in a soft-money job category and for the most part I don't have a really serious problem with the idea that one is working entirely "for the NIH". My critiques have more to do with the fact that I lament the passing era (if it ever existed) in which local institutions took their half of the NIH/University partnership seriously. I'd like to have a hard money appointment, sure but I took my job with eyes wide open.
    I do have a big problem with the way NIH officiousness seemed willfully blind to the soft-money "system" that was developing. "Well, if that University isn't willing to give you a FTE then why would you ever want to take a job there?" being a typical example of the PO line up to only a few years ago.
    It is also worth noting in a general sense that while I decry many of the ways the NIH game works, you can bet your boots I work hard in many ways to optimize my results within that system. I work the revision thing. I try to burnish my OldBoyz cred where possible. I chat up Program. While I try to help career "senior scientists" where I can, I for dang sure am willing to use them in my lab when they come my way. Should I be lucky enough to get a 1%le or whatever it takes, on a competing continuation I'd for sure be hinting around about MERIT, you betcha.
    In other words it is likely that I am a (small) part of what is bad with the system. I am not one who completely falls on the sword for high principles in this respect. My most basic principle is that I want to keep doing the science that I find to be important after all. I assuage my guilt (!) by doing what I can to change attitudes from within (to the degree that I am "within").

  • neurolover says:

    Systems that require people to fall on their sword in order to change never do change. I mean, if you fall on your sword, you're out of the system, and it can go happily along without you. Sometimes, in extreme circumstances, to quote guy gavriel kay, you have no choice but to say "no, and die." But, I don't think the state of affairs in biomedical science has reached that extreme yet. So, drugmonkey, I give you permission not to fall on your sword. I'm trying not to fall on mine either.

  • drdrA says:

    Once again- I'm like 10 steps behind you- just shows you how much I have to learn and what an education I will get scouring your blog. There was a very informative discussion on that previous post...

  • DrugMonkey says:

    likewise drdrA, I've been enjoying your career posts.
    i'll also note that writedit ( typically keeps a low profile but that blog is frequently more on top of things like the Nature article than anyone else.

  • drdrA says:

    I looked at writedit- that's an awesome site... I'll have to add it to my blogroll. The latest post is just perfect (things to do if you absolutely DON'T want to get the grant)... I loved it!

Leave a Reply