Boom and bust instability in NIH funding

May 13 2014 Published by under Fixing the NIH, NIH

The Tweep known as @dr24hours made a comment on grant strategy:

which I have to admit makes a lot of sense. Get your mental health together by not worrying about yet another grant deadline...you've earned it! Take the time to do what your job really is about...looking at data, planning studies, bringing them to fruition. Publishing. Have some science fun and bask in the sunshine*.

In a different age of the NIH grant game, this would have been a fine strategy for your career as well.

It no longer is fine.

We are in an era of boom and bust instability when it comes to NIH funding. It is the very rare flower indeed, in my estimation, that will be completely free of the cycle in the coming decade or two.

As always, my view is quite possibly colored by my experiences. But I have seen the boom and bust cycle play out across a large number of labs. Some of my close acquaintance. Some labs that I know only through the grant review process. Some labs that happen to make it to the scuttlebutt news channel for some reason or other.

It usually plays out like this. "Yeah, Dr. So-and-so is really well funded.....what? What do you mean they are on the ropes? [checks RePORTER]..how in the hell did THAT happen". ....Two years later "Oh phew, glad to see So-and-so got another grant. ....what? TWO grants? and an R21? how in the hell did THAT happen?"...

Repeat.

There are a couple of ways the current uncertainty amps up the gain on the boom and bust cycle of grant funding.

Getting back to Dr24hours' assertion in the Twitt, yes, a lot of PIs do their grant submitting in bursts. If you have enough funding, why submit applications? And if you are approaching the end of your current funding, well, you are going to start shooting those applications out on full-auto fire. This makes a lot of sense and I think a lot of us, including YHN, do this by reflex.

If you apply the current grant success odds to more (down cycle) versus fewer (up cycle) applications, well, you can see that a PI is herself contributing to the amplitude of the cycle.

Then we get to the way grant reviewers look at a PI and the application that is in front of them. Yes, Virginia, perceptions of "too much funding" do contaminate the reviewer's mind. It is inevitable. Just look at all the screaming over at RockTalking and on my posts about how the "real solution" is to limit the amount of funding any one PI can secure. And when a PI appears to be on the ropes, grant-wise, the reviewers are likely to feel....sympathy. I know, I know, this sentiment is thin stuff compared with the negative value of the perception of "too much funding" but it is most certainly a contributing factor.

What this means is that if you take a PI's grant proposals as more-or-less equally deserving on objective grounds, the ones that are submitted when she has a few grants are less likely to get a fundable score. This accelerates the downfall after an interval of healthy funding. On the other side, when the PI looks like she is at the end of her funding, the sympathy cred will make an application relatively more likely to fund. And since there will be a lengthy interval of time in which the lab looks to be running on fumes, it could be that several applications (perhaps in different study sections) will be pushed over the funding line by sympathy.

Finally, your friendly Program Officer plays a role in this as well. As we've seen over the past few years they have been very overt and explicit about "saving" long-running programs that are on the ropes. They will perhaps not admit it, but you know damn well that when it comes to making out-of-order funding decisions, if there is a perception that PI Jones is "healthily funded" and PI Smith is on the skids, the Smith app will get picked up and the Jones app will not. Again, the down cycle is accelerated and the up-cycle can be inflated. In some perfect world where such considerations didn't matter, the deflation of the labs on the up cycle would be attenuated by a Program pick up about as often as the rescue of labs on the down cycle would be accomplished. This should work against the boom-and-bust (albeit potentially at the risk of increasing lab-death).

As you are well aware, Dear Reader, I pursue a two-pronged approach when it comes to this stuff.

First is always my advice on how the individual PI is best to navigate the system. I see no solution to this phenomenon other than to keep a steady flow of apps, even when you are relatively well funded. Five years elapses VERY rapidly and the confidence that even a very productive project, or one that hits all the expectations laid out in the original application, will be continued is low. Very low. There is still a degree to which the revise-and-resubmit cycle improves your odds. That can take, what, two years from original submission to eventual funding of the A1? Well two years out from the end of your current award, you still look like you are healthily funded. But what can you do? You sure as heck can't count on sympathy to push your A0 version over the funding line if you wait until the last six months of your current award.

Second, I continue to discuss ways to stabilize funding. It certainly is all the rage with Rockey and friends. (Sally Rockey showed up to a symposium on "Bridging Career Pathways for an Evolving Biomedical Workforce" at Experimental Biology this year, btw.) As always, it is a long slow slog to get the powers that be to even understand how to ask the right questions, nevermind how to arrive at the right answers. One of the problems is that official NIHdom never thinks about careers. It took a fair while for Rockey to blog about funding rates by PI (here) as opposed to by grant. And it was still hard for her / her data minions to grasp that the success rates of PIs had to look over longer intervals of time, five to ten years. Jeremy Berg has been doing a good job of starting the process of examining the career aspects, see here and here. I think understanding the PI dynamics is critical to achieving some sort of stabilization of the uncertainty of funding that is so paralytic to science right now.

Part of this understanding should be a recognition of the boom-and-bust cycle.

If it exists. My perceptions could be very skewed, I realize. There should be a way for Rockey to get her data miners to characterize periodicity of funding cycles versus relatively sustained funding. To see if the relative proportions of PIs enjoying sustained, relatively invariant levels of funding differ across time. My prediction is, of course, that Rockey would find that there is more variability in a given PI's funding across time then in the past.

As far as fixes for the boom / bust cycle go, I think we can't do much about the PIs' behavior or the way that study sections bias their reviews on the perception of how much funding a person has at that particular moment. This leaves it in the hand of Program to try to stabilize matters. So far their attention seems mainly focused on saving labs as they are approaching "bust". This is admirable from many perspectives.

But I suspect that if they focused on arresting the "bust" before it actually happens it would have the same overall effect on the PI population while avoiding so much inefficiency of production that attends a lab plummeting toward the funding abyss.

__
*because Winter is Coming.

22 responses so far

  • dr24hours says:

    Winter is here.

  • Dave says:

    Perhaps I don't get it, but why does a lab/PI have to have more than one or two (max) R01s? We know that there is not a strong relationship between increasing lab size and 'productivity', so what is the scientific benefit? Or are you just talking number of applications here, and you are assuming 10 R01 applications over 3 - 4 years = one funded grant?

    We have had this discussion many times, but my concern is that one cannot see the wood for the trees when your head is buried in a computer monitor constantly. Why can't we get to a point where a grant is a means to an end, not the goal itself?

  • Physician Scientist says:

    Dave-

    One R01 is $205K from NIGMS. My costs per year:

    1. $50K + 31% benefits for my salary = $65.5K
    2. Technician = 45K + 31% benefits = $59K
    3. Mouse costs = $35K
    4. Supply/reagents - $2K/person/month = $24K

    So at my R1 school, an R01 will support a 2 person lab. Nobody in their right mind stops at 1 R01, hence the churn. Personally, I require $450K/year to have a 6 person lab. There is no superfluous spending, but this requires a grant at least every cycle to continue. A grant has to be the goal when you 've got mouths to feed.

  • Dave says:

    But wouldn't you rather spend less time writing grants and just make do with a smaller lab (i.e. 3 vs. 6 people)? Your research would likely slow down, or you would have to pursue less projects, but it seems like a more sustainable situation. I think $450K/year will be tough to maintain for even the best investigators, of which I am sure your are one.

  • Physician Scientist says:

    The thing is, Dave, that you can't do it with one R01, the timing of the NIH review process and the rate of money out the door. My timeline for R01 renewal on a very productive cycle...

    Grant runs out Nov. 30, 2013
    1st competing renewal submitted Nov, 2012
    Score = 17th percentile, Feb. 2013
    Revision submitted = July, 2013
    Score = 9th percentile, Nov. 2013
    Council review = Jan. 2014 (R01 has ended at this point)
    Grant funded = May, 2014

    There was a 6 month gap in funding with a grant I knew in late 2013 was going to be renewed. I needed an additional R01 to overlap so that the lab had financial stability. I could go to three people, but I'd be on the edge of a cliff with each renewal (which is no sure thing these days). The system is forcing us to put in 2-3 cycles/year.

  • drugmonkey says:

    Perhaps I don't get it, but why does a lab/PI have to have more than one or two (max) R01s?

    they don't "have to". They think that this is what is necessary to do the science they want to accomplish. as far as that goes, well, this depends entirely on the specifics of what the science is that we are discussing. Some science is a lot cheaper, some is more expensive. Some PIs depend entirely on trainees who are funded by mechanisms other than their R-grants and other PIs work exclusively with postdocs and/or techs who are funded from their grants. Some PIs have to cover their entire salary from grants and some don't.

    Really, this isn't rocket science. If what you really mean is "waaaaaah, I think we should only have science done exactly like I do it", well, good luck with that.

    We know that there is not a strong relationship between increasing lab size and 'productivity', so what is the scientific benefit?

    That is a considerable misrepresentation of the data that have been presented on this, one. and two, those analyses focus on the metrics of paper count and occasionally JIF. The relationship to real scientific progress is not captured. Those analyses are tied to a single grant award (i.e., do not consider the sustained career issues being discussed in this post) and are entirely blind to the above issues about scientific cost.

    Or are you just talking number of applications here, and you are assuming 10 R01 applications over 3 - 4 years = one funded grant?

    This discussion doesn't really have anything to do with the number of grants. An up-and-down cycle could be a lab going from 1 grant to none or from 4 grants to 2 or whatever. The baseline is irrelevant to the point about boom/bust. All you have to consider is that for a given investigator doing a given kind of science over time, there can be minor funding wobble and major funding wobble.

  • Ola says:

    I've successfully renewed 3 times (twice renewed one R01, once renewed another). Every single time there's been a gap in funding, often 6-8 months no-cost-extension, "running on fumes" as you say. I am firmly of the belief that the sympathy vote played a big role in the proposals eventually getting funded. Like Physician Scientist, the A0s were all submitted at least a year before the old grant ran out.

    Re: Dave's comment on 2 R01s, this of course assumes a modular budget actually gets fully funded. For most of the past decade NIGMS has been cutting 2 modules off every award (so they can fund a 5th, extra grant for every 4 regular). 200k is just not much cash, especially at schools with large salary coverage expectations (60% for me currently).

    Example: Modular $200k real budget = $50k for my salary (30% effort with benefits), $60k for relatively senior post-doc (incl. benefits), $40k for tech' (incl. benefits), $25k mouse costs, leaving about $25k ($2k a month) for everything else. It's just about do-able, but if a major piece of equipment breaks then I'm fucked.

  • LIZR says:

    My approach to this boom bust cycle has been to try and maintain a mix of NIH and NSF fundable projects. As soon I am awarded a grant from one agency, I immediately switch most of my grant submission efforts over to the other agency. My NSF funded work is stuff that would clearly never be funded by NIH, and my NIH work would likewise not be readily funded through NSF. This approach has helped me to obtain staggered grants and avoid a complete loss of funding. It's too early yet to see whether this will be effective in the long term (I am a relatively new PI), but so far so good.

  • Dave says:

    I was trying to understand the strategies that you all (y'all?) use to avoid the boom and bust cycle, as DM puts it. I guess I was coming from the angle that if you had one successful R01, it should be fairly routine to renew it eventually, but that's old-school thinking and this from DM probably drives this point home:

    Five years elapses VERY rapidly and the confidence that even a very productive project, or one that hits all the expectations laid out in the original application, will be continued is low. Very low.

    That just doesn't seem like an efficient way to fund science, but I guess that's a different discussion.

  • drugmonkey says:

    Actually the NIH folks *are* discussing stabilization. See Collins' proposal for HHMI like funding (for an elite few). See Rockey's discussion points, the Alberts piece in PNAS, etc.

    And the PO behavior in "saving" labs is, I bet, done in a misguided intent to stabilize.

    I just don't think that only saving the desperate case is the way to go.

    Also, keep in mind that stabilization comes at the cost of opportunity for new entrants. It entrenches the established. There can be downsides.... And we know that how one views the risks/benefits is intimately tied to a person's personal standing in the system.

    Me, I still want to see an expansion of R37 MERIT to extend projects to a 10 yr interval of no competing. It can be done in a way that reaches a middle-ground compromise between stability and opportunity, IMO.

  • Dave says:

    Yeh I think more longer-term funding stability is clearly important, but it should focus not only on The Elites, but your Average Joe investigators too. But I'm skeptical this will happen. Like you say, any attempt to increase grant stability will inevitably benefit those at the very top of the NIH system because of limited NIH resources (see 'HHMI style awards').

  • Fuzzy Dizzy says:

    Surely, Collins has turned out to be the forever champion in maintaining the "status quo" in spite of all his theatrical performance on concerns for the future of young investigators, high risk research, innovation and all that jazz!. What a disappointment to say the least.

  • Joe says:

    More MERIT awards would seem like the best way to stabilize funding (as DM suggests) and would be a step toward a more HHMI-like system. My worry is that the resulting funding stability will only be available to those with the right pedigree and the right connections and will have little to do with merit.

  • drugmonkey says:

    Joe- if you imagine that *any* conceivable fix to the system is going to make those features disappear, well, I have a bridge to sell you. (It comes with an R01). All we can hope for is that the current R37 elite cohort is expanded. That makes it more democratic. But perhaps if one is still on the outside this is somehow worse in your view?

  • drugmonkey says:

    I'll also point out that one really stupid way to go after stability in the funding system is to pretend we are identifying the objective *merit* of some PI. We are in this pickle precisely because the discriminative resolution on ranking "merit" of project proposals is not sufficient given current paylines. If we could make such fine discriminations then the boom and bust cycle probably wouldn't exist, again, I want to see some sort of data workup on stability of funding for mid-career and above in the 80s and 90s versus the current decade.

    This is why I favor taking a look at the PI dynamics that Jeremy Berg has been reporting on recently.

    The first clue is to try to get a feel for how many scientists are at present, or should be (using historical trends), approximately continually funded. In a general sense, we can conclude that if these people enjoyed HHMI-like or R37-like funding, then things would be a lot more efficient from the perspective of the NIH.

    The second bit is trickier because it involves trying to predict who is or should be in this group. That is where you start running the risk of giving someone a sinecure for life who shouldn't have it at the risk of skipping someone who should have it. Nobody said it was going to be easy. But I do believe that consideration of the PI dynamics up to this point should show the NIH how silly it is to pretend that giving "HHMI-like" support to a tiny, tiny sliver of the population isn't going to fix the instability. Giving it to 20% (adjust as per the above graph analysis and your ideas about "approximately continual") of the population? That has some potential.

  • Wowchem says:

    DM
    I'm the exact opposite. I think we should have more short-term findings (like r21s but maybe 1 extra yr, larger budget) and increase the paylines. The only democratic system is to increase funding %. This would change funding rate to 30%. I think the top 1/3 should be solid. Also Universities need 50% skin in the game to each scientist's salary or increase score.

  • anonymous postdoc says:

    One assumption that has gone unquestioned (perhaps for good reason, please instruct me if so) is that longer grants should provide more money per year.

    This increases the burden on NIH in the given fiscal year to maintain funding for R01 investigators that were funded 4 years ago, which they have responded to by cutting the amount of the R01.

    What if there were a 10-year, $100K a year grant? This could provide a level of funding stability to a lab no other mechanism can. It could fund a technician and some reagents, nothing fancy, but something to keep people afloat. Perhaps, as a way of using this as a lever to get universities to invest more in their people, it could only be available to tenured/TT faculty. (Much love to my research track people, and it hurts me to suggest they should not compete for such an award, since they could use it the most - but we need to disincentivise universities from milking these positions.)

  • IGrrrl says:

    I Don 't know, wowchem. What you describe is pretty much the NSF model, with slightly smaller awards for usually a 3 year duration. Rarely are NSY R01-equivalent grants (Standard Grants) renewed in the way NIH grants are. Each new application is a new project. (Arguably this should also hold true in some respects for R01 renewals.) The "NSF model" might work for some mechanistic, bench-based work, but poses challenges for clinical studies, complex animal models, etc.

    On the flip side I seem to recall some data about the Javits awards where productivity and successful competing renewals were lower, but they do still give them. The model has changed from 7 years to 4 guaranteed and 3 more "may be awarded", I assume on productivity.

  • DrugMonkey says:

    Anonymous postdoc-
    Again, the data. We have reports from Rockey that most NIH funded PIs have more than $100k per year. There is probably some way to take Berg's continually-funded numbers and figure out the average funding for those folks. That will tell you a similar story , I bet.

    Introspection about what would be enough for *you* is less effective than looking at the research enterprise as it actually functions.

    It should be worth considering that the NIH set the full mod limit two decades ago for a reason . As we know, it hasn't been changed to keep up with inflation. The purchasing power that is needed by a lab is more on the order of $375K, not $100K.

  • drugmonkey says:

    Berg has a new post up at Datahound that relates to this discussion. The first graph is a reminder of a prior post but it also goes into the estimation of the approximately continually funded.

    It shows that after a two year gap in funding, some 20% of those individuals are back in the game. How can we not see this as an inefficiency in the system? Wouldn't the vast majority of these cases have been better off if there were some mechanism to continue the funding without a gap?

    It also shows you the trickier side. Well over half of the PIs that suffer a funding gap do not return to major NIH funding in 6 years (perhaps forever). So at the point of a gap....who do we select?

    If I were Rockey I'd be chasing the data minions to figure out as much as they possibly can about what predicts re-funding of the lab and what predicts a PI disappearing forever from the ranks.

  • Dave says:

    How can we not see this as an inefficiency in the system?

    Right.

    Well over half of the PIs that suffer a funding gap do not return to major NIH funding in 6 years

    Do we know that these people are still employed? If we don't know this, then it might say more about the intolerance of med schools for gaps in funding then the PIs ability to recover.

  • drugmonkey says:

    We don't know, no. Some will be retirees, obviously. Some will be continuing to submit unsuccessful apps - Rockey's minions could find that out. Some will have lost their jobs, as you suggest. Some will, presumably, be in a population of NIH funded investigators that only ever has a one-and-done award - Rockey's minions could probably figure out how big that population has been over time. Some may continue to be funded, just not as PIs (probably some fraction of the one-and-dones?).

Leave a Reply