Archive for the 'NIH Budgets and Economics' category
In his latest column at ASBMB Today, Steve McKnight attempts to further his assertion that peer review of NIH grants needs to be revamped so that more qualified reviewers are doing the deciding about what gets funded.
He starts off with a comment that further reveals his naivete and noobitude when it comes to these issues.
Reviewers judge the application using five criteria: significance, investigator, innovation, approach and environment. Although study sections may weigh the importance of these criteria to differing degrees, it seems to me that feasibility of success of the proposed research plan (approach) tends to dominate. I will endeavor to provide a quantitative assessment of this in next month’s essay.
The NIH, led by then-NIGMS Director Berg, already provided this assessment. Ages ago. Try to keep up. I mention this because it is becoming an obvious trend that McKnight (and, keep in mind, many of his co-travelers that don't reveal their ignorance quite so publicly) spouts off his ill-informed opinions without the benefit of the data that you, Dear Reader, have been grappling with for several years now .
As reported last month, 72 percent of reviewers serving the HHMI are members of the National Academy of Sciences. How do things compare at the NIH? Data kindly provided by the CSR indicate that there were 7,886 reviewers on its standing study sections in 2014. Evaluation of these data reveals the following:
48 out of 324 HHMI investigators (15 percent) participated in at least one study section meeting.
47 out of 488 NIH-funded NAS members (10 percent) participated in at least one study section meeting.
11 of these reviewers are both funded by HHMI and NAS members.
These 84 scientists constituted roughly 1.1 percent of the reviewer cadre utilized by the CSR.
This tells us nearly nothing of importance. How many investigators from other pertinent slices of the distribution serve? ASBMB members, for example? PIs from the top 20, 50, 100 funded Universities and Medical Schools? How many applications do NAS / HHMI investigators submit each year? In short, are they over- or under-represented in the NIH review system?
Anyway, why focus on these folks?
I have focused on the HHMI investigators and NAS members because it is straightforward to identify them and quantify their participation in the review process. It is my belief that HHMI investigators and NIH-funded members of the NAS are substantively accomplished. I readily admit that scientific accomplishment does not necessarily equate to effective capacity to review. I do, however, believe that a reasonable correlation exists between past scientific accomplishment and capacity to choose effectively between good and poor bets. This contention is open for debate and is — to me — of significant importance.
So confused. First, the supposed rationale that these elite scientists are readily discernible folks amongst a host of well qualified so that's why he has used them for his example, aka the Street Lamp excuse. Next we get a ready admission that his entire thesis he's been pursuing since the riff-raff column is flawed, followed immediately by a restatement of his position based on..."belief". While admitting it is open to debate.
So how has he moved the discussion forward? All that we have at this point is his continued assertion of his position. The data on study section participation do exactly nothing to address his point.
Third, it is clear that HHMI investigators and NIH-funded members of the NAS participate in study sections charged with the review of basic research to a far greater extent than clinical research. It is my belief that study sections involving HHMI investigators and NAS members benefit from the involvement of highly accomplished scientists. If that is correct, the quality of certain basic science study sections may be high.
Without additional information this could be an entirely circular argument. If HHMI and NAS folks are selected disproportionally for their pursuit of basic science (I believe they are, Professor McKnight. Shall you accept my "belief" as we are expected to credit yours? or perhaps should you have looked into this?) they of course they would be disproportioanlly on "basic" study sections. If only there were a clinically focused organization of elite good-old-backslappers-club folks to provide a suitable comparison of more clinically-focused scientists.
McKnight closes with this:
I assume that it is a common desire of our biomedical community that all sources of funding, be they private or public, find their way to the support of our most qualified scientists — irrespective of age, gender, ethnicity, geographical location or any other variable. In subsequent essays, I will offer ideas as to how the NIH system of grant award distribution might be altered to meet this goal.
Nope. We want the funding to go to the most important science. Within those constraints we want the funding to go to highly qualified scientists but we recognize that "the most qualified" is a fool's errand. Other factors come in to play. Such as "the most qualified who are not overloaded with other research projects at the moment". Or, "the most qualified who are not essentially carbon copies of the three other folks funded in similar research at the moment".
This is even before we get into the very thorny argument over qualifications and how we identify the "most" qualified for any particular purpose.
McKnight himself admits to this when he claims that there are lots of other qualified people but he selected HHMI/NAS out of mere convenience. I wonder if it will eventually trickle into his understanding that this mere convenience pollutes his entire thinking on this matter?
I hope you all have read through the Bridges to Independence (2005) report. Yes? It's freely downloadable and told us a lot about the state of NIH extramural funding, age cohorts and demographic disparities....a DECADE ago.
So when Rockey posts abbreviated data sets.....yeah.
Rockey had posted on the amount of grant money going to age groups, this Tweep divided by the number of PIs in each group.
I had two immediate thoughts.
When the end of the doubling hit, if you were 50 or under you felt it immediately.
If you were 56 or older at that point, you didn't feel anything until 2012.
Funny how nicely this maps onto attitudes. We've seen the older types get vocal only in the last 2-3 years and we have been bemused.
My response has been "welcome to the reality the rest of us have been under for a decade."
Nice to see some actual data confirming that the Boomers really have been insulated from pain until recently.
UPDATED: More from @MHendr1cks
The piechart really brings it home, doesn't it?
The NIH Director, Francis Collins, was speaking to Congress this week and was widely quoted as lamenting the fate of junior scientists. As per this Sam Stein bit in HuffPo:
“This is the issue that wakes me up at night when I try to contemplate the future of where biomedical research can go in the United States,” Collins said. “They are finding themselves in a situation that is the least supportive of that vision in 50 years. They look ahead of them and see the more senior scientists struggling to keep their labs going and suffering rejection after rejection of grants that previously would have been supportive. And they wonder, 'Do we really want to sign up for that?' And many of them, regrettably, are making the decision to walk away.”
Obviously he is talking about trainees and perhaps the very newest of assistant professors, aka ESI qualified NIH applicants.
This goes along with a continued trend from the NIH. To wring their collective hands over those who are in their mid to late 30s and younger. To take some steps to help them out, most definitively with special paylines for the Early Stage Investigators who must be no more than 10 years away from the PhD award. To nod sagely about "eating our seed corn" as if they have the slightest clue what that might mean and whether it actually applies here (it doesn't).
It ignores another trend from the NIH, i.e. working busily to shore up the ability of the oldest guard of scientists to remain funded. You know about the Emeritus award they are considering. You have observed how well the very oldest slice of our PI applicant pool is treated at study section. And you have seen how NIGMS, the IC most serious about this workforce stability stuff*, put the oldsters at the front of the line with their MIRA initiative. Of course, the second in line (and in fact the only ones in line) for this little MIRA project are, you guessed it, ESIs.
We plan to issue a MIRA funding opportunity for early stage investigators as quickly as possible. We hope the first application due date will be sometime this summer.
As per usual, the demographic of the mid-career investigator is overlooked.
One of the comments on the NIGMS MIRA post is heart breaking and incredibly truthful. BioScientist wrote:
However, I have genuine concerns about the idea to roll it out first to either well-funded labs or early stage investigators. From what I can see, where it is most needed is in mid-career labs that do not have multiple R01’s, which in many cases are imploding in the present environment. These are the PI’s who are writing 10 grants to get 1 funded right now. The well-funded empires are doing just fine, and I have not found the PI’s of such labs to be the egalitarian types would would give up a dime so that someone else could keep a lab running.
For ESI’s, this could be an interesting experiment in how to launch successful careers. Many of us who endured the system of the last decade are discouraged and demoralized. Personally, I will never live up to my scientific potential after so many years of wasting time on failed proposals and preliminary results for projects that were never funded.
emphasis added. As if I have to do so.
Do you wonder why the current greybearded and silver haired people who remain powerful in science are so keen to cry over the poor, poor Millennial generation of scientists and wring their hands over the future of science, all the while doing nothing about the present of science?
Because the Boomers (and a few years' worth of pre-War folks) cannot acknowledge what they have done to the Gen X scientists. Some of the charges are as follows.
1) Extended graduate school training from 4 years to 6+. Sure they used all sorts of very truthy sounding excuses about mastering different domains, getting those three publications in CNS journals, the collaborative nature of vertically ascending science, etc. But they accomplished it...and their own successes prove it unneeded.
2) Extended postdoctoral "training". The moved us from where even two years as a postdoc prior to professorial appointment was slightly suspicious (in the early to mid 1970s) to a situation in which two sequential 3-5 year postdocs are viewed as the necessary minimum (just a few years ago, prior to the ESI foofraw). The oldest generation oversaw this.
3) Even during the NIH doubling, they grabbed all the grants and kept beating up the newly appointed GenX scientists with Stock Critiques, sent them around the airport traffic pattern in endless revisions and with "good scores" that were clearly unfundable. Anything to delay entry and preserve their expanding empires.
3) The R29 FIRST was dismantled** but was replaced by a NI check box. It supposedly took the oldster power brokers 10 years to realize was to the benefit of, you guessed it, themselves. I.e. those highly established scientists that simply didn't have NIH funding yet. It took me about 3 hours of my first study section meeting to see this.
4) ...aaaand what do you know? By the time the old guard power brokers "realized" this NI problem, they were able to fix it with a time-limited ESI designation tagged to the time of PhD award, instead of the time of Asst Prof appointment. This conveniently skipped right over the Gen X scientists.
So what did this accomplish? Well, on the trainee end of the screw-job this just meant more time in which a venerated or even hard charging mid career lab head could benefit from the intellectual contributions of the Gen X scientists. Pretty much like intellectual vampires. The crediting system whereby author lines expanded and the senior author got all the glory was refined and elaborated from the 1990s through the Naughties as the NIH budget doubled. The number of "postdocs" supported on research grants soared through the roof. And the new models, conceptual breakthroughs and new theoretical approaches continued to give subsequent grant largesse and subsequent paper / finding laurels to the lab head. While the Gen X scientists continued as postdocs, or were shelled out of the system or manged to get a job but couldn't get funded very easily.
I was there. I know who did the actual work in the labs in my fields of interest. I know the way a finding or paper or model resulted in the lab head having copious funding for a decade and a half, verging on two decades now. I know which of those scientists of my generation failed to make it big. There are a lot of them that will never achieve their promise. A lot who had to bail entirely on the career after what would have been a career-making paper as a trainee, if they were just a generation older. I can point to very few of the Gen X people in my fields of closest interest who have hit mid career with anything like the funding, verve and accomplishment of even some of the more, shall we say, pedestrian*** members of the generation just prior to mine****. Actually, come to think of it, I am hard pressed to point to a single one.
I am not suggesting the older folks who benefited had no right to do so. I am not saying they didn't deserve any credit, nor am I claiming they didn't contribute intellectually.
I am saying that they (as a generation) arranged things so that they got ALL OF the credit and benefit of the collaborative breakthroughs. And this is not right. They did not suffer a similar fate at the hands of their more-senior colleagues because times were very different. Expansive. Lab sizes were smaller and the trainees were more consistently encouraged to fly away and shine on their own. This is what happened through the 70s and 80s when they were transitioning. And yet they have the nerve to call us riff raff. To question our commitment to science in oh so many ways. To continue to credit themselves for breakthroughs and advances that rested on the intellectual labors of a younger generation that they now disparage.
Some of us are surviving. Yes. This is obvious. Some of us are thriving. Some of us squeaking by on fumes and prayers.
Some Most of us yo-yo between these extremes.
As the comment said, however, we will never reach our potential as scientists. Not in the way we witnessed the generation or two before us reach theirs. Not as a generation and not as the vast majority of individuals. Ever. It cannot be recovered.
Do you wonder why we are angry at each and every NIH initiative that comes down the pike that is explicitly designed to skip over our generation of scientists, yet again?
This is why.
*yeah. This is as good as it gets.
**for good reason, it had problems
***to be clear, I count myself in this category
****Who managed to get past the noob-abuse and hazing ritual juuuuuuust as the doubling hit stride. This is the generation that managed to land the last few R29s, for reference*****.
*****In truth, those who were never eligible for either R29 or ESI designation makes a much better and tighter demarcation than Gen X versus the Boomers and Millennials when it comes to this stuff. But it is pretty damn inside baseball to use such terms....
Related Reading: Does anybody want to be president? Anyone?
Popular thought. But it is nonsense.
A close collaborator was recently experiencing this common denial trope from one of the more established type of scientists. The thinking is that
"...sure, things are tough for younger scientists right now but hey, things have been tough before. It's all just a cycle and oh, stop complaining kiddos. We had it hard too."
Here is why it is in error to argue this- the magnitude of the downturn was lesser and it lasted for a shorter duration in those prior "cycles". Let us refer to the infamous Undoubling graph.
Figure 1. NIH Appropriations (Adjusted for Inflation in Biomedical Research) from 1965 through 2007, the President's Request for 2008, and Projected Historical Trends through 2010.
All values have been adjusted according to the Biomedical Research and Development Price Index on the basis of a standard set of relevant goods and services (with 1998 as the base year). The trend line indicates average real annual growth between fiscal years 1971 and 1998 (3.34%), with projected growth (dashed line) at the same rate. The red square indicates the president's proposed NIH budget for fiscal year 2008, also adjusted for inflation in biomedical research.
The previous downturns in the NIH funding (and you can verify the scientist complaining by looking through old Science magazines, btw) occurred approximately in the late 1960s, the early 1980s* and the early 1990s. I happened to join this career path right around the 1990s downturn and I remember the whining about grant funding quite clearly. That 1990s downturn was what led to the infamous NIH Doubling. The late 1960s downturn led to Congressional action as well. In both cases you can see where the lapse in Congressional interest led to the following episode of downturn. It is here that we should also review the subsequent update on the Undoubling graph, the even more sinister Defunding Graph.
Via Michael White, presumably via John F Sargent, Jr.
It should be emphatically clear to even the casual observer that the magnitude of the decline in the NIH budget and the duration of the downturn prior to the next Congressional rescue differs. Dramatically. Make sure you check the corresponding longitudinal trends in grant success rates. In case you are wondering about the most recent numbers, according to Sally Rockey, the overall RPG success rates for FY 2012-2014 are 17.6%, 16.8% and 18.1%, respectively. Things are most emphatically not good for the kids these days.
These are the facts. We can argue until the cows come home over how and why various up and down cycles have occurred. We can dispute whether Congressional appropriations intended to rescue the NIH extramural community do harm, good, a balance of the two and what this means for the future.
It is not optional, however, to act like the present downturn is of the same magnitude or impact as the prior ones.
*"I remember multiple study section rounds in which nothing ended up getting funded" --a senior colleague
The strategies for sustaining a vibrant biomedical research enterprise are complex and multi-faceted. NIH strives to support not only the biomedical research workforce, but to support the foundation of research programs that our workforce has created.
An important issue for NIH is the long term success for the research we support. Over the years, NIH has been persistent and creative in efforts to support early career investigators through policy changes and new programs. But we must also consider the needs of our mid-career investigators and how NIH can assist with the continuation of their well-established research programs, since evidence has shown that the most innovative and productive years of work come from PIs in the 40-59 year old age range. While these highly productive investigators are happy pursuing their research questions in the laboratory, current funding stresses have hindered the current generation relative to past generations of mid-career scientist. Our most vibrant investigators have invested their careers to establish the intellectual and technical infrastructure needed to pursue their research questions, and it is in our interest to facilitate progress in their established programs.
Yes, I agree! Totally true.
Therefore, NIH would like to explore potential mechanisms to facilitate the needs of the most productive members of our biomedical workforce. We would like to gauge community interest in a new type of award that could allow established investigators to maximize their output under funding from NIH research grants, while greatly advancing our scientific knowledge and resources. Such an award could permit an established investigator to form partnerships with other faculty members in order to facilitate research inquiry in an efficient and cost-effective way under P-mechanisms as with prior generations. The established investigator would, of course, be expected to train and equip junior colleagues to contribute to mutual interests and research projects while working with them in a mentoring role. If such a collaboration is not feasible, a mid-career award might allow some established investigators to complete expansive projects within their own laboratories.
Wow. Really good stuff here NIH. Glad to see you finally recognizing what brung ya and what you need to bring to the race to keep on winning.
Request for Information This Request for Information (RFI) seeks input from the research community, including scientists from all career stages; research administration professionals; departmental chairs; deans; professional societies; and other interested stakeholders. Public comment is sought for the following:
- Community interest in an award that allows a mid-career investigator to flourish without being dependent on submitting so many NIH research grant applications
- Ideas for how one would utilize a mid-career award (e.g., to facilitate laboratory sustainability; to promote novel research inquiry; to provide opportunities for expansion of larger collaborative research projects)
- Suggestions for the specific characteristics for a mid-career award (e.g., number of years of support; amount of support; mechanisms of evaluation)
- Ways in which NIH could incentivize the use of a mid-career award, from the perspectives of both mid-career investigators and institutions
- Impediments to the participation in such an award program, from the perspectives of both mid career investigators and institutions
- Any additional comments you would like to offer to NIH on this topic
Oh, for sure. I'm going to run, not walk, right on over to the form to submit my approval.
This is for EMERITUS faculty? Such as those past the age of 65 who keep on submitting copious numbers of research grants? And the NIH wants to somehow use this to persuade the unwilling* to wind down their lab in good order?
What a disaster.
*This mechanism for winding down a PI's career while sustaining his** "legacy" laboratory and program already exists and is in current practice. A senior PI simply steps down from the PI position and the University nominates a junior person to take over. Maybe with continued Co-investigator status for the Emeritizing person. It works to serve this goal. It is proven.
**yeah, "his". that's who these people are. For the most part.
One David Korn of the Massachusetts General Hospital and Harvard Medical School has written a letter to Nature defending the indirect cost (IDC; "overhead") rates associated with NIH grants. It was submitted in response to a prior piece in Nature on IDC which was, to my eye, actually fairly good and tended to support the notion that IDC rates are not exorbitant.
But overall, the data support administrators’ assertions that their actual recovery of indirect costs often falls well below their negotiated rates. Overall, the average negotiated rate is 53%, and the average reimbursed rate is 34%.
The original article also pointed out why the larger private Universities have been heard from loudly, while the frequent punching-bag smaller research institutes with larger IDC rates are silent.
Although non-profit institutes command high rates, together they got just $611 million of the NIH’s money for indirect costs. The higher-learning institutes for which Nature obtained data received $3.9 billion, with more than $1 billion of that going to just nine institutions, including Johns Hopkins University in Baltimore, Maryland, and Stanford (see ‘Top 10 earners’).
Clearly Dr. Korn felt that this piece needed correction:
Aspects of your report on US federal funding of direct research costs and the indirect costs of facilities and administration are misleading (Nature 515, 326–329; 2014).
Contrary to your claim, no one is benefiting from federal largesse. Rather, the US government is partially reimbursing research universities for audit-verified indirect costs that they have already incurred.
Ok, ok. Fair enough. At the very least it is fine to underline this point if it doesn't come across in the original Nature article to every reader.
The biomedical sciences depend on powerful technologies that require special housing, considerable energy consumption, and maintenance. Administration is being bloated by federal regulations, many of which dictate how scientists conduct and disseminate their research. It is therefore all the more remarkable that the share of extramural research spending on indirect costs by the US National Institutes of Health (NIH) has been stable at around 30% for several decades.
Pretty good point.
But then Korn goes on to step right in a pile.
Negotiated and actual recovery rates for indirect costs vary across the academic community because federal research funding is merit-based, not a welfare programme.
You will recognize this theme from a prior complaint from Boston-area institutions.
“There’s a battle between merit and egalitarianism,” said Dr. David Page, director of the Whitehead Institute, a prestigious research institution in Cambridge affiliated with MIT.
Tone deaf, guys. Totally tone deaf. Absolutely counter-productive to the effort to get a majority of Congress Critters on board with support for the NIH mission. Hint: Your various Massachusetts Critters get to vote once, just like the Critters from North and South Dakota, Alabama and everywhere else that doesn't have a huge NIH-funded research enterprise.
And why Korn chooses to use a comment about IDC rates to advance this agenda is baffling. The takeaway message is that he thinks that higher IDC rates are awarded because His Awesome University deserves it due to the merit of their research. This totally undercuts the point he is trying to make, which is presumably "institutions may be private or public, urban or rural, with different structures, sizes, missions and financial anatomies.".
I just don't understand people who are this clueless and selfish when it comes to basic politics.
According to the policy on this webpage, the NIGMS will now restrict award of its grants when the applicant PI has substantial other research support. It is effective as of new grants submitted on or after 2 Jan, 2015.
The clear statement of purpose:
Investigators with substantial, long-term, unrestricted research support may generally hold no more than one NIGMS research grant.
For the purposes of these guidelines, investigators with substantial, long-term, unrestricted support (“unrestricted investigators”) would have at least $400,000 in unrestricted support (direct costs excluding the principal investigator’s salary and direct support of widely shared institutional resources, such as NMR facilities) extending at least 2 years from the time of funding the NIGMS grant. As in all cases, if NIGMS funding of a grant to an investigator with substantial, long-term, unrestricted support would result in total direct costs from all sources exceeding $750,000, National Advisory General Medical Sciences Council approval would be required
This $400,000 limit, extending for two years would appear to mean $200,000 per year in direct costs? So basically the equivalent of a single additional R01-worth of direct cost funding?
I guess they are serious about the notion that two grants is fine but three-R01-level funding means you are a greedy commons-spoiling so-and-so.