Archive for the 'NIH' category

Harold Varmus is stepping down from the NCI Director position

Mar 04 2015 Published by under NIH, NIH Careerism

His letter to the NCI community is here.

Finally, when I return to New York City full time on April 1st, I will establish a modestly sized research laboratory in the Meyer Cancer Center at the Weill-Cornell Medical College...

Harold Varmus is 75 years old. I will be very fascinated to see if his idea of "modestly sized" accords in even the slightest with what I think of as a modest laboratory operation.

4 responses so far

Gen X will never live up to its scientific potential

Mar 04 2015 Published by under Anger, NIH, NIH Budgets and Economics, NIH Careerism

The NIH Director, Francis Collins, was speaking to Congress this week and was widely quoted as lamenting the fate of junior scientists. As per this Sam Stein bit in HuffPo:

“This is the issue that wakes me up at night when I try to contemplate the future of where biomedical research can go in the United States,” Collins said. “They are finding themselves in a situation that is the least supportive of that vision in 50 years. They look ahead of them and see the more senior scientists struggling to keep their labs going and suffering rejection after rejection of grants that previously would have been supportive. And they wonder, 'Do we really want to sign up for that?' And many of them, regrettably, are making the decision to walk away.”

Obviously he is talking about trainees and perhaps the very newest of assistant professors, aka ESI qualified NIH applicants.

This goes along with a continued trend from the NIH. To wring their collective hands over those who are in their mid to late 30s and younger. To take some steps to help them out, most definitively with special paylines for the Early Stage Investigators who must be no more than 10 years away from the PhD award. To nod sagely about "eating our seed corn" as if they have the slightest clue what that might mean and whether it actually applies here (it doesn't).

It ignores another trend from the NIH, i.e. working busily to shore up the ability of the oldest guard of scientists to remain funded. You know about the Emeritus award they are considering. You have observed how well the very oldest slice of our PI applicant pool is treated at study section. And you have seen how NIGMS, the IC most serious about this workforce stability stuff*, put the oldsters at the front of the line with their MIRA initiative. Of course, the second in line (and in fact the only ones in line) for this little MIRA project are, you guessed it, ESIs.

We plan to issue a MIRA funding opportunity for early stage investigators as quickly as possible. We hope the first application due date will be sometime this summer.

As per usual, the demographic of the mid-career investigator is overlooked.

One of the comments on the NIGMS MIRA post is heart breaking and incredibly truthful. BioScientist wrote:

However, I have genuine concerns about the idea to roll it out first to either well-funded labs or early stage investigators. From what I can see, where it is most needed is in mid-career labs that do not have multiple R01’s, which in many cases are imploding in the present environment. These are the PI’s who are writing 10 grants to get 1 funded right now. The well-funded empires are doing just fine, and I have not found the PI’s of such labs to be the egalitarian types would would give up a dime so that someone else could keep a lab running.

For ESI’s, this could be an interesting experiment in how to launch successful careers. Many of us who endured the system of the last decade are discouraged and demoralized. Personally, I will never live up to my scientific potential after so many years of wasting time on failed proposals and preliminary results for projects that were never funded.

emphasis added. As if I have to do so.

Do you wonder why the current greybearded and silver haired people who remain powerful in science are so keen to cry over the poor, poor Millennial generation of scientists and wring their hands over the future of science, all the while doing nothing about the present of science?

Because the Boomers (and a few years' worth of pre-War folks) cannot acknowledge what they have done to the Gen X scientists. Some of the charges are as follows.

1) Extended graduate school training from 4 years to 6+. Sure they used all sorts of very truthy sounding excuses about mastering different domains, getting those three publications in CNS journals, the collaborative nature of vertically ascending science, etc. But they accomplished it...and their own successes prove it unneeded.

2) Extended postdoctoral "training". The moved us from where even two years as a postdoc prior to professorial appointment was slightly suspicious (in the early to mid 1970s) to a situation in which two sequential 3-5 year postdocs are viewed as the necessary minimum (just a few years ago, prior to the ESI foofraw). The oldest generation oversaw this.

3) Even during the NIH doubling, they grabbed all the grants and kept beating up the newly appointed GenX scientists with Stock Critiques, sent them around the airport traffic pattern in endless revisions and with "good scores" that were clearly unfundable. Anything to delay entry and preserve their expanding empires.

3) The R29 FIRST was dismantled** but was replaced by a NI check box. It supposedly took the oldster power brokers 10 years to realize was to the benefit of, you guessed it, themselves. I.e. those highly established scientists that simply didn't have NIH funding yet. It took me about 3 hours of my first study section meeting to see this.

4) ...aaaand what do you know? By the time the old guard power brokers "realized" this NI problem, they were able to fix it with a time-limited ESI designation tagged to the time of PhD award, instead of the time of Asst Prof appointment. This conveniently skipped right over the Gen X scientists.

So what did this accomplish? Well, on the trainee end of the screw-job this just meant more time in which a venerated or even hard charging mid career lab head could benefit from the intellectual contributions of the Gen X scientists. Pretty much like intellectual vampires. The crediting system whereby author lines expanded and the senior author got all the glory was refined and elaborated from the 1990s through the Naughties as the NIH budget doubled. The number of "postdocs" supported on research grants soared through the roof. And the new models, conceptual breakthroughs and new theoretical approaches continued to give subsequent grant largesse and subsequent paper / finding laurels to the lab head. While the Gen X scientists continued as postdocs, or were shelled out of the system or manged to get a job but couldn't get funded very easily.

I was there. I know who did the actual work in the labs in my fields of interest. I know the way a finding or paper or model resulted in the lab head having copious funding for a decade and a half, verging on two decades now. I know which of those scientists of my generation failed to make it big. There are a lot of them that will never achieve their promise. A lot who had to bail entirely on the career after what would have been a career-making paper as a trainee, if they were just a generation older. I can point to very few of the Gen X people in my fields of closest interest who have hit mid career with anything like the funding, verve and accomplishment of even some of the more, shall we say, pedestrian*** members of the generation just prior to mine****. Actually, come to think of it, I am hard pressed to point to a single one.

I am not suggesting the older folks who benefited had no right to do so. I am not saying they didn't deserve any credit, nor am I claiming they didn't contribute intellectually.

At all.

I am saying that they (as a generation) arranged things so that they got ALL OF the credit and benefit of the collaborative breakthroughs. And this is not right. They did not suffer a similar fate at the hands of their more-senior colleagues because times were very different. Expansive. Lab sizes were smaller and the trainees were more consistently encouraged to fly away and shine on their own. This is what happened through the 70s and 80s when they were transitioning. And yet they have the nerve to call us riff raff. To question our commitment to science in oh so many ways. To continue to credit themselves for breakthroughs and advances that rested on the intellectual labors of a younger generation that they now disparage.

Some of us are surviving. Yes. This is obvious. Some of us are thriving. Some of us squeaking by on fumes and prayers. Some Most of us yo-yo between these extremes.

As the comment said, however, we will never reach our potential as scientists. Not in the way we witnessed the generation or two before us reach theirs. Not as a generation and not as the vast majority of individuals. Ever. It cannot be recovered.

Do you wonder why we are angry at each and every NIH initiative that comes down the pike that is explicitly designed to skip over our generation of scientists, yet again?

This is why.

__
*yeah. This is as good as it gets.

**for good reason, it had problems

***to be clear, I count myself in this category

****Who managed to get past the noob-abuse and hazing ritual juuuuuuust as the doubling hit stride. This is the generation that managed to land the last few R29s, for reference*****.

*****In truth, those who were never eligible for either R29 or ESI designation makes a much better and tighter demarcation than Gen X versus the Boomers and Millennials when it comes to this stuff. But it is pretty damn inside baseball to use such terms....

Related Reading: Does anybody want to be president? Anyone?

19 responses so far

Sustaining NIH funding then and now: 58% as many Type 2 awards in FY2014

Feb 25 2015 Published by under Fixing the NIH, NIH, NIH Careerism

Datahound has a cool new analysis posted on the distribution of competing continuation R01/R37 awards (Type 2 in NIH grant parlance).

There is one thing that I noticed that makes for a nice simple soundbite to go along with your other explanations to the willfully blind old guard about how much harder the NIH grant game is at the moment.

Datahound reports that in FY 1995 there were 2653 Type 2 competing continuation R01/R37 awards funded by the NIH. In FY 2014 there were only 1532 Type 2 competing continuation R01/R37 grants awarded.

I make this out to be 58% of the 1995 total.

This is a huge reduction. I had no idea that this was the case. I mean sure, I predicted that there would be a big decline in Type 2 following the ban of A2 revisions*. And I would have predicted that the post-Doubling, Undoubling, Defunding dismality would have had an impact on Type 2 awards. And I complained for years that the increasing odds of A0 apps being sent into the traffic holding pattern itself put a kibosh on Type 2 because PIs simply couldn't assume a competing continuation would be funded in time to avoid a gap. Consequently PIs were strategically putting in closely related but "new" apps in say Year 3 of the original noncompeting interval.

But I think if I had been asked to speculate I would have estimated a much smaller reduction.

__
*I can't wait until Datahound brackets this interval so we can see if this was the major effect or if the trend has developed more gradually since 1995.

9 responses so far

NIH clumsily tries to .. [something] ... for grant reviewers

Feb 20 2015 Published by under Careerism, NIH, NIH Careerism

I noticed a funny one in the NIH Guide notices today.

NOT-OD-15-035 Reinforcing Service to the Biomedical Research Community

Yes, yes. I see. "Reinforcement" of a behavior like "Service to the Biomedical Research Community" means increasing the strength or probability of the behavior. So yes, that's good. What are they trying to do here?


Purpose
This Notice gratefully acknowledges, and seeks to reinforce, service to the biomedical research community by recipients of National Institutes of Health (NIH) research funding (see NOT-OD-10-089). Obtaining input from qualified experts across the entire spectrum of the extramural research enterprise furthers diversity of scientific thought, inclusiveness, and breadth of perspectives necessary to evaluate applications in a review process that strives for integrity and fairness. The interdisciplinary, collaborative, and global nature of biomedical research today requires increasingly complex review panels that need both broad and specific expertise in countless topic areas. Thus, the NIH, the biomedical research community, and the general public benefit from the service of NIH-funded investigators and maximize the Nation's investment in biomedical research.

Yes, yes. Very nice. but what are they actually doing to reinforce the behavior?


Policy
The NIH expects principal investigators of NIH supported grants and contracts to serve on NIH peer review groups, when asked. Therefore, the NIH expects grantee institutions and R&D contract recipients to encourage their NIH-funded investigators to serve on NIH peer review and advisory groups. These groups include Scientific Review Groups (or “study sections”) in the initial peer review of grant applications and technical evaluation of R&D contract proposals, National Advisory Boards or Councils (NACs) for second-level peer review, NIH Boards of Scientific Counselors (BSCs) for intramural programs, and Program Advisory Committees (PACs) for initiative development and concept review.

emphasis added.

Okay, so any University with a pulse is already encouraging their PIs to serve on study section. Right? They know about how this will help their bottom IDC line, yes? And if they are discouraging any subset of investigators from serving I imagine it is the Assistant Professors...who the NIH / CSR isn't looking to recruit anyway.

Hmm.

I have a suggestion. Two actually. The first one is hey, if you want to reinforce a behavior, why don't you use the delivery of a rewarding stimulus? I mean sure, you give us reviewers a delay in the submission deadlines, that's cool and all. But obviously the NIH thinks they need something more. How about protection from budget reductions? A couple of extra percentile points on newly competing awards?

No?

Okay, that costs you money, I realize. How about something very cheap with some motivational value? Journals often publish a list of their reviewers at the end of the calendar year and thank them for their service. It's nice. But the NIH can do this one better. Set up a website with a list of reviewers and the number of grants they've been assigned to review. Maybe do it by year too and provide permalinks.

Trust me, academics will eat this up. They will check out how many reviews their buddies are/are not doing and give them a little hell for not matching up around the conference coffee table. They will start linking to their entry from their websites and bragging about it in their P&T documentation.

I wonder. Really, NIH. Do you have anyone making policy that understands people even the tiniest little bit? I am about the opposite of a people person and it took me like two tweets to think of this.

31 responses so far

If you care about the Emeritus NIH Grant...

Feb 20 2015 Published by under Fixing the NIH, NIH Careerism

...you need to take action on the official form.

Click through and get that done.

3 responses so far

"We've seen it all before, this is just a cyclical downturn for the NIH funding"

Popular thought. But it is nonsense.

A close collaborator was recently experiencing this common denial trope from one of the more established type of scientists. The thinking is that

"...sure, things are tough for younger scientists right now but hey, things have been tough before. It's all just a cycle and oh, stop complaining kiddos. We had it hard too."

Here is why it is in error to argue this- the magnitude of the downturn was lesser and it lasted for a shorter duration in those prior "cycles". Let us refer to the infamous Undoubling graph.

Heinig07-NIHbudget-trend.jpeg.jpg

Figure 1. NIH Appropriations (Adjusted for Inflation in Biomedical Research) from 1965 through 2007, the President's Request for 2008, and Projected Historical Trends through 2010.
All values have been adjusted according to the Biomedical Research and Development Price Index on the basis of a standard set of relevant goods and services (with 1998 as the base year). The trend line indicates average real annual growth between fiscal years 1971 and 1998 (3.34%), with projected growth (dashed line) at the same rate. The red square indicates the president's proposed NIH budget for fiscal year 2008, also adjusted for inflation in biomedical research.

The previous downturns in the NIH funding (and you can verify the scientist complaining by looking through old Science magazines, btw) occurred approximately in the late 1960s, the early 1980s* and the early 1990s. I happened to join this career path right around the 1990s downturn and I remember the whining about grant funding quite clearly. That 1990s downturn was what led to the infamous NIH Doubling. The late 1960s downturn led to Congressional action as well. In both cases you can see where the lapse in Congressional interest led to the following episode of downturn. It is here that we should also review the subsequent update on the Undoubling graph, the even more sinister Defunding Graph.
NIHBudget-MAW-edit-497x400
Via Michael White, presumably via John F Sargent, Jr.

It should be emphatically clear to even the casual observer that the magnitude of the decline in the NIH budget and the duration of the downturn prior to the next Congressional rescue differs. Dramatically. Make sure you check the corresponding longitudinal trends in grant success rates. In case you are wondering about the most recent numbers, according to Sally Rockey, the overall RPG success rates for FY 2012-2014 are 17.6%, 16.8% and 18.1%, respectively. Things are most emphatically not good for the kids these days.

These are the facts. We can argue until the cows come home over how and why various up and down cycles have occurred. We can dispute whether Congressional appropriations intended to rescue the NIH extramural community do harm, good, a balance of the two and what this means for the future.

It is not optional, however, to act like the present downturn is of the same magnitude or impact as the prior ones.
__
*"I remember multiple study section rounds in which nothing ended up getting funded" --a senior colleague

98 responses so far

Generational privilege under the NIH system in one easy video

Feb 17 2015 Published by under Fixing the NIH, NIH, NIH Careerism

Another key bit of information to which I frequently refer when describing generational privilege is depicted in this video from the NIH.

Facts matter.

16 responses so far

Repost: More data on historical success rates for NIH grants

Feb 17 2015 Published by under Fixing the NIH, NIH, NIH Careerism, NIH funding

Our recent discussion of topics related to the Emeritus Award being considered by the NIH powers that be has been robust. I, of course, have been reminding one of the target demographic scientists that she and her generation have had a pretty good run under the NIH system. It seemed like a good moment to remind everyone that there are data upon which to base our understanding of how difficult it has and has not been for various scientific generations. Time to repost an older blog entry.

This was first posted 11 July 2012.


Thanks to a query from a reader off the blog and a resulting request from me, our blog-friend microfool pointed us to some data. Since I don't like Tables, and the figure on the excel file stinks, here is a different graphical depiction:

The red trace depicts success rates from 1962 to 2008 for R01 equivalents (R01, R23, R29, R37). Note that they are not broken down by experienced/new investigators status, nor are new applications distinguished from competing continuation applications. The blue line shows total number of applications reviewed...which may or may not be of interest to you. [update 7/12/12: I forgot to mention that the data in the 60s are listed as "estimated" success rates.]

The bottom line here is that looking at the actual numbers can be handy when playing the latest round of "We had it tougher than you did" at the w(h)ine and cheese hour after departmental seminar. Success rates end at an unusually low point...and these numbers stop in 2008. We're seeing 15% for R01s (only) in FY2011.

Things are worse than they've ever been and these dismal patterns have bee sustained for much longer. If we look at the ~30% success rates that ruled the day from 1980-2003, the divergence from the trend from about 1989 to 1996 was interrupted in the middle and, of course, saw steady improvement in the latter half. The badness that started in FY2004 has been 8 unrelieved Fiscal Years and shows no sign of abatement. Plus, the nadir (to date) is much lower.

Anyone who tries to tell you they had it as hard or harder at any time in the past versus now is high as a kite. Period.

Now, of course, it IS true that someone may have had it more difficult in the past than they do now, simply because it has always been harder for the inexperienced PIs to win their funding.

RPGsuccessbyYear.png
source
As we know from prior posts, career-stage differences matter a LOT. In the 80s when the overall success rate was 30%, you can see that newcomers were at about 20% and established investigators were enjoying at least a 17%age point advantage (I think these data also conflate competing continuation with new applications so there's another important factor buried in the "Experienced" trace.) Nevertheless, since the Experienced/New gap was similar from 1980 to 2006, we can probably assume it held true prior to that interval as well.

3 responses so far

A tiny bias goes a long way when it comes to grant review

Feb 11 2015 Published by under Fixing the NIH, NIH

From ScienceInsider:

Now, a new computer simulation explores just how sensitive the process might be to bias and randomness. Its answer: very. Small biases can have big consequences, concludes Eugene Day, a health care systems engineer at the Children's Hospital of Philadelphia, in Research Policy. He found that bias that skews scores by just 3% can result in noticeable disparities in funding rates.

T. E. Day, The big consequences of small biases: A simulation of peer review, 2015, Research Policy [epub ahead of print 28 Jan] [Publisher Site]

from the paper Abstract:

When total review bias exceeds 1.9% of grant score, statistically significant variation in scores between PC and NPC investigators is discernable in a pool of 2000 grant applications. When total review bias exceeds 2.8% of total grant score, statistically significant discrepancies in funding rates between PC and NPC investigators are detectable in a simulation of grant review.

Day generated a Preferred Class of applications and a NonPreferred Class of applications and ran a bunch of 3-reviewer scenarios with and without reviewer bias against the NPC applications. As far as I can tell the takeaway conclusion about funding here refers to a situation in which the effective payline is 10%. You will immediately grasp that NIH grant review was a strong contributor to the model parameters.

I will admit I am only able to grasp the main points here and I am in no way able to evaluate the nitty gritty.

But it appears to have a very strong message. Namely, that our introspections that "well, if there is bias it is very tiny so we don't have to be worried about it" needs to change.

There is something even scarier in this paper. From the Discussion:

The threshold level of bias in this environment seems to be 2.8% of the total possible score of the grant; this is the level at which the 95% CI of the odds ratio “kisses” 1.00. This represents a single reviewer with a bias of 0.75 points (or three reviewers each with biases of 0.25 points), which is less than half (44.4%) of the standard deviation in a single reviewer’s score. What this suggests is that levels of bias which are sub-noise – that is, that are dramatically less detectable than normal variation in reviewer scores – are sufficient to substantially bias the number of funded applications in favor of preferred investigators.

RIGHT???? The bias can be of smaller effect size than many "normal" sources of variability in scoring that we accept as the resolution of the system. And it still leads to a statistically significant bias in funding outcome.

We are talking in recent days about bias in favor of highly established, older scientists. It has been longer but the Ginther report indicating disparity of grant review outcome for African-American PIs is clearly relevant here.

What this simulation cannot do, of course, is to model the cumulative, iterative effects of review bias. Namely, the way that selection of PC applications for funding has a tendency to increase the bias in the reviewer pool, since those beneficiaries become the next reviewers. Also, the way that over the long haul, disparity of the first award can lead to actual quality differences in the subsequent applications because PI #1 had the money to pursue her science and PI #2 did not have as easy of a time generating data, publishing papers and recruiting postdoc talent.

5 responses so far

Thought of the Day

Feb 11 2015 Published by under Grant Review, Grantsmanship, NIH Careerism

I hate when I review grant proposals that are good, but clearly have been made pedestrian and conservative through the school of hard knocks. There is so much awesome that could be done by these people. It is so clear to me what the really high impact version of this grant should look like. (Not having any illusions about my own unique brilliance, I assume they could see it too. )

But the review realities batter PIs down into a defensive crouch, worried that if they step too far past their Preliminary Data or established expertise they will get crushed.

Because, of course, they would get crushed.

Sometimes I wish I were the Boss of Science more than other times.

28 responses so far

Older posts »