Is this affirmative action for younger investigators or something else?

Nov 11 2008 Published by under NIH Budgets and Economics

A couple of recent pieces in Science will be of interest to those of you concerned about the demographics of the NIH-funded independent scientist pool [h/t: writedit]. The first one details the prior NIH head Elias Zerhouni's attempt to make his administration's support for New Investigators permanent NIH policy. The second News Focus details interviews conducted with NIH funded investigators over the age of 70.
Much of this is familiar territory but I was struck by the use of the term "affirmative action" to describe the NIH's attempt to keep funding for new awardees viable.


First up, what did Zerhouni attempt to correct?

In 2003, when NIH's budget stopped growing, the situation "collapsed," Zerhouni says: The number of R01-like research grants (known as R01 equivalents) going to first-time investigators slipped to 1354 in 2006, the lowest level in 9 years.

In 2007, he set a target of funding 1500 new-investigator R01s, based on the previous 5 years' average. Some institutes struggled to reach their targets, NIH officials say. At the National Institute of Neurological Disorders and Stroke, for example, the shift to new grants meant that only 9% to 10% of established investigators with strong peer-review scores received funding, whereas 25% of comparable new investigators did, says NINDS Director Story Landis. She maintains, however, that "it's not as though a huge number of investigators lost out."

[program directors] came on board when NIH noticed a change in behavior by peer reviewers. Told about the quotas, study sections began "punishing the young investigators with bad scores," says Zerhouni. That is, a previous slight gap in review scores for new grant applications from first-time and seasoned investigators widened in 2007 and 2008, Berg says. It revealed a bias against new investigators, Zerhouni says.

FirstTimeAwardeeFigNov08Scimag.gif
Credit: NIH
Okay, I applaud them all for noticing and getting on board and all that but someone's missing the point. New Investigator awards were going down 2004, 2005 and 2006 before the heavy hand of NIH programmatic pickups descended. It is the "slight-gap" that revealed the bias against new investigators! The widening gap after the news about New Investigator policies began getting around just puts a fine point on the main issue. Study section behavior is what needs to be directly addressed and the application of BandAid solutions at the Program level is making things worse!
How so? The use of the term "affirmative action" tells us all we need to know. This brings in the GoodOldWhite(haired)Boyz dogwhistle to assert that New Investigators are rightfully receiving poorer scores and need an artificial leg up. That, my friends, is what most people holding the reins of power in academic science hear when the term affirmative action is brought up. What people are not hearing in any meaningful way is the term bias. Which implies, correctly in my view, that irrelevant considerations of career status, prior R01-funding, etc are being applied to diminish highly meritorious proposals in favor of equally- or less-meritorious proposals from those on the inside.
From the News Focus:

Biochemist Carl Frieden, 79, of Washington University in St. Louis, Missouri, also says he will let peer reviewers tell him when it's time to close his lab. He says that although he's sympathetic to the struggles of young scientists, funding should be based strictly on scientific merit, not age. "We're the only profession judged by our peers every 3 to 5 years. If older scientists can pass that trial, I'm comfortable with that," he says.

Right. Because the ImmaculatePeerReview is a perfect judge of scientific merit...sigh. Oh, well, some of those interviewed understand:

For molecular geneticist Robert Wells of Texas A&M University in College Station, who is just 70 but gave up his NIH grant and has been closing his lab for the past 2 years, the difficulties young investigators face were foremost in his thoughts. "If I and other old birds continue to land the grants, the [young scientists] are not going to get them," he says. He worries that the budget won't be able to support the 100 people "I've trained ... to replace me."

Back to the first piece on Zerhouni's parting gift I note something interesting that has been trickling in the back of my mind ever since I started paying attention on study section.

NIH will also fine-tune its policy to tilt it in favor of early-career scientists. The goal is to adjust for the recently discovered fact that only about 55% of investigators who receive their first NIH grants are at an early stage of their career. The rest are scientists who had been funded by other agencies or came from NIH's intramural program or from Europe after being forced to retire there. "It was embarrassing" to realize, for example, that the new investigators included two department chairs with Veterans Administration funding, Landis says.


There is an awful lot of text in this post, so I am going to break it up with something totally hot.Redline_Conquest_Team-425.jpg
The Redline Conquest Team, reviewed here. $899.99 frameset and $1,159 Ultegra build kit from Excel Sports.

Ahem.
Funny, I noticed this trend after about two rounds of study section. Namely that the New Investigator pool included many quite-senior investigators who had enjoyed quite substantial support from nonNIH sources in their careers. Surprise, surprise, proposals from these "New Investigators" did better than did those of recently transitioned New Investigators. I still find it mind-boggling what properties of the NIH grant review / award system seem obvious to applicants and study section reviewers and yet is a "surprise" to the people who actually hold the raw data that would allow them to perform definitive analyses.
This is why I continue to exhort you all, from grad student to seasoned investigators, to take every opportunity that arises to talk NIH grants shop with any NIH representative you come across.

At this little meeting there was a nice dinner session on obtaining NIH grants. The session was hosted by two well funded extramural researchers (albeit in clinical areas) and one program officer. They all gave excellent talks although this particular dinner was not very well attended (my fellow junior investigators, WTF are you thinking skipping out on these things?).

Word up, jp!!
Final shot from the Zerhouni piece:

some scientists may be uneasy about the cost, says Howard Garrison, spokesperson for the Federation of American Societies for Experimental Biology in Bethesda, Maryland: "Every time you give a leg up to a young investigator, you're pushing someone off the edge of the cliff."

A total and scurrilous lie. The older investigators are disproportionately (compared to the genuine New Investigators) likely to hold multiple grants and to have hard money jobs. The idea that each award to a New Investigator pushes an established investigator "off the cliff" is laughable. Please challenge this silly notion if you happen to run across it DearReader. Let us turn this statement around, shall we?
"Each time you give yet another award to an established investigator who has full salary support and/or other research support, you are kicking a transitioning investigator back off the edge of the towering cliff she has just surmounted."
I like my version better.

31 responses so far

  • Dr. Isis says:

    First and foremost, DM, that bike is totally hot and I am pleased to see you picking up on my excellent blogging stylings. Do you think, if I bought that bike, I could get it outfitted so that I could slip my 4" heels into the pedals?
    Second, the part of me that longs for things like equality and a level, playing field agrees that grants should be scored on the quality of the science. But then I realize that is total crap and give myself the spanking I deserve. The truth is, a well-established investigator with 15 lab members and years of pilot data is going to be able to turn out a sounder grant than a 1 or 2 year out asst prof who's just trying to get his shit together.
    But, I long to shove an old dude off the cliff.

  • Becca says:

    1) The bike is totally hot.
    *whispers* Don't tell Dr. Isis, but I'd rather have the bike than all the pointy toed shoes in her closet.
    2) *gratutiously controversial silly comment* I blame the physiologists. Particularly the ones that work on heart disease. Do we really need to keep old white USian males alive for another 20 years of eating McDonalds and oppressing everyone else? Geeze. The only people worse than them are the ones trying to find ways to cure heroin addiction. You want to cure heroin addiction? Let natural selection take it's course. */silly*
    3) As a tangential but somewhat related question, is there any way someone who is listed on CRISP with an R01 can legitimately be considered a "new investigator" for another application? How carefully does NIH check if someone filling out the "new investigator" bubble really meets the official criteria?

  • Dr. Zeek says:

    Maybe the recent experience we had in our lab was a fluke-- my stodgy PI, an innovator in the field, blah-blah-blah, submitted a poorly written grant (i.e. poorly in the terms of just overall writing, no cohesiveness, etc.-we have a lot of collaborations and study only certain "parts," shall we say, of a project--there are very few labs that do what we do). He submitted a renewal grant, and low-and-behold, the grant he has held for 20+ years scored around a 50%! It is not enough, anymore, just to have a big-huge name on the grant.
    But I will tell you it helps. The second R01 that my PI holds was up for renewal two months later. Another lab member and I wrote it, with input from our PI and collaborator's (more along the lines of change this sentence, add a comma here). Anyways, I digress (sorry, can't post any totally hot picture here). We submitted scored an 11% and the comments were riddled with "project too ambitious, but because of big name PI, we think you can do it." It was good science, but had either of us who really wrote it been the PI, it would have been shot down.
    So where does this put us? The double-talk, we won't fund you if you have been around since the first ice-age, but we don't take that into account, but having a big name PI on your grant helps'.AGGHH...its enough to make your head spin. I think you are right that it comes down to the behavior in the study sections (although never having the "pleasure" of sitting in one, I could be talking out of my ass..) . But seriously, if peer-review and study sections are the underlying problem, then how in the hell do we fix things?

  • The only people worse than them are the ones trying to find ways to cure heroin addiction. You want to cure heroin addiction? Let natural selection take it's course.

    I'm sure DrugMonkey knows more than I do, but my understanding is that people die from heroin addition mainly because of the fact that it is illegal: you can't get heroin of known potency, you can't get clean needles, and you have to live an underground illegal lifestyle. If heroin of precisely known potency were made freely available to heroin addicts, very few would die prematurely, correct?

  • DrugMonkey says:

    Isis and Dr. Zeek: The point here is not that any type of status is an automatic guarantee of funding. Highly established types get proposals killed and occasionally an untried n00b gets her award. We are talking most specifically about what happens with the pool of highly meritorious proposals that extends from about 5%ile to 30%ile. Most of these are pretty well written and have good scientific ideas. The factors that make the difference are not directly related to objective theoretical merit.
    is there any way someone who is listed on CRISP with an R01 can legitimately be considered a "new investigator" for another application?
    The only way this can occur at the study section phase is if the applicant has an award pending that funds just prior to the study section meeting. Since reviewers do not habitually CRISP the applicants people might not notice. At the actual award stage, Program has to be notified just prior to award (via the 'Other Support' page) of the PIs funded and pending awards. So I don't see how this could slip through the cracks.
    Post-review and prior to funding would a Program Officer do some checking in the NIH database to see if an applicant they are considering for a New Investigator pickup still qualifies? I'd hope so, but who knows.
    if peer-review and study sections are the underlying problem, then how in the hell do we fix things?
    I am one that believes there is a LOT of room for reviewer education and instruction. The NIH / CSR likes to keep the hands off when it comes to telling reviewers their job....except for certain things. It can be maddening as a reviewer. In the absence of explicit instruction, reviewer behavior is going to be a random mishmash of personal beliefs, personal experiences with the grant game and emerging properties of the specific study section.
    [added in proof] It is also my assertion that we should leverage our best available solutions for any situation of individual bias. That solution is the competition of opposing biases. This is at the heart of diversity on decision making panels. In this case, I think that the explicit non-representation of those who do not yet hold NIH grants is a big problem. I would like to see greater participation from assistant professors in their first few years and even consider senior post-docs for review service. With the usual anecdote of one panel, people tend to sympathize with PIs that are like themselves IME...

  • I am one that believes there is a LOT of room for reviewer education and instruction. The NIH / CSR likes to keep the hands off when it comes to telling reviewers their job....except for certain things. It can be maddening as a reviewer. In the absence of explicit instruction, reviewer behavior is going to be a random mishmash of personal beliefs, personal experiences with the grant game and emerging properties of the specific study section.

    I saw this problem in action while serving on a K99 review panel. One of the panel members asked a question of the NIH staff present that was on many of our minds: Is the K99 meant to reward outstanding and outstandingly productive senior post-docs--all of whom would surely transition to independence equally smoothly without the K99--or is it meant to "lift up" some less-productive post-docs who, nonetheless show promise, would otherwise possibly have trouble transitioning to independence.
    The NIH staff present essentially refused to answer the question, and told us that it was a matter for the conscience of each reviewer to interpret the K99 funding opportunity announcement.
    Sheesh!

  • BikeMonkey says:

    Do you think, if I bought that bike, I could get it outfitted so that I could slip my 4" heels into the pedals?
    That should pose no serious problems Isis...

  • DrugMonkey says:

    One of the panel members asked a question of the NIH staff present that was on many of our minds:
    Exactly. We get similar all the time in which people will ask things about how we are supposed to be treating "Significance" (after a CSR instruction to concentrate on this criterion without any definition!!!), how to review R21s or R03s or R15s....
    So when you hear me complaining about reviewer behavior it is not exclusively an indictment of individual reviewers. People are for the most part able to do the job they are asked to do. Prioritize "track record"? can do. Prioritize "promise" and "investment in the future"? no problem. But in the absence of instruction they are free to express whatever random (or nonrandom) inputs are at hand...

  • qaz says:

    1. There is a myth that new investigators are where the trouble is. This is what the K99 was designed to attack. In fact, the data (apparently) do not support that myth. After the big "what's-wrong-with-NIH" symposium at SFN a few years ago, I stayed afterwards to talk to some of the heads. Nora Volkow (head of NIDA) said that they had done a study and found that the real crash comes at the *first renewal*. This is where you no longer get the new investigator boost. Of course, this is also where you are likely facing your tenure decision. Apparently, most of the scientists-to-be leave science at the first renewal. And they get declared "failures" for it.
    2. Comrade PhysioProf. Thank you for this comment. NIH has abdicated its role in selection to study sections. This makes things very unstable (as grant review varies dramatically from study section to study section). It also forces tactical solutions (the "conscience of each reviewer") to strategic problems ("what's the goal of the K99?").
    3. Finally, I want to know what's going to happen to those of us in the lost generation between the damn baby boomers and the next generation that will be gloriously brought help when the baby boomers finally leave. Those of us who have been trapped by this famine with no help. We're no longer "new investigators", but we're still significantly behind the eight-ball on lab funding without either the big-dog or the n00bie help.

  • whimple says:

    One solution is to break up the toxic cultures in the study sections. My vote goes to a "jury pool" type of system, where all eligible reviewers for a given study section are semi-randomly (not allowing one person to be picked too many times) assigned to make up the study sections for each and every review cycle.
    The small loss in terms of "continuity of review", I think would be vastly outweighed by spreading the review burden more evenly on the potential reviewer pool, and by hurting the ability of people to target their apps to study sections populated with friends and individuals owing favors to the applicant.
    As pointed out by others, firm quotas on New Investigator grants funded is just a delaying tactic that screws the next older cohort of investigators.

  • DrugMonkey says:

    qaz and whimple, I hear what you are saying. Really I do. But I think we need to be fair and not just perpetuate the sins of the past for the same selfish reasons.
    Who is going to take one for the team here?
    Is it all on the youngsters? Or are we going to share the pain?
    Are we just mouthing platitudes about the future of science when we really just mean our own personal grant success futures?

  • qaz, Comrade PP,
    I'm glad to see that other people think NIH has abdicated a big chunk of programmatic responsibility to the study sections.

  • neurolover says:

    "As pointed out by others, firm quotas on New Investigator grants funded is just a delaying tactic that screws the next older cohort of investigators."
    I think this doesn't just screw the "next cohort." I think giving a leg up to young/new/whatever investigators can hurt *them* if it merely delays the time when they have to write extremely competitive grants. For those with no money (i.e. soft money folks, underfunded folks) this may be necessary, because they simply can't be competitive without the leg up, so it gives them an opportunity. For those who have substantial start up funds, for example, a leg up can hurt, because it delays the time they need to write a grant properly, until the dreaded tenure decisions.

  • neurolover says:

    I think the biggest help NIH could provide to NI's is aggressive advise on how to write grants, preferably to the actual study sections likely to review their grants. And, better advise to study sections on how to judge grants, including their tendencies to use bias in making judgments.

  • DrugMonkey says:

    I think the biggest help NIH could provide to NI's is aggressive advise on how to write grants, preferably to the actual study sections likely to review their grants.
    Bzzzzzzzt!!! Stop with this stuff. Yes it is true that on average the less-experienced could stand some education on how to write better proposals. But this is not the fundamental problem here. The fundamental problem is the really great and well written proposal from a NI that is not evaluated fairly. That's the harder fix.
    And, better advise to study sections on how to judge grants, including their tendencies to use bias in making judgments.
    This is what I'm saying. We need to de-emphasize certain properties such as "track record" and emphasize other properties which have a chance of being fair. for example even though we might not like an obsession with grantspersonship, a well-crafted grant is something that younger and newer investigators are capable of constructing, whereas by definition creating a 30yr track record is not possible.

  • msphd says:

    First of all, I could care less about the bike.
    Thanks for this post. I had no idea that there were senior people lumped in with "new" investigators. That's fucking hilarious.
    And I totally agree that grants are not evaluated fairly. But that brings us back to the problem of whether grants should be reviewed without any names attached, which would not be compatible with an affirmative-action type solution.
    I fully plan to push as many old geezers off the cliff into retirement as I possibly can. No mercy, you old fuckers.

  • Dr. Isis says:

    1) The bike is totally hot.
    *whispers* Don't tell Dr. Isis, but I'd rather have the bike than all the pointy toed shoes in her closet.
    2) *gratutiously controversial silly comment* I blame the physiologists. Particularly the ones that work on heart disease. Do we really need to keep old white USian males alive for another 20 years of eating McDonalds and oppressing everyone else? Geeze. The only people worse than them are the ones trying to find ways to cure heroin addiction. You want to cure heroin addiction? Let natural selection take it's course. */silly*

    Et tu, Becca???
    You know what I think would fix this problem? A $700 billion bailout to the NIH. I hear the gov't has half of the original bailout just sitting around, chillin.'
    BikeMonkey, that is totally hot.

  • becca says:

    Dr. Isis- yeah, I'll probably end up working on some basic research that is totally hott as science, but that'll do something utterly pointless, like cure baldness. Currently I'm studying malaria as penance for previously working in a lung cancer lab.
    I'm absolutely with you on the stimulus package!
    Given how badly the two-body problem afflicts so many NIH-funded female scientists, it will surely trikle down to a greening auto industry.
    ...and I didn't mean it about the shoes, I swear! Or, I did, but that's only because the shoes in Dr. Isis's closet would fall off my freakishly teensy munchkin feet.

  • DSKS says:

    CPP said,
    Is the K99 meant to reward outstanding and outstandingly productive senior post-docs--all of whom would surely transition to independence equally smoothly without the K99--or is it meant to "lift up" some less-productive post-docs who, nonetheless show promise, would otherwise possibly have trouble transitioning to independence."
    What was the feeling about the importance of the proposed training plan in your study section? I think I underestimated this (K99 proposal = FAIL). My research proposal, which had a decent body of prelim data, received positive reviews. The smack downs focused on insufficient justification for further training, for which the research proposal was put forth by one reviewer as evidence. Get a job and submit an R01, was the general tone. Assuming, of course, the reviewers were being completely forthright, and not simply trying to let a n00b down easy.

  • TeaHag says:

    Interestingly, I understand that in the UK and Ireland, at least for Wellcome Trust funding, applicants over the age of 60 are warned that their funding will not extend beyond their mandatory retirement age/approved extension. Doesn't matter how good they are!

  • rotifer88 says:

    My concern with this policy is the separate evaluation of Early Stage Investigator applications. An Early Stage Investigator is defined as someone within 10 years of their Ph.D. This is going to severely penalize anyone (like me) who took time away from the academy after their degree either to work in industry, to raise children or to care for aging parents. My grants will be scored against senior researchers while most of my Assistant Professor colleagues will be competing against each other. It seems like this policy will disproportionately penalize women scientists as they are more likely to have had time away from the academy for family reasons. I would argue that an Early Stage Investigator should be defined as an untenured faculty or research faculty who has never had an RO1. The ten year post-Ph.D. limit seems arbitrarily punative against some groups.

  • neurolover says:

    They say they're going to correct for "time off" the track, but I've haven't heard how, yet.
    I agree that "untenured faculty who have never previously held and RO1 seems like a good definition", and somewhat similar to the NSF CAREER definition (an untenured faculty, up until they get promoted, and you get 3 shots). NSF doesn't allow research faculty to apply, though -- you have to be on the tenure track.

  • Nic says:

    Sorry I'm new to the party. Would it be possible to have an entirely anonymous review of all grants? I suppose if pubs are a part of the submission, journal name and date could be sufficient?
    Are you a CX fan? Or do you race CX?
    I've been thinking of dropping some coin on this.

  • neurowoman says:

    Where does the SRA (scientific review adminstrator) come into all this?
    Thanks for the tips - really great post. Another reason to double check your NRSA or funding info on eRA Commons- your PO can actually change! I plan on going to introduce myself to the new PO on my current grant...

  • neurowoman says:

    oops -sorry DM, that last comment was meant for the SNF 2008: Put NIH row on your Itinerary topic- feel free to move or delete....what can i say it's 2 am and I'm working on my itinerary..

  • neurolover says:

    "Would it be possible to have an entirely anonymous review of all grants?"
    People always suggest this as a solution to bias, and in my field, it has no chance at all of working. The pool of people who could propose the grant is so small (

  • mxracer652 says:

    I got lost after someone mentioned cross.
    Nic,
    I have the 08 frame/fork, if you're tall, you may want to test shoulder it. The compact frame makes it difficult for me (6'-1") to get through the triangle.

  • BikeMonkey says:

    Awesome. A genuine off-topic thread jacking! Compact frames suck anyway but for cross they are right out.
    Nic, dunno if you are planning for racing or general use but if for general use definitely look around for something disc-brake capable

  • Nic says:

    I am short and I was worried about the compact geometry being too small, but my one trial with the bike worked out well. I am currently on a borrowed Redline Conquest, which I do like, but I have a toe overlap issue that makes me a little nervous on hairpin turns.
    I love disc brakes; I have them on my commuter and mtb. The lack of disc tabs is a downside but I am planning on doing at least 2 UCI races next year so I'm going to go with the cantis (and yes that rule is incredibly silly).

  • msphd says:

    yes, there's no way to do anonymous review right now because there's no way to verify that the pubs are real, which is critical if it's part of the evaluation.
    I guess it's possible people could fabricate pub names and dates now, too, but probably harder? Does anyone check?
    I know of plenty of examples where people are multiply funded to do the same project by different agencies... nobody checks for that. As long as the titles are different, people get away with it.
    No, I've proposed before that grants should be based on the proposal only, not on past performance or reputation, etc. But it would mean proposals would have to have preliminary data and be written in a way that they could stand alone... I'm sure it's too radical a notion to ever actually happen!

  • I've been meaning to comment for a few days so this may seem out of the groupthink right now: in my Forrest Gump of a life, I had the chance to get some personal time with financier, philanthropist and prostate cancer survivor, Michael Milken, after a talk of his three summers ago. He is a very strong advocate of the new/junior investigator and supports review reform - from my blog notes back then:
    . . .Similarly, he has observed over the last 30 years that while soliciting the advice of Nobel laureates is invaluable, actual financial support of scientists is most cost-effective and return-on-investment is greatest when provided to those in their late 20s, 30s, and early 40s. There is no need for a senior scientist to hold eight R01 grants.
    The NIH grant review process has grown too cumbersome, inefficient, and fails to reward true innovation and risk-taking. Compare the 25-page NIH R01 application with its nine-month turnaround (if funded on the first cycle) to the Prostate Cancer Foundation (and others) 5-6 page proposal and 60-90 day submission-review-award notification timeline.

Leave a Reply