Archive for the 'NIH Careerism' category

More evidence of the generational screw job in NIH grant award

Sep 02 2016 Published by under NIH, NIH Budgets and Economics, NIH Careerism

ScienceHound has posted a new analysis related to the NIH budget and award policy. He's been beavering away with mathematical models lately that are generally going to be beyond my ability to understand. In a tweet however, he made it pretty clear.

As expanded in his blog post:

The largest difference between the curves occurs at the beginning of the doubling period (1998-2003) where the model predicts a large increase in the number of grants that was not observed. This is due to the fact that NIH initiated a number of larger non–RPG-based programs when substantial new funding was available rather than simply funding more RPGs (although they did this to some extent). For example, in 1998, NIH invested $17 million through the Specialized Center–Cooperative Agreements (U54) mechanism. This grew to $146 million in 1999, $188 million in 2000, $298 million in 2001, $336 million in 2002, and $396 million in 2003. Note that the change each year matters for the number of new and competing grants that can be made because, for a given year, it does not matter whether funds have been previously committed to RPGs or to other mechanisms.

This interval of time, in my view, is right around when the first of the GenXers were getting (or should have been getting) appointed Assistant Professor. Certainly, YHN was appointed in this interval.

Let us recall a couple of graphs. First, this one:

The red trace depicts success rates from 1962 to 2008 for R01 equivalents (R01, R23, R29, R37). Note that they are not broken down by experienced/new investigators status, nor are new applications distinguished from competing continuation applications. The blue line shows total number of applications reviewed...which may or may not be of interest to you. [update 7/12/12: I forgot to mention that the data in the 60s are listed as "estimated" success rates.]

Ok, Ok, Not much to see here, right? The 30% success rate was about the same in the doubling period as it was in the 80s. Now view this broken down by noobs and experienced investigators.
RPGsuccessbyYear.png
source

As we know from prior posts, career-stage differences matter a LOT. In the 80s when the overall success rate was 30%, you can see that newcomers were at about 20% and established investigators were enjoying at least a 17%age point advantage (I think these data also conflate competing continuation with new applications so there's another important factor buried in the "Experienced" trace.) Nevertheless, since the Experienced/New gap was similar from 1980 to 2006, we can probably assume it held true prior to that interval as well.

Again, first time applicants had about the same lack of success in the 80s as they did in the early stages of the doubling (ok, actually a few points higher in the 80s). About 20%. Things didn't go severely into the tanker for the noobs until the end of the doubling around 2004. But think of the career arc. A person who started in the 80s with their first grant jumped up to enjoy 30% success rates and a climbing trend. Someone who managed to land a five year R01 in 2000, conversely, faced steeply declining success rates just when they were ready to get their next grant 4-5 years later.

This is for Research Project Grants (R01, R03, R15, R21, R22, R23, R29, R33, R34, R35, R36, R37, R55, R56, RC1, P01, P42, PN1, U01, U19, UC1) and does not refer to the Centers or U54 that ScienceHound discussed. Putting his analysis and insider explanation (if you don't know, ScienceHound was NIGMS Director from 2003-2010) to work, we can assume that these RPG or R01-equiv success rates would have been much higher during the doubling, save for the choice of NIH not to devote the full largesse to RPGs.

So. Instead of restoring experienced investigator success to where it had been during the early 80s and instead of finally (finally) doing something about noob-investigator success rates that had resulted in handwringing since literally the start of the NIH (ok, the 60s anyway) the NIH decided to spend money on boondoggles.

The NIH decided to assign a disproportionate share of the doubling to the very best funded institutions and scientists using mechanisms that were mostly peer reviewed by....the best funded scientists from the best-funded institutions. One of the CSR rules, after all, is that apps for a given mechanism should be reviewed mostly by those who have obtained such a mechanism. You have to have an R01 to be in a regular R01-reviewing panel and P50/P60/P01 are reviewed mostly by those who have been funded by such mechanisms.

One way to look at this is that a lot of the doubling was sequestered from the riff-raff by design.

This is part of the reason that Gen X will never live up to its scientific potential. The full benefit of the doubling was never made available to us in a competitive manner. Large-mech projects under the elite, older generation kept us shadowed. Maybe a couple of us* shared in the Big-Mechanism wealth in minor form but we were by no means ready to make a play to lead them and get the full benefit. Meantime, our measly R01 applications were being beat up mercilessly by the established and compared unfavorably to Senior PI apps supported by their multi-R01 and BigMech labs.

The story is not over.

Given that I grew up as a scientist in this era, and given that like most of us I was pretty ignorant of longitudinal funding trends, etc, my perception was that a Big Mech was...expected. As in eventually, we were supposed to get to the point where not just the very tippy-top best of us, but basically anyone with maybe top-25% verve and energy could land a BigMech. Maybe a P01 Program Project, maybe a Center. The Late-Boomers felt it too. I saw several of the late Boomers get into this mode right as the badness struck. They were semi-outraged, let me tell you, when the nearly universal Program Officer response was "We're not funding P01s anymore. We suggest you don't submit one.".

AYFK? For people who were used to hearing POs say "We advise you to revise and resubmit" at the drop of a hat and who had never been told by a PO not to try (with a half decent idea) this was quite surprising. Especially when they looked at the lucky ducks who had put their Big Mechs together just a few years before....well there was a lot of screaming about bias and unfairness at first.

P01s are relatively easy for Program to shut down. As always, YMMV when it comes to NIH matters. But in general, I'd say that P01s tended to be a lot more fluid** than Centers (P50/P60). Once a Big Hitter group got a-hold of a Center award, they tended to stay funded. For decades. IME, anyway. or in my perception, more accurately.

Take a look at the history of Program Projects versus Centers in your field / favorite ICs, DearReader and report back, eh?

Don't get me wrong. There is much to like about Program Projects and Centers. Done right, they can be very good at shepherding the careers of transitioning / new scientists. But they are profoundly undemocratic and tend to consolidate NIH funding in the hands of the few elite of the IC in question. Often times they appear to be less productive than those of us not directly in them would calculate "should" happen for the the same expenditure on R01s. Such complaints are both right and wrong and often simultaneously when it comes to the same Center award. It is something that depends on your perspective and what you value and/or predict as outcome.

I can think of precisely one GenX Center Director in the stable of my favorite ICs at the moment. No doubt there are more because I don't do exhaustive review and I don't recognize every name to put to a face right off if I were to go RePORTERing. But still. I can rattle off tons of Boomer and pre-Boomer Center Directors.

It goes back to a point I made in a prior post. Gen X scientists were not just severely filtered. Even the ones that managed to transition to faculty appointments were delayed at every step. Funding came harder and at a delay. Real purchasing power was reduced. Publication expectations went up. We were not ready and able to take up the reins of larger efforts to anywhere near the same extent when we approached mid career. We could not rely upon clockwork schedules of grant renewal. We could not expect that a high percentage of our new proposals would be funded. We did not have as extensive a run of successful individual productivity on which to base a stretch for BigMech science.

And this comes back to a phenomenon ScienceHound identifies. The NIH decided*** to put a disproportionate share of the doubling monies into Centers rather than R01s for the struggling new PIs. This had a very long tail of lasting effects.

__
*I certainly did.

**Note: The P01 is considered an RPG with the R01s, etc, but Centers are not. There is some floofraw about these being "different pots of money" from an appropriation standpoint. They are not directly substitutable in immediate priority, the way I hear it.

***Any NIH insiders that start in on how Congress tied their hands can stop before starting. Appropriations language involved back and forth with NIH, believe me.

18 responses so far

NIH sued for promotion bias against women in the Intramural Research Program

Aug 29 2016 Published by under Intramural Research Programs, NIH, NIH Careerism

via Lenny Bernstein at the Washington Post:

What Bielekova doesn’t have, at age 47, is tenure, the coveted guarantee of recognition, job security and freedom to pursue controversial ideas that is critical to long-term success in an academic career. She was not put forward as a candidate for the second time last year, despite a positive recommendation from a panel of outside experts who reviewed her qualifications.

To me the kicker is this part. NIH intramural is weird the way they have big deal lab heads and a lot of career scientists under them that would be standard tenure rank folks elsewhere. So when the big deal head dies or retires it is always a little weird. Do they hand the lab to one of the folks already there? And boot the rest? Or do they spawn off a couple of new jobs? or find homes for people in other big-deal groups?

Bielekova alleges retaliation and discrimination based on gender after what she describes as a “power struggle” following the retirement of her mentor, who was chief of the neuro-immunology branch. She said male scientists were provided numerous advantages in the aftermath and that she has been harmed by groundless accusations from male colleagues of unprofessional conduct. A male colleague from her branch, she said, was nominated for tenure at the same time that she was held back.

Yep.

Amazingly Story Landis, prior NINDS Director, gave the full reveal quote:

While tenure awards are supposed to be based largely on merit, it is widely acknowledged that personality conflicts, budget constraints, internal politics and other factors affect them.

“Tenure decisions are complicated, and not just about what you’ve published,” Landis said.

In this, the NIH IRPs are no different than anywhere else, eh? It isn't about objective merits but about the subjective views of your colleagues, when it comes right down to it.

And lets in a whole lot o' bias.

Bielekova, ..has filed an Equal Employment Opportunity complaint against her institute’s director and two others,

Twill be interesting to watch this play out.

13 responses so far

Great lens to use on your own grants

Aug 26 2016 Published by under Grant Review, NIH, NIH Careerism

If your NIH grant proposal reads like this, it is not going to do well.

9 responses so far

Your Grant in Review: Throwing yourself on the mercy of the study section court

Aug 24 2016 Published by under Careerism, Grant Review, NIH, NIH Careerism, Uncategorized

A question and complaint from commenter musclestumbler on a prior thread introduces the issue.

So much oxygen is sucked up by the R01s, the med schools, etc. that it tends to screw over reviews for the other mechanisms. I look at these rosters, then look at the comments on my proposals, and it's obvious that the idea of doing work without a stable of postdocs and a pool of exploitable Ph.D. students is completely alien and foreign to them.

and extends:

I personally go after R15 and R03 mechanisms because that's all that can be reasonably obtained at my university. ... Postdocs are few and far between. So we run labs with undergrads and Masters students. Given the workload expectations that we have in the classroom as well as the laboratory, the R15 and R03 mechanisms support research at my school. Competing for an R01 is simply not in the cards for the productivity level that we can reasonably pursue...

This isn't simply fatalism, this is actual advice given by multiple program officers and at workshops. These mechanisms are in place to facilitate and foster our research. Unfortunately, these are considered and reviewed by the same panels that review R01s. We are not asking that they create an SEP for these mechanisms - a "little kids table" if you will - but that the panels have people with these similar institutions on them. I consider it a point of pride that my R15 is considered by the same reviewers that see the R01s, and successfully funded as well.

The point is that, the overwhelming perception and unfortunate reality is that many, many, many of the panelists have zero concept of the type of workload model under which I am employed. And the SROs have a demonstrably poor track record of encouraging institutional diversity. Sure, my panel is diverse- they have people from a medical school, an Ivy League school, and an endowed research institution on the West Coast. They have Country, and Western!

I noted the CSR webpage on study section selection says:

Unique characteristics of study sections must be factored into selection of members. The breadth of science, the multidisciplinary or interdisciplinary nature of the applications, and the types of applications or grant mechanisms being reviewed play a large role in the selection of appropriate members.

It seems very much the case to me that if R15s are habitually being reviewed in sections without participation of any reviewers from R15-eligible institutions, this is a violation of the spirit of this clause.

I suggested that this person should bring this up with their favorite SROs and see what they have to say. I note that now that there is a form for requesting "appropriate expertise" when you submit your NIH grant, it may also be useful to use this to say something about R15-eligible reviewers.

But ultimately we come to the "mercy of the court" aspect of this issue. It is my belief that while yes, the study section is under very serious constraints these days, it is still a human behavior that occasionally lets real humans make rational decisions. Sometimes, reviewers may go for something that is outside of the norm. Outside of the stereotype of what "has" to be in the proposal of this type. Sometimes, reviewers may be convinced by the peculiarities of given situation to, gasp, give you a break. So I suggested the following for this person who had just indicated that his/her R15s do perfectly well in a study section that they think would laugh off their R01 application.

I think this person should try a trimmed down R01 in this situation. Remember the R01 is the most flexible in terms of scope- there is no reason you cannot match it to the budget size of any of the other awards. The upside is that it is for up to five years, better than AREA/R15 (3 y) or R03 (2 y). It is competitively renewable, which may offer advantages. It is an R01, which, as we are discussing in that other thread, may be the key to getting treated like a big kid when it comes to study section empanelment.

The comments from musclestubmler make it sound as if the panels can actually understand the institutional situation, just so long as they are focused on it by the mechanism (R15). The R15 is $100K direct for three years, no? So why not propose an R01 for $100K direct for five years? or if you, Dear Reader, are operating at an R03 level, ask for $50K direct or $75K direct. And I would suggest that you don't just leave this hidden in the budget, sprinkle wording throughout everywhere that refers to this being a go-slow but very inexpensive (compared to full mod) project.

Be very clear about your time commitment (summers only? fine, just make it clear) and the use of undergrads (predict the timeline and research pace) in much the same way you do for an R15 but make the argument for a longer term, renewable R01. Explain why you need it for the project, why it is justified and why a funded version will be productive, albeit at a reduced pace. See if any reviewers buy it. I would.

Sometimes you have to experiment a little with the NIH system. You'd be surprised how many times it works in ways that are not exactly the stereotypical and formal way things are supposed to work.

27 responses so far

Projected NRSA salary scale for FY2017

NOT-OD-16-131 indicates the projected salary changes for postdoctoral fellows supported under NRSA awards.

Being the visual person that I am...
NRSAFY16-17chart

As anticipated, the first two years were elevated to meet the third year of the prior scale (plus a bit) with a much flatter line across the first three years of postdoctoral experience.

What think you o postdocs and PIs? Is this a fair* response to the Obama overtime rules?

Will we see** institutions (or PIs) where they just extend that shallow slope out for Years 3-7+?

h/t Odyssey and correction of my initial misread from @neuroecology
__
*As a reminder, $47,484 in 2016 dollars equals $39,715 in 2006 dollars, $30,909 in 1996 dollars and $21,590 in 1986 dollars. Also, the NRSA Yr 0 for postdocs was $20,292 for FY1997 and $36,996 for FY2006.

**I bet yes***.

***Will this be the same old jerks that already flatlined postdoc salaries? or will PIs who used to apply yearly bumps now be in a position where they just flatline since year 1 has increased so much?

38 responses so far

The R01 still doesn't pay for itself and reviewers are getting worse

Jul 11 2016 Published by under NIH, NIH Careerism, NIH funding

I pointed out some time ago that the full modular R01 grant from the NIH doesn't actually pay for itself.

In the sense that there is a certain expectation of productivity, progress, etc on the part of study sections and Program that requires more contribution than can be afforded (especially when you put it in terms of 40 hr work weeks) within the budget. Trainees on individual fellowships or training grants, undergrads working for free or work study discount, cross pollination with other grants in the lab (which often leads to whinging like your comment), pilot awards for small bits, faculty hard money time...all of these sources of extra effort are frequently poured into a one-R01 project. I think they are, in essence, necessary.

I had some additional thoughts on this recently.

It's getting worse.

Look, it has always been the case that reviewers want to see more in a grant proposal. More controls, usually. Extra groups to really nail down the full breadth of...whatever it is that you are studying. This really cool other line of converging evidence... anything is possible.

All I can reflect is my own experience in getting my proposals reviewed and in reviewing proposals that are somewhat in the same subfields.

What I see is a continuing spiral of both PI offerings and of reviewer demands.

It's inevitable, really. If you see a proposal chock full of nuts that maybe doesn't quite get over the line of funding because of whatever reason, how can you give a fundable score to a very awesome and tight proposal that is more limited?

Conversely, in the effort to put your best foot forward you, as applicant, are increasingly motivated to throw every possible tool at your disposal into the proposal, hoping to wow the reviewers into submission.

I have reviewed multiple proposals recently that cannot be done. Literally. They cannot be accomplished for the price of the budget proposed. Nobody blinks an eye about this. They might talk about "feasibility" in the sense of scientific outcomes or preliminary data or, occasionally, some perceived deficit of the investigators/environment. But I have not heard a reviewer say "nice but there is no way this can be accomplished for $250K direct". Years ago people used to crab about "overambitious" proposals but I can't say I've heard that in forever. In this day and age of tight NIH paylines, the promises of doing it all in one R01 full-modular 5 year interval are escalating.

These grants set a tone, btw. I'm here to tell you that I've seen subfield related proposals that do seem feasible, money-wise, get nailed because they are too limited in scope. In some cases there is enough study-section continuity involved for me to be certain that this is due to reviewer contamination from the aforementioned chock-full-o-nuts impossible proposals. Yes, some of this is due to SABV but not all of it. It ranges from "why you no include more co-investigators?" (a subtle spread-the-wealth knock on big labs? maybe) to "You really need to add X, Y and Z to be convincing" (mkay but... $250K dude) to "waaah, I just want to see more" (even though they don't really have a reason to list).

Maybe this is just me being stuck in the rut I was trained in. In my formative years, grant review seemed to expect you would propose a set of studies that you could actually accomplish within the time frame and budget proposed. I seem to remember study section members curbing each other with "Dude, the PI can't fit all that stuff into one proposal, back off.". I used to see revisions get improved scores when the PI stripped a bloated proposal down to a minimalist streamlined version.

Maybe we are just experiencing a meaningless sea change in grant review to where we propose the sky and nobody cares on competing renewal if we managed to accomplish all of that stuff.

37 responses so far

Power in the NIH review trenches

Jun 18 2016 Published by under Fixing the NIH, NIH, NIH Careerism

That extensive quote from a black PI who had participated in the ECR program is sticking with me.

Insider status isn't binary, of course. It is very fluid within the grant-funded science game. There are various spectra along multiple dimensions.

But make no mistake it is real. And Insider status is advantageous. It can be make-or-break crucial to a career at many stages.

I'm thinking about the benefits of being a full reviewer with occasional/repeated ad hoc status or full membership.

One of those benefits is that other reviewers in SEPs or closely related panels are less likely to mess with you.

Less likely.

It isn't any sort of quid pro quo guarantee. Of course not. But I guarantee that a reviewer who thinks this PI might be reviewing her own proposal in the near future has a bias. A review cant. An alerting response. Whatever.

It is different. And, I would submit, generally to the favor of the applicant that possesses this Mutually Assured Destruction power.

The Ginther finding arose from a thousand cuts, I argue. This is possibly one of them.

3 responses so far

Nakamura reports on the ECR program

Jun 17 2016 Published by under Fixing the NIH, Grant Review, NIH, NIH Careerism

If I stroke out today it is all the fault of MorganPhD.

Jeffery Mervis continues with coverage of the NIH review situation as it pertains to the disparity for African-American PIs identified in 2011 (that's five years and fifteen funding rounds ago, folks) by the Ginther report.

The main focus for this week is on the Early Career Reviewer program. As you will recall, this blog has advocated continually and consistently for the participation of more junior PIs on grant review panels.

The ECR program was created explicitly to deal with underrepresented groups. However, what happened is that there was immediate opposition which insisted that the ECR program had to be open to all junior faculty/applicants, regardless of representation in the NIH game.

One-quarter of researchers in ECR's first cohort were from minority groups, he notes. “But as we've gone along, there are fewer underrepresented minorities coming into the pool.”
...
Minorities comprise only 13% of the roughly 5100 researchers accepted into the program (6% African-American and 7% Hispanic), a percentage that roughly matches their current representation on study sections.

Ok, but how have the ECR participants fared?

[Nakamura] said ECR alumni have been more than twice as successful as the typical new investigator in winning an R01 grant.

NIIIIIICE. Except they didn't flog the data as hard as one might hope. This is against the entire NI (or ESI?) population.

The pool of successful ECR alumni includes those who revised their application, sometimes more than once, after getting feedback on a declined proposal. That extra step greatly improves the odds of winning a grant. In contrast, the researchers in the comparison group hadn't gone through the resubmission process.

Not sure if this really means "hadn't" or "hadn't necessarily". The latter makes more sense if they are just comparing to aggregate stats. CSR data miners would have had to work harder to get this isolated to those who hadn't revised yet, and I suspect if they had gone to that effort, they could have presented the ESIs who had at least one revision under their belt. But what about the underrepresented group of PIs that are the focus of all this effort?

It's also hard to interpret the fact that 18% of the successful ECRs were underrepresented minorities because NIH did not report the fraction of minorities among ECR alumni applicants. So it is not clear whether African-Americans participating in the program did any better than the cohort as a whole—suggesting that the program might begin to close the racial gap—or better than a comparable group of minority scientists who were not ECR alumni.

SERIOUSLY Richard Nakamura? You just didn't happen to request your data miners do the most important analysis? How is this even possible?

How on earth can you not be keeping track of applicants to ECR, direct requests from SROs, response rate and subsequent grant and reviewing behavior? It is almost as if you want to look like you are doing something but have no interest in it being informative or in generating actionable intelligence.

Moving along, we get a further insight into Richard Nakamura and his position in this situation.

Nakamura worries that asking minority scientists to play a bigger role in NIH's grantsmaking process could distract them from building up their lab, finding stable funding, and earning tenure. Serving on a study section, he says, means that “those individuals will have less time to write applications. So we need to strike the right balance.”

Paternalistic nonsense. The same thing that Scarpa tried to use to justify his purge of Assistant Professors from study sections. My answer is the same. Let them decide. For themselves. Assistant Professors and underrepresented PIs can decide for themselves if they are ready and able to take up a review opportunity when asked. Don't decide, paternalistically, that you know best and will refrain from asking for their own good, Director Nakamura!

Fascinatingly, Mervis secured an opinion that echoes this. So Nakamura will surely be reading it:

Riggs, the only African-American in his department, thinks the program is too brief to help minority scientists truly become part of the mainstream, and may even exacerbate their sense of being marginalized.

“After I sat on the panel, I realized there was a real network that exists, and I wasn't part of that network,” he says. “My comments as a reviewer weren't taken as seriously. And the people who serve on these panels get really nervous about having people … that they don't know, or who they think are not qualified, or who are not part of the establishment.”

If NIH “wants this to be real,” Riggs suggests having early-career researchers “serve as an ECR and then call them back in 2 years and have them serve a full cycle. I would have loved to do that.”

The person in the best position to decide what is good or bad for his or her career is the investigator themself.

This comment also speaks to my objection to the ECR as a baby-intro version of peer review. It isn't necessary. I first participated on study section in my Asst Prof years as a regular ad hoc with a load of about six grants, iirc. Might have been 2 less than the experienced folks had but it was not a baby-trainee experience in the least. I was treated as a new reviewer, but that was about the extent of it. I thought I was taken seriously and did not feel patronized.

__
Related Reading:
Toni Scarpa to leave CSR

More on one Scientific Society’s Response to the Scarpa Solicitation

Your Grant In Review: Junior Reviewers Are Too Focused on Details

The problem is not with review…

Peer Review: Opinions from our Elders

23 responses so far

NIH Director Collins and CSR Director Nakamura continue to kick the funding disparity can down the road

A News piece in Science by Jeffrey Mervis details the latest attempt of the NIH to kick the Ginther can down the road.

Armed with new data showing black applicants suffer a 35% lower chance of having a grant proposal funded than their white counterparts, NIH officials are gearing up to test whether reviewers in its study sections give lower scores to proposals from African-American applicants. They say it’s one of several possible explanations for a disparity in success rates first documented in a 2011 report by a team led by economist Donna Ginther of the University of Kansas, Lawrence.

Huh. 35%? I thought Ginther estimated more like a 13% difference? Oh wait. That's the award probability difference. About 16% versus 29% for white applicants which would be about a 45% lower chance. And this shows "78-90% the rate of white...applicants". And there was Nakamura quoted in another piece in Science:

At NIH, African-American researchers “receive awards at “55% to 60% the rate of white applicants,” Nakamura said. “That's a huge disparity that we have not yet been able to seriously budge,” despite special mentoring and networking programs, as well as an effort to boost the number of scientists from underrepresented minorities who evaluate proposals.

Difference vs rate vs lower chance.... Ugh. My head hurts. Anyway you spin it, African-American applicants are screwed. Substantially so.

Back to the Mervis piece for some factoids.

Ginther..noted...black researchers are more likely to have their applications for an R01 grant—the bread-and-butter NIH award that sustains academic labs—thrown out without any discussion...black scientists are less likely to resubmit a revised proposal ...whites submit at a higher rate than blacks...

So, what is CSR doing about it now? OK HOLD UP. LET ME REMIND YOU IT IS FIVE YEARS LATER. FIFTEEN FUNDING ROUNDS POST-GINTHER. Ahem.

The bias study would draw from a pool of recently rejected grant applications that have been anonymized to remove any hint of the applicant’s race, home institution, and training. Reviewers would be asked to score them on a one-to-nine scale using NIH’s normal rating system.

It's a start. Of course, this is unlikely to find anything. Why? Because the bias at grant review is a bias of identity. It isn't that reviewers are biased against black applicants, necessarily. It is that they are biased for white applicants. Or at the very least they are biased in favor of a category of PI ("established, very important") that just so happens to be disproportionately white. Also, there was this interesting simulation by Eugene Day that showed a bias that is smaller than the non-biased variability in a measurement can have large effects on something like a grant funding system [JournalLink].

Ok, so what else are they doing?

NIH continues to wrestle with the implications of the Ginther report. In 2014, in the first round of what NIH Director Francis Collins touted as a 10-year, $500 million initiative to increase the diversity of the scientific workforce, NIH gave out 5-year, $25 million awards to 10 institutions that enroll large numbers of minority students and created a national research mentoring network.

As you know, I am not a fan of these pipeline-enhancing responses. They say, in essence, that the current population of black applicant PIs is the problem. That they are inferior and deserve to get worse scores at peer review. Because what else does it mean to say the big money response of the NIH is to drum up more black PIs in the future by loading up the trainee cannon now?

This is Exhibit A of the case that the NIH officialdom simply cannot admit that there might be unfair biases at play that caused the disparity identified in Ginther and reinforced by the other mentioned analyses. The are bound and determined to prove that their system is working fine, nothing to see here.

So....what else ?

A second intervention starting later this year will tap that fledgling mentoring network to tutor two dozen minority scientists whose R01 applications were recently rejected. The goal of the intervention, which will last several months, is to prepare the scientists to have greater success on their next application. A third intervention will educate minority scientists on the importance of resubmitting a rejected proposal, because resubmitted proposals are three times more likely to be funded than a de novo application from a researcher who has never been funded by NIH.

Oh ff..... More of the same. Fix the victims.

Ah, here we go. Mervis finally gets around to explaining that 35% number

NIH officials recently updated the Ginther study, which examined a 2000–2006 cohort of applicants, and found that the racial disparity persists. The 35% lower chance of being funded comes from tracking the success rates of 1054 matched pairs of white and black applicants from 2008 to 2014. Black applicants continue to do less well at each stage of the process.

I wonder if they will be publishing that anywhere we can see it?

But here's the kicker. Even faced with the clear evidence from their own studies, the highest honchos still can't see it.

One issue that hung in the air was whether any of the disparity was self-inflicted. Specifically, council members and NIH officials pondered the tendency of African-American researchers to favor certain research areas, such as health disparities, women’s health, or hypertension and diabetes among minority populations, and wondered whether study sections might view the research questions in those areas as less compelling. Valantine called it a propensity “to work on issues that resonate with their core values.” At the same time, she said the data show minorities also do less well in competition with their white peers in those fields.

Collins offered another possibility. “I’ve heard stories that they might have been mentored to go into those areas as a better way to win funding,” he said. “The question is, to what extent is it their intrinsic interest in a topic, and to what extent have they been encouraged to go in that direction?”

Look, Ginther included a huge host of covariate analyses that they conducted to try to make the disparity go away. Now they've done a study with matched pairs of investigators. Valantine's quote may refer to this or to some other analysis I don't know but obviously the data are there. And Collins is STILL throwing up blame-the-victim chaff.

Dude, I have to say, this kind of denialist / crank behavior has a certain stench to it. The data are very clear and very consistent. There is a funding disparity.

This is a great time to remind everyone that the last time a major funding disparity came to the attention of the NIH it was the fate of the early career investigators. The NIH invented up the ESI designation, to distinguish it from the well established New Investigator population, and immediately started picking up grants out of the order of review. Establishing special quotas and paylines to redress the disparity. There was no talk of "real causes". There was not talk of strengthening the pipeline with better trainees so that one day, far off, they magically could better compete with the established. Oh no. They just picked up grants. And a LOT of them.

I wonder what it would take to fix the African-American PI disparity...

Ironically, because the pool of black applicants is so small, it wouldn’t take much to eliminate the disparity: Only 23 more R01 applications from black researchers would need to be funded each year to bring them to parity.

Are you KIDDING me? That's it?????

Oh right. I already figured this one out for them. And I didn't even have the real numbers.

In that 175 bin we'd need 3 more African-American PI apps funded to get to 100%. In the next higher (worse) scoring bin (200 score), about 56% of White PI apps were funded. Taking three from this bin and awarding three more AA PI awards in the next better scoring bin would plunge the White PI award probability from 56% to 55.7%. Whoa, belt up cowboy.

Moving down the curve with the same logic, we find in the 200 score bin that there are about 9 AA PI applications needed to put the 200 score bin to 100%. Looking down to the next worse scoring bin (225) and pulling these 9 apps from white PIs we end up changing the award probability for these apps from 22% to ..wait for it..... 20.8%.

Mere handfuls. I had probably overestimated how many black PIs were seeking funding. If this Mervis piece is to be trusted and it would only take 23 pickups across the entire NIH to fix the problem....

I DON'T UNDERSTAND WHAT FRANCIS COLLINS' PROBLEM IS.

Twenty three grants is practically rounding error. This is going to shake out to one or maybe three grants per year for the ICs, depending on size and what not.

Heck, I bet they fund this many grants every year by mistake. It's a big system. You think they don't have a few whoopsies sneak by every now and again? Of course they do.

But god forbid they should pick up 23 measly R01s to fix the funding disparity.

37 responses so far

NIH gets serious about the mouths-at-the-trough. Sortof.

Jun 06 2016 Published by under Fixing the NIH, NIH, NIH Careerism

The latest Open Mike blogpost from NIH Deputy Director for the OER, Mike Lauer, ventures into analysis of TheRealProblem at last.

The setup, in and of itself, is really good information.

We first looked at all research project grants (RPGs) funded between 2003 and 2015. For each year, we identified unique principal investigators who were named on at least one RPG award in that year. Figure 1 shows that the number of NIH-supported investigators has increased only slightly, and has remained fairly constant at about 27,500 over the past thirteen years.

Burn that one into your brain, people. There are about 27,500 unique PIs funded at any given time and this number has been rock steady for at least thirteen years. Sure, it is crazy-making stuff that they do not go back past the doubling interval to see what is really going on but hey, this is a significant improvement. At last the NIH is grappling with their enterprise by funded-investigators instead of funded-applications. This is a key addition and long, long overdue. I approve.

There are some related analyses from DataHound that lead into these considerations as well. I recommend you go back and read Longitudinal PI Analysis: Distributions, Mind the Gap and especially A longitudinal analysis of NIH R-Funded Investigators: 2006-2013. This latter one estimated a similar number of unique PIs but it also estimated the churn rate, that is, the number in each fiscal year that are new and the number who have left the funded-PI distribution (it was about 5,300 PIs per FY).

Back to Lauer's post for the supplicant information that DataHound couldn't get:

To determine how many unique researchers want to be funded, we identified unique applicants over 5-year windows. We chose to look at a multi-year window for two reasons: most research grants last for more than one year and most applicants submit applications over a period of time measured in years, not just 12 months, that may overlap with their periods of funding, if they are funded. Figure 2 shows our findings for applicants as well as awardees: the number of unique applicants has increased substantially, from about 60,000 investigators who had applied during the period from 1999 to 2003 to slightly less than 90,000 in who had applied during the period from 2011 to 2015.

via https://nexus.od.nih.gov/all/2016/05/31/how-many-researchers/

The too-many-mouths problem is illustrated. Simply. Cleanly. We can speculate about various factors until the cows come home but this is IT.

Too Many Mouths At The Trough And Not Enough Slops.

The blogpost then goes on to calculate a Cumulative Investigator Rate which is basically how many PIs get funded over a 5 year interval out of those who wish to be funded. In 2003 it was 43% and this declined to 31% in 2015. This was for RPGs. If you limit to R01 only, the CIR goes from 45% to 34% over this interval of time. For R21s, the CIR was at 20% in 2003 and is down around 11% for 2015. Newsbreak: Funding rates for R21s are terrible, despite what you would imagine should be the case for this mechanism.

Now we get to the hard part. Having reviewed these data the person responsible for the entire Extramural Research enterprise of NIH boots the obvious. Hard. First, he tries to off load the responsibility by citing Kimble et al and Pickett et al. Then he basically endorses their red herring distractions (when it comes to this particular issue).

NIH leadership is currently engaged in efforts to explore which policies or policy options best assure efficient and sustainable funding given the current hypercompetitive environment. These efforts include funding opportunity announcements for R35 awards which focus on programs, rather than highly specific projects; new models for training graduate students and postdoctoral fellows; establishment of an office of workforce diversity;

Right? It's right there in front of you, dude, and you can't even say it as one of a list of possible suggestions.

We need to stop producing so many PhD scientists.

This is the obvious solution. It is the only thing that will have sustained and systematic effect, while retaining some thin vestige of decency towards the people who have already devoted years and decades to the NIH extramural enterprise.

Oh and don't get me too wrong. From a personal perspective, clearly Lauer is not completely idiotic:

and even what we are doing here, namely drawing attention to numbers of unique investigators and applicants.

HAHAHHAHAHAAH. What a bureaucratic weasel. He sees it all right. He does. And he's trying to wink it into the conversation without taking any responsibility whatsoever. I see you, man. I see you. Okay. I'll take up the hard work for you.

We need to stop training so many PhDs. Now. Yesterday in fact. All of us. Stop pretending your high-falutin program gets to keep all their students and those inferior jerks, over there, need to close up shop. Significant reductions are called for.

Personally, I call for a complete moratorium on new PhD admits for 5 years.

Go.

123 responses so far

Older posts »