Archive for the 'Fixing the NIH' category

Grant Supplements and Diversity Efforts

The NIH announced an "encouragement" for NIMH BRAINI PIs to apply for the availability of Research Supplements to Promote Diversity in Health-Related Research (Admin Supp).

Administrative supplements for those who are unaware, are extra amounts of money awarded to an existing NIH grant. These are not reviewed by peer reviewers in a competitive manner. The decision lies entirely with Program Staff*. The Diversity supplement program in my experience and understanding amounts to a fellowship- i.e., mostly just salary support - for a qualifying trainee. (Blog note: Federal rules on underrepresentation apply....this thread will not be a place to argue about who is properly considered an underrepresented individual, btw.) The BRANI-directed the encouragement lays out the intent:

The NIH diversity supplement program offers an opportunity for existing BRAIN awardees to request additional funds to train and mentor the next generation of researchers from underrepresented groups who will contribute to advancing the goals of the BRAIN Initiative. Program Directors/Principal Investigators (PDs/PIs) of active BRAIN Initiative research program grants are thus encouraged to identify individuals from groups nationally underrepresented to support and mentor under the auspices of the administrative supplement program to promote diversity. Individuals from the identified groups are eligible throughout the continuum from high school to the faculty level. The activities proposed in the supplement application must fall within the scope of the parent grant, and both advance the objectives of the parent grant and support the research training and professional development of the supplement candidate. BRAIN Initiative PDs/PIs are strongly encouraged to incorporate research education activities that will help prepare the supplement candidate to conduct rigorous research relevant to the goals of the BRAIN Initiative

I'll let you read PA-16-288 for the details but we're going to talk generally about the Administrative Supplement process so it is worth reprinting this bit:

Administrative supplement, the funding mechanism being used to support this program, can be used to cover cost increases that are associated with achieving certain new research objectives, as long as the research objectives are within the original scope of the peer reviewed and approved project, or the cost increases are for unanticipated expenses within the original scope of the project. Any cost increases need to result from making modifications to the project that would increase or preserve the overall impact of the project consistent with its originally approved objectives and purposes.

Administrative supplements come in at least three varieties, in my limited experience. [N.b. You can troll RePORTER for supplements using "S1" or "S2" in the right hand field for the Project Number / Activity Code search limiter. Unfortunately I don't think you get much info on what the supplement itself is for.] The support for underrepresented trainees is but one category. There are also topic-directed FOAs that are issued now and again because a given I or C wishes to quickly spin up research on some topic or other. Sex differences. Emerging health threats. Etc. Finally, there are those one might categorize within the "unanticipated expenses" and "increase or preserve the overall impact of the project" clauses in the block I've quoted above.

I first became aware of the Administrative Supplement in this last context. I was OUTRAGED, let me tell you. It seemed to be a way by which the well-connected and highly-established use their pet POs to enrich their programs beyond what they already had via competition. Some certain big labs seemed to be constantly supplemented on one award or other. Me, I sure had "unanticipated expenses" when I was just getting started. I had plenty of things that I could have used a few extra modules of cash to pay for to enhance the impact of my projects. I did not have any POs looking to hand me any supplements unasked and when I hinted very strongly** about my woes there was no help to be had***. I did not like administrative supplements as practiced one bit. Nevertheless, I was young and still believed in the process. I believed that I needn't pursue the supplement avenue too hard because I was going to survive into the mid career stretch and just write competing apps for what I needed. God, I was naive.

Perhaps. Perhaps if I'd fought harder for supplements they would have been awarded. Or maybe not.

When I became aware of the diversity supplements, I became an instant fan. This was much more palatable. It meant that at any time a funded PI found a likely URM recruit to science, they could get the support within about 6 weeks. Great for summer research experiences for undergrads, great for unanticipated postdocs. This still seems like a very good thing to me. Good for the prospective trainees. Good for diversity-in-science goals.

The trouble is that from the perspective of the PIs in the audience, this is just another rich-get-richer scheme whereby free labor is added to the laboratory accounts of the already advantaged "haves" of the NIH game. Salary is freed up on the research grants to spend on more toys, reagents or yet another postdoc. This mechanism is only available to a PI who has research grant funding that has a year or more left to run. Since it remains an administrative decision it is also subject to buddy-buddy PI/PO relationship bias. Now, do note that I have always heard from POs in my ICs of closest concern that they "don't expend all the funds allocated" for these URM supplements. I don't know what to make of that but I wouldn't be surprised in the least if any PI with a qualified award, who asks for support of a qualified individual gets one. That would take the buddy/buddy part out of the equation for this particular type of administrative supplement.

It took awhile for me to become aware of the FOA version of the administrative supplement whereby Program was basically issuing a cut-rate RFA. The rich still get richer but at least there is a call for open competition. Not like the first variety I discussed whereby it seems like only some PIs, but not others, are even told by the PO that a supplement might be available. This seems slightly fairer to me although again, you have to be in the funded-PI club already to take advantage

There are sometimes competing versions of the FOA for a topic-based supplement issued as well. In one case I am familiar with, both types were issued simultaneously. I happen to know quite a bit about that particular scenario and it was interesting to see the competing variety actually were quite bad. I wished I'd gone in for the competing ones instead of the administrative variety****, let me tell you.

The primary advantage of the administrative supplement to Program, in my viewing, is that it is fast. No need to wait for the grant review cycle. These and the competing supplements are also cheap and can be efficient, because of leverage from the activities and capabilities under the already funded award.

As per usual, I have three main goals with this post. First, if you are an underrepresented minority trainee it is good to be aware of this. Not all PIs are and not all think about it. Not to mention they don't necessarily know if you qualify for one of these. I'd suggest bringing it up in conversations with a prospective lab you wish to join. Second, if you are a noob PI I encourage you to be aware of the supplement process and to take advantage of it as you might.

Finally, DearReader, I turn to you and your views on Administrative Supplements. Good? Bad? OUTRAGE?

__
COI DISCLAIMER: I've benefited from administrative supplements under each of the three main categories I've outlined and I would certainly not turn up my nose at any additional ones in the future.

*I suppose it is not impossible that in some cases outside input is solicited.

**complained vociferously

***I have had a few enraging conversations long after the fact with POs who said things like "Why didn't you ask for help?" in the wake of some medium sized disaster with my research program. I keep to myself the fact that I did, and nobody was willing to go to bat for me until it was too late but...whatevs.

****I managed to get all the way to here without emphasizing that even for the administrative supplements you have to prepare an application. It might not be as extensive as your typical competing application but it is much more onerous than Progress Report. Research supplements look like research grants. Fellowship-like supplements look like fellowships complete with training plan.

20 responses so far

What if it were about deserve?

Oct 26 2016 Published by under Fixing the NIH

Imagine that the New Investigator status (no prior service as PI of major NIH grant) required an extra timeline document? This would be a chronology of the PI's program to date with emphasis on funding (startup, institutional grants, foundation), how publications were generated, and the PI's scrambling. Another part would focus on grants submitted, score outcomes, revisions, how preliminary data was generated, etc.

Would this improve the way the NIH awards grants?

__
(Keep in mind the NIH has wrung its hands about the dismal fate of the not-yet-funded for many decades and created numerous "fixes" over the years.)

32 responses so far

NINDS tweaks their approach to the F32 / NRSA

NOT-NS-17-002 indicates that NINDS will no longer participate in the NIH-wide parent F32/NRSA funding opportunity because they will be customizing their approach.

 

As previously described in NOT-NS-16-012 and NOT-NS-16-013, NINDS is restructuring its funding support for postdoctoral researchers.  Beginning with the December 8, 2016 due date, research training support for postdoctoral fellows under the F32 activity code will be available through NINDS using PAR-16-458 "NINDS Ruth L. Kirschstein National Research Service Award (NRSA) for Training of Postdoctoral Fellows (F32)."  This NINDS F32 will support postdocs who are within the first 3 years of research training in the sponsor's laboratory, and includes several other key differences from the parent F32. Most notably, applicants are only eligible for the NINDS F32 prior to starting, or within the first 12 months of starting, their postdoctoral training in the sponsor's laboratory or research environment. Because of the very early application, no preliminary data are expected.  It is anticipated that another Funding Opportunity Announcement for postdocs, which utilizes the K01 activity code, will be published in time for the February 12, 2017 initial receipt date. This will be available to applicants in their second through fourth year of cumulative postdoctoral research experience (see NOT-NS-16-013). 

I remember the initial troll on this but managed to overlook the part where they were going to have a new K01 announcement focused on later-stage postdocs.

I like this, actually. We've gotten into a situation where F32s are stuck in the escalating-expectations holding pattern of endless revisions and resubmissions lately. I just don't see the point of a 3rd year postdoc writing for "training" support that will only arrive in year 4 or 5. Particularly when at that point the postdocs who are gunning hard for a faculty research type job should be focusing on the K99/R01. This has been a waste of time, let the awardees languish for extra time so that they get at least a year or two on the F32 and make a mockery of the idea of the F32.

I am likewise encouraged that instead of leaving the 2+ year postdocs at the tender mercies of the K99/R00 process, NINDS has a fill-in with a K01. I note that their warning notice on this looks good.

The NINDS K01 is intended for candidates with a Ph.D. or equivalent research doctoral degree. Candidates will be eligible to apply for the K01 anytime within the second through fourth year of cumulative mentored, postdoctoral research experience, and may be supported by the NINDS K01 within the first 6 years of cumulative postdoctoral research experience. Successful K01 applications will be designed to facilitate the continuation of outstanding, innovative projects, combined with career development activities that will prepare outstanding postdoctoral, mentored investigators for an independent research career. The K01 application will describe a project that, as demonstrated by preliminary data collected by the applicant, holds promise to result in highly significant results and future discoveries. The K01 candidate will continue to be guided by a postdoctoral mentor, but will be primarily responsible for oversight and conduct of the research project. By the end of the proposed K01 award period, the candidate will be poised to begin an independent research career and will have a well-developed, highly significant project that he/she can take with him/her to an independent research position.

The devil, of course, is in the details. In my most frequent experience, the K01 tends to be won by people already in quasi-faculty positions. People who have been promoted to "Instructor" or "Assistant Research Project Quasi-faculty but not really Scientist" or whatever word salad title your University prefers. I do not see this being favored for award to any old run of the mill year 2 postdoc. Maybe your frame of reference differs, DearReader?

It will be interesting to see how this is used in practice. Will it only be for the people who just-miss on the K99/R00? Or will it occupy the place currently occupied by the F32 with successful applicants having 2-3 years of postdoc work under their belt before applying? [Mayhap these are the same thing these days?]

But I digress.

The most pressing issue of the day is whether the NINDS will succeed in funding 1) a substantial number of F32s from applicants who are finishing their graduate studies and 2) from first year postdocs without much Preliminary Data in the application.

In my estimation if they don't get to at least 50% of awards on #1, this isn't working.

I also predict that the #2 scenario is going to produce a lot of applications with lots of Preliminary Data, just stuff that wasn't completed directly by the applicant herself.

Thoughts folks? Would you like to see this extended to your favorite ICs?

29 responses so far

Bring back the 2-3 year Developmental R01

Sep 19 2016 Published by under Fixing the NIH, NIH, NIH funding

The R21 Mechanism is called the Exploratory/Developmental mechanism. Says so right in the title.

NIH Exploratory/Developmental Research Grant Program ( Parent R21)

In the real world of NIH grant review, however, the "Developmental" part is entirely ignored in most cases. If you want a more accurate title, it should be:

NIH High Risk / High Reward Research Grant Program ( Parent R21)

This is what reviwers favor in my experiences sitting on panels and occasionally submitting an R21 app. Mine are usually more along the lines of developing a new line of research that I think is important rather than being truly "high risk/high reward".

And, as we all know, the R01 application (5 years, full modular at $250K per annum direct costs if you please) absolutely requires a ton of highly specific Preliminary Data.

So how are you supposed to Develop an idea into this highly specific Preliminary Data? Well, there's the R21, right? Says right in the title that it is Developmental.

But....it doesn't work in practice.

So the R01 is an alternative. After all it is the most flexible mechanism. You could submit an R01 for $25K direct costs for one year. You'd be nuts, but you could. Actually you could submit an R03 or R21 for one $25K module too, but with the R01 you would then have the option to put in a competitive renewal to continue the project along.

The only thing stopping this from being a thing is the study section culture that won't accept it. Me, I see a lot of advantages to using shorter (and likely smaller) R01 proposals to develop a new line of work. It is less risky than a 5 year R01, for those that focus on risk/$. It has an obvious path of continuation as a genuinely Developmental attempt. It is more flexible in scope and timing- perhaps what you really need is $100K per year for 3 years (like the old R21) for your particular type of research or job type. It doesn't come laden with quite the same "high risk, high reward" approach to R21 review that biases for flash over solid workmanlike substance.

The only way I see this working is to try it. Repeatedly. Settle in for the long haul. Craft your Specific Aims opening to explain why you are taking this approach. Take the Future Directions blurb and make it really sparkle. Think about using milestones and decision points to convince the reviewers you will cut this off at the end if it isn't turning out to be that productive. Show why your particular science, job category, institute or resources match up to this idea.

Or you could always just shout aimlessly into the ether of social media.

41 responses so far

Where the NIGMS argument doesn't add up

Jul 08 2016 Published by under Fixing the NIH, NIH Budgets and Economics

The NIGMS continues its ongoing argument for funding more labs with ever decreasing amounts of grant funding in a new Feedback Loop post.

This one focuses, yet again, on "productivity" as assessed by publication counts and (this time) citations of those publications. It is, as always, significantly flawed by ignoring the effects of Glamour publications. I've done that before and it is starting to bore me. In short, you cannot compare apples to oranges because of the immense difference in the cost of generating your average Nature paper versus a Brain Research paper. And citations don't help because getting into a Glam journal does not mean your paper will get any particular number of citations. Furthermore, there is very little chance that papers that cost 10 or 20 times more will generate ten or twenty times the citations, on average, given the skew in citation distributions and the fact that Glam journals are only hitting means in the 30-40 range. Finally, their "efficiency" measures completely ignore the tremendous inefficiencies of interrupted funding, which is a reality under the current system and also not necessarily fixed with their spread-the-wealth schemes.

The real issue of the day is the opinion of the fans of NIGMS's "conclusion*", which reads:

Overall, however, the data suggest that supporting a greater number of investigators at moderate funding levels is a better investment strategy than concentrating high amounts of funding in a smaller number of researchers.

The Sally Rockey blog entry on "mythbusting" is relevant here. As of FY2009 about 72% of NIH funded investigators had one RPG. Another 20% had two and maybe 5% had three.

That's all.

The NIGMS data analyses are big on fitting productivity lines to about the single R01 level of direct costs (~$200K per year) and showing how the productivity/cost drops off as the grant funding increases. Take a good look at the most recent analysis. Linear productivity up to $300K direct costs with the 75%ile sustained all the way to $500K. The famous original 2010 analysis by Jeremy Berg at NIGMS is pretty similar in the sense that you don't get much change in the line fit to mean publications until you get to the $600-$700K direct costs range.

There is a critical point in lining up these two bits of information which is that the NIGMS policy intent is not supported by their analysis and it can't be. One or two RPG level from Rockey's post should be interpreted in full modular R01 terms ($250K direct, usually cut to $200K, $225K direct and in NIGMS' case to 4 years by default) with a little bit of float upwards for the rare cases. Consequently, it is obvious that most NIH awardees operate in the ~$200-250K part of NIGMS' dataset. Another 20% operate in the $400-$500K direct range. In other words, well within the linear part of the productivity/cost curve.

Mean publications as represented by the 2010 Berg analysis are increasing linearly well up to the three to four grant level of $750K direct costs.

In either case, the "inefficient" grant levels are being obtained by a vanishingly small number of investigators.

Fine, screw them, right?

Sure....but this does nothing to address either the stated goal of NIGMS in hedging their bets across many labs or the goal of the unfunded, i.e., to increase their chances substantially.

A recent Mike Lauer Blog post showed that about a third of those PI's who seek RPG funding over a rolling 5 year interval achieve funding. Obviously if you take all the multi-grant PIs and cut them down to one tomorrow, you'd be able to bump funded investigators up by 15-20%, assuming the FY2009 numbers are relatively good still**. It isn't precise because if you limit the big guys to one award then these are going to drift up to $499K direct at a minimum and a lot more will have special permission to crest the $500K threshold.

There will be a temporary sigh of relief and some folks will get funded at 26%ile. Sure. And then there will be even more PIs in the game seeking funding and it will continue to be a dogfight to retain that single grant award. And the next round of newbies will face the same steep odds of entry. Maybe even steeper.

So the ONLY way for NIGMS' plan to work is to cut per-PI awards way, way down into the front part of their productivity curves. Well below the point of inflection($300-500K or even $750K depending on measure) where papers-per-grant dollar drops off the linear trend. Even the lowest estimate of $300K direct is more than one full-modular grant. It will take a limit substantially below this level*** to improve perceptions of funding ease or to significantly increase the number of funded labs.

Which makes their argument based on those trends a lie, if they truly intend it to support their "better investment strategy". Changing the number of investigators they support in any fundamental way means limiting per-PI awards to the current full modular limit (with typical reductions) at the least, and very likely substantially below this level to produce anything like a phase change.

That's fine if they want to just assert "we think everyone should only have X amount of direct costs" but it is not so fine if they argue that they have some objective, productivity-based data analysis to support their plans. Because it does not.

__
*This is actually their long standing assertion that all of these seemingly objective analyses are designed to support.

**should be ballpark, given the way Program has been preserving unfunded labs at the expense of extra awards to funded labs these days.

***I think many people arguing in favor of the NIGMS type of "small grants for all" strategy operate from the position that they personally deserve funding. Furthermore that some grant award of full modular level or slightly below is sufficient for them. Any dishonest throwaway nod to other types of research that are more expensive (as NIGMS did "We recognize that some science is inherently more expensive, for example because of the costs associated with human and animal subjects.") is not really meant or considered. This is somewhat narrow and self-involved. Try assuming that all of the two-granters in Rockey's distribution really need that amount of funding (remember the erosion of purchasing power?) and that puts it at more like 92% of awardees that enjoy basic funding at present. Therefore the squeeze should be proportional. Maybe the bench jockeys should be limited to $100K or even $50K in this scenario? Doesn't seem so attractive if you consider taking the same proportional hit, does it?

22 responses so far

Power in the NIH review trenches

Jun 18 2016 Published by under Fixing the NIH, NIH, NIH Careerism

That extensive quote from a black PI who had participated in the ECR program is sticking with me.

Insider status isn't binary, of course. It is very fluid within the grant-funded science game. There are various spectra along multiple dimensions.

But make no mistake it is real. And Insider status is advantageous. It can be make-or-break crucial to a career at many stages.

I'm thinking about the benefits of being a full reviewer with occasional/repeated ad hoc status or full membership.

One of those benefits is that other reviewers in SEPs or closely related panels are less likely to mess with you.

Less likely.

It isn't any sort of quid pro quo guarantee. Of course not. But I guarantee that a reviewer who thinks this PI might be reviewing her own proposal in the near future has a bias. A review cant. An alerting response. Whatever.

It is different. And, I would submit, generally to the favor of the applicant that possesses this Mutually Assured Destruction power.

The Ginther finding arose from a thousand cuts, I argue. This is possibly one of them.

3 responses so far

Nakamura reports on the ECR program

Jun 17 2016 Published by under Fixing the NIH, Grant Review, NIH, NIH Careerism

If I stroke out today it is all the fault of MorganPhD.

Jeffery Mervis continues with coverage of the NIH review situation as it pertains to the disparity for African-American PIs identified in 2011 (that's five years and fifteen funding rounds ago, folks) by the Ginther report.

The main focus for this week is on the Early Career Reviewer program. As you will recall, this blog has advocated continually and consistently for the participation of more junior PIs on grant review panels.

The ECR program was created explicitly to deal with underrepresented groups. However, what happened is that there was immediate opposition which insisted that the ECR program had to be open to all junior faculty/applicants, regardless of representation in the NIH game.

One-quarter of researchers in ECR's first cohort were from minority groups, he notes. “But as we've gone along, there are fewer underrepresented minorities coming into the pool.”
...
Minorities comprise only 13% of the roughly 5100 researchers accepted into the program (6% African-American and 7% Hispanic), a percentage that roughly matches their current representation on study sections.

Ok, but how have the ECR participants fared?

[Nakamura] said ECR alumni have been more than twice as successful as the typical new investigator in winning an R01 grant.

NIIIIIICE. Except they didn't flog the data as hard as one might hope. This is against the entire NI (or ESI?) population.

The pool of successful ECR alumni includes those who revised their application, sometimes more than once, after getting feedback on a declined proposal. That extra step greatly improves the odds of winning a grant. In contrast, the researchers in the comparison group hadn't gone through the resubmission process.

Not sure if this really means "hadn't" or "hadn't necessarily". The latter makes more sense if they are just comparing to aggregate stats. CSR data miners would have had to work harder to get this isolated to those who hadn't revised yet, and I suspect if they had gone to that effort, they could have presented the ESIs who had at least one revision under their belt. But what about the underrepresented group of PIs that are the focus of all this effort?

It's also hard to interpret the fact that 18% of the successful ECRs were underrepresented minorities because NIH did not report the fraction of minorities among ECR alumni applicants. So it is not clear whether African-Americans participating in the program did any better than the cohort as a whole—suggesting that the program might begin to close the racial gap—or better than a comparable group of minority scientists who were not ECR alumni.

SERIOUSLY Richard Nakamura? You just didn't happen to request your data miners do the most important analysis? How is this even possible?

How on earth can you not be keeping track of applicants to ECR, direct requests from SROs, response rate and subsequent grant and reviewing behavior? It is almost as if you want to look like you are doing something but have no interest in it being informative or in generating actionable intelligence.

Moving along, we get a further insight into Richard Nakamura and his position in this situation.

Nakamura worries that asking minority scientists to play a bigger role in NIH's grantsmaking process could distract them from building up their lab, finding stable funding, and earning tenure. Serving on a study section, he says, means that “those individuals will have less time to write applications. So we need to strike the right balance.”

Paternalistic nonsense. The same thing that Scarpa tried to use to justify his purge of Assistant Professors from study sections. My answer is the same. Let them decide. For themselves. Assistant Professors and underrepresented PIs can decide for themselves if they are ready and able to take up a review opportunity when asked. Don't decide, paternalistically, that you know best and will refrain from asking for their own good, Director Nakamura!

Fascinatingly, Mervis secured an opinion that echoes this. So Nakamura will surely be reading it:

Riggs, the only African-American in his department, thinks the program is too brief to help minority scientists truly become part of the mainstream, and may even exacerbate their sense of being marginalized.

“After I sat on the panel, I realized there was a real network that exists, and I wasn't part of that network,” he says. “My comments as a reviewer weren't taken as seriously. And the people who serve on these panels get really nervous about having people … that they don't know, or who they think are not qualified, or who are not part of the establishment.”

If NIH “wants this to be real,” Riggs suggests having early-career researchers “serve as an ECR and then call them back in 2 years and have them serve a full cycle. I would have loved to do that.”

The person in the best position to decide what is good or bad for his or her career is the investigator themself.

This comment also speaks to my objection to the ECR as a baby-intro version of peer review. It isn't necessary. I first participated on study section in my Asst Prof years as a regular ad hoc with a load of about six grants, iirc. Might have been 2 less than the experienced folks had but it was not a baby-trainee experience in the least. I was treated as a new reviewer, but that was about the extent of it. I thought I was taken seriously and did not feel patronized.

__
Related Reading:
Toni Scarpa to leave CSR

More on one Scientific Society’s Response to the Scarpa Solicitation

Your Grant In Review: Junior Reviewers Are Too Focused on Details

The problem is not with review…

Peer Review: Opinions from our Elders

23 responses so far

NIH Director Collins and CSR Director Nakamura continue to kick the funding disparity can down the road

A News piece in Science by Jeffrey Mervis details the latest attempt of the NIH to kick the Ginther can down the road.

Armed with new data showing black applicants suffer a 35% lower chance of having a grant proposal funded than their white counterparts, NIH officials are gearing up to test whether reviewers in its study sections give lower scores to proposals from African-American applicants. They say it’s one of several possible explanations for a disparity in success rates first documented in a 2011 report by a team led by economist Donna Ginther of the University of Kansas, Lawrence.

Huh. 35%? I thought Ginther estimated more like a 13% difference? Oh wait. That's the award probability difference. About 16% versus 29% for white applicants which would be about a 45% lower chance. And this shows "78-90% the rate of white...applicants". And there was Nakamura quoted in another piece in Science:

At NIH, African-American researchers “receive awards at “55% to 60% the rate of white applicants,” Nakamura said. “That's a huge disparity that we have not yet been able to seriously budge,” despite special mentoring and networking programs, as well as an effort to boost the number of scientists from underrepresented minorities who evaluate proposals.

Difference vs rate vs lower chance.... Ugh. My head hurts. Anyway you spin it, African-American applicants are screwed. Substantially so.

Back to the Mervis piece for some factoids.

Ginther..noted...black researchers are more likely to have their applications for an R01 grant—the bread-and-butter NIH award that sustains academic labs—thrown out without any discussion...black scientists are less likely to resubmit a revised proposal ...whites submit at a higher rate than blacks...

So, what is CSR doing about it now? OK HOLD UP. LET ME REMIND YOU IT IS FIVE YEARS LATER. FIFTEEN FUNDING ROUNDS POST-GINTHER. Ahem.

The bias study would draw from a pool of recently rejected grant applications that have been anonymized to remove any hint of the applicant’s race, home institution, and training. Reviewers would be asked to score them on a one-to-nine scale using NIH’s normal rating system.

It's a start. Of course, this is unlikely to find anything. Why? Because the bias at grant review is a bias of identity. It isn't that reviewers are biased against black applicants, necessarily. It is that they are biased for white applicants. Or at the very least they are biased in favor of a category of PI ("established, very important") that just so happens to be disproportionately white. Also, there was this interesting simulation by Eugene Day that showed a bias that is smaller than the non-biased variability in a measurement can have large effects on something like a grant funding system [JournalLink].

Ok, so what else are they doing?

NIH continues to wrestle with the implications of the Ginther report. In 2014, in the first round of what NIH Director Francis Collins touted as a 10-year, $500 million initiative to increase the diversity of the scientific workforce, NIH gave out 5-year, $25 million awards to 10 institutions that enroll large numbers of minority students and created a national research mentoring network.

As you know, I am not a fan of these pipeline-enhancing responses. They say, in essence, that the current population of black applicant PIs is the problem. That they are inferior and deserve to get worse scores at peer review. Because what else does it mean to say the big money response of the NIH is to drum up more black PIs in the future by loading up the trainee cannon now?

This is Exhibit A of the case that the NIH officialdom simply cannot admit that there might be unfair biases at play that caused the disparity identified in Ginther and reinforced by the other mentioned analyses. The are bound and determined to prove that their system is working fine, nothing to see here.

So....what else ?

A second intervention starting later this year will tap that fledgling mentoring network to tutor two dozen minority scientists whose R01 applications were recently rejected. The goal of the intervention, which will last several months, is to prepare the scientists to have greater success on their next application. A third intervention will educate minority scientists on the importance of resubmitting a rejected proposal, because resubmitted proposals are three times more likely to be funded than a de novo application from a researcher who has never been funded by NIH.

Oh ff..... More of the same. Fix the victims.

Ah, here we go. Mervis finally gets around to explaining that 35% number

NIH officials recently updated the Ginther study, which examined a 2000–2006 cohort of applicants, and found that the racial disparity persists. The 35% lower chance of being funded comes from tracking the success rates of 1054 matched pairs of white and black applicants from 2008 to 2014. Black applicants continue to do less well at each stage of the process.

I wonder if they will be publishing that anywhere we can see it?

But here's the kicker. Even faced with the clear evidence from their own studies, the highest honchos still can't see it.

One issue that hung in the air was whether any of the disparity was self-inflicted. Specifically, council members and NIH officials pondered the tendency of African-American researchers to favor certain research areas, such as health disparities, women’s health, or hypertension and diabetes among minority populations, and wondered whether study sections might view the research questions in those areas as less compelling. Valantine called it a propensity “to work on issues that resonate with their core values.” At the same time, she said the data show minorities also do less well in competition with their white peers in those fields.

Collins offered another possibility. “I’ve heard stories that they might have been mentored to go into those areas as a better way to win funding,” he said. “The question is, to what extent is it their intrinsic interest in a topic, and to what extent have they been encouraged to go in that direction?”

Look, Ginther included a huge host of covariate analyses that they conducted to try to make the disparity go away. Now they've done a study with matched pairs of investigators. Valantine's quote may refer to this or to some other analysis I don't know but obviously the data are there. And Collins is STILL throwing up blame-the-victim chaff.

Dude, I have to say, this kind of denialist / crank behavior has a certain stench to it. The data are very clear and very consistent. There is a funding disparity.

This is a great time to remind everyone that the last time a major funding disparity came to the attention of the NIH it was the fate of the early career investigators. The NIH invented up the ESI designation, to distinguish it from the well established New Investigator population, and immediately started picking up grants out of the order of review. Establishing special quotas and paylines to redress the disparity. There was no talk of "real causes". There was not talk of strengthening the pipeline with better trainees so that one day, far off, they magically could better compete with the established. Oh no. They just picked up grants. And a LOT of them.

I wonder what it would take to fix the African-American PI disparity...

Ironically, because the pool of black applicants is so small, it wouldn’t take much to eliminate the disparity: Only 23 more R01 applications from black researchers would need to be funded each year to bring them to parity.

Are you KIDDING me? That's it?????

Oh right. I already figured this one out for them. And I didn't even have the real numbers.

In that 175 bin we'd need 3 more African-American PI apps funded to get to 100%. In the next higher (worse) scoring bin (200 score), about 56% of White PI apps were funded. Taking three from this bin and awarding three more AA PI awards in the next better scoring bin would plunge the White PI award probability from 56% to 55.7%. Whoa, belt up cowboy.

Moving down the curve with the same logic, we find in the 200 score bin that there are about 9 AA PI applications needed to put the 200 score bin to 100%. Looking down to the next worse scoring bin (225) and pulling these 9 apps from white PIs we end up changing the award probability for these apps from 22% to ..wait for it..... 20.8%.

Mere handfuls. I had probably overestimated how many black PIs were seeking funding. If this Mervis piece is to be trusted and it would only take 23 pickups across the entire NIH to fix the problem....

I DON'T UNDERSTAND WHAT FRANCIS COLLINS' PROBLEM IS.

Twenty three grants is practically rounding error. This is going to shake out to one or maybe three grants per year for the ICs, depending on size and what not.

Heck, I bet they fund this many grants every year by mistake. It's a big system. You think they don't have a few whoopsies sneak by every now and again? Of course they do.

But god forbid they should pick up 23 measly R01s to fix the funding disparity.

37 responses so far

NIH gets serious about the mouths-at-the-trough. Sortof.

Jun 06 2016 Published by under Fixing the NIH, NIH, NIH Careerism

The latest Open Mike blogpost from NIH Deputy Director for the OER, Mike Lauer, ventures into analysis of TheRealProblem at last.

The setup, in and of itself, is really good information.

We first looked at all research project grants (RPGs) funded between 2003 and 2015. For each year, we identified unique principal investigators who were named on at least one RPG award in that year. Figure 1 shows that the number of NIH-supported investigators has increased only slightly, and has remained fairly constant at about 27,500 over the past thirteen years.

Burn that one into your brain, people. There are about 27,500 unique PIs funded at any given time and this number has been rock steady for at least thirteen years. Sure, it is crazy-making stuff that they do not go back past the doubling interval to see what is really going on but hey, this is a significant improvement. At last the NIH is grappling with their enterprise by funded-investigators instead of funded-applications. This is a key addition and long, long overdue. I approve.

There are some related analyses from DataHound that lead into these considerations as well. I recommend you go back and read Longitudinal PI Analysis: Distributions, Mind the Gap and especially A longitudinal analysis of NIH R-Funded Investigators: 2006-2013. This latter one estimated a similar number of unique PIs but it also estimated the churn rate, that is, the number in each fiscal year that are new and the number who have left the funded-PI distribution (it was about 5,300 PIs per FY).

Back to Lauer's post for the supplicant information that DataHound couldn't get:

To determine how many unique researchers want to be funded, we identified unique applicants over 5-year windows. We chose to look at a multi-year window for two reasons: most research grants last for more than one year and most applicants submit applications over a period of time measured in years, not just 12 months, that may overlap with their periods of funding, if they are funded. Figure 2 shows our findings for applicants as well as awardees: the number of unique applicants has increased substantially, from about 60,000 investigators who had applied during the period from 1999 to 2003 to slightly less than 90,000 in who had applied during the period from 2011 to 2015.

via https://nexus.od.nih.gov/all/2016/05/31/how-many-researchers/

The too-many-mouths problem is illustrated. Simply. Cleanly. We can speculate about various factors until the cows come home but this is IT.

Too Many Mouths At The Trough And Not Enough Slops.

The blogpost then goes on to calculate a Cumulative Investigator Rate which is basically how many PIs get funded over a 5 year interval out of those who wish to be funded. In 2003 it was 43% and this declined to 31% in 2015. This was for RPGs. If you limit to R01 only, the CIR goes from 45% to 34% over this interval of time. For R21s, the CIR was at 20% in 2003 and is down around 11% for 2015. Newsbreak: Funding rates for R21s are terrible, despite what you would imagine should be the case for this mechanism.

Now we get to the hard part. Having reviewed these data the person responsible for the entire Extramural Research enterprise of NIH boots the obvious. Hard. First, he tries to off load the responsibility by citing Kimble et al and Pickett et al. Then he basically endorses their red herring distractions (when it comes to this particular issue).

NIH leadership is currently engaged in efforts to explore which policies or policy options best assure efficient and sustainable funding given the current hypercompetitive environment. These efforts include funding opportunity announcements for R35 awards which focus on programs, rather than highly specific projects; new models for training graduate students and postdoctoral fellows; establishment of an office of workforce diversity;

Right? It's right there in front of you, dude, and you can't even say it as one of a list of possible suggestions.

We need to stop producing so many PhD scientists.

This is the obvious solution. It is the only thing that will have sustained and systematic effect, while retaining some thin vestige of decency towards the people who have already devoted years and decades to the NIH extramural enterprise.

Oh and don't get me too wrong. From a personal perspective, clearly Lauer is not completely idiotic:

and even what we are doing here, namely drawing attention to numbers of unique investigators and applicants.

HAHAHHAHAHAAH. What a bureaucratic weasel. He sees it all right. He does. And he's trying to wink it into the conversation without taking any responsibility whatsoever. I see you, man. I see you. Okay. I'll take up the hard work for you.

We need to stop training so many PhDs. Now. Yesterday in fact. All of us. Stop pretending your high-falutin program gets to keep all their students and those inferior jerks, over there, need to close up shop. Significant reductions are called for.

Personally, I call for a complete moratorium on new PhD admits for 5 years.

Go.

124 responses so far

Collins announces NRSA salaries will meet Obama's overtime rule

In a piece on HuffPo, NIH Director Francis Collins announces the NIH response to Obama's new rules on overtime for salaried employees. Collins:

Under the new rule, which was informed by 270,000 public comments, the threshold will be increased to $47,476 effective December 1, 2016. ....In response to the proposed FLSA revisions, NIH will increase the awards for postdoctoral NRSA recipients to levels above the threshold.

"levels". Meaning, presumably the entire scale will start around $47.5K and move upward with years of postdoctoral experience, as the NRSA scale usually does.

What about the larger population of postdocs that are paid from non-NRSA funds, Dr. Collins?

..we recognize that research institutions that employ postdocs will need to readjust the salaries they pay to postdocs that are supported through other means, including other types of NIH research grants. While supporting the increased salaries will no doubt present financial challenges to NIH and the rest of the U.S. biomedical research enterprise, we plan to work closely with leaders in the postdoc and research communities to find creative solutions to ensure a smooth transition.

Imprecise and highly disappointing when it comes to the postdocs supported on "other types of NIH research grants". This would have been a great opportunity to state that the NIH expects any postdocs paid from RPGs to be on the NRSA scale, wouldn't it? Most postdocs are supported on NIH grants. This Rock Talk post shows in FY2009 something like 11,000 basic biomed postdocs on Federal research grants and only 1,000 on Federal fellowships and training grants (and ~7,800 on nonFederal support). So Francis Collins is talking the happy talk about 10% of the postdocs who work for him and throwing 90% into the storm.

The OER head, Michael Lauer, has a few more interesting points on the Open Mike blog.

Institutions that employ postdocs through non-NRSA support can choose how to follow the new rule. They may choose to carefully track their postdocs’ hours and pay overtime. Or, keeping with the fact that biomedical research – as in many professional and scientific careers – does not fit into neatly defined hourly shifts, institutions can choose to raise salaries to the new FLSA salary threshold or above it, if they do not yet pay postdocs at or above that level.

This would imply that Dr. Collins' supposed plan to "work closely with" and "ensure a smooth transition" is more realistically interpreted as "hey, good luck with the new Obama regs, dudes".

Before we get at it in the comments, a few lead off points from me:

The current NRSA scale pays 0 year postdocs $43,692 so in December the brand new postdoc will see a $4,000 raise, roughly. There is currently something on the order of $1,800 increases for each successive year of experience, this estimate is close enough for discussion purposes. If this yearly raise interval is maintained we can expect to see that same $4,000 pay rise applied to every salary level. Remember to apply your local benefits rate for the cost to a grant, if you are paying your postdocs at NRSA scale from RPG funds. Could turn this into a $5,000-$6,000 cost to the grant.

Postdocs getting paid more is great. Everyone in science should be paid more but there is something specific here. Postdocs frequently work more than 40 h per week for their salaried positions. This is right down the middle of the intent of Obama's change for the overtime rules. He is right on this. Period.

With that said, there is a very real disconnect here between the need to pay postdocs more and the business model which funds them. As mentioned above, 90% of Federally funded postdocs are supported by research grants, and 10% on fellowships or traineeships. (A population almost 8 times as large as the latter are supported by nonFederal funds- the percentage of these working on Federal research projects is likely to be substantial.) A grant may have one or two postdocs on it so adding another $5,000-$10,000 per year isn't trivial. Especially since the research grant budgets are constrained in a number of ways.

First, in time. We propose grants in a maximum of 5 year intervals but often the budget is designed one or two years prior to funding. These grant budgets are not supplemented in the middle of a competitively-awarded interval just because NRSA salary levels are increased. Given the way NRSA rises have been coming down randomly over the years, it is already the case that budgets are stretched. Despite what people seem to think (including at NIH), we PIs do not pad the heck out of our proposed research budgets. We can't. Our peers would recognize it on review and ding us accordingly.

Second, grants are constrained by the modular budgeting process which limits direct costs to $250,000 per year. This a soft and nebulous limit which depends on the culture of grant design, review and award. Formally speaking, one can choose a traditional budget process at any time if one needs to request funds in excess of $250,000 per year. Practically speaking, a lot of people choose to use the modular budget process. For reasons. The purchasing power has been declining for 15 years and there is no sign of a change in the expectations for per-grant scientific output.

Third, grant budgets are often limited by reductions to the requested budget that are imposed by the NIH. This can be levied upon original funding of the award or upon the award of each of the annual non-competing intervals of funding. These can often range to 10%, for argument's sake let's keep that $25,000 figure in mind when assessing the impact of such a reduction on paying a salary for a staff member, such as a postdoc. Point being, it's a big fraction of a salary. This new postdoc policy isn't going to result in fewer cuts or shallower cuts. Believe me.

I will be watching the way that local Universities choose to deal with the new policy with curiosity. I think we all see that trying to limit postdocs to 40 h a week of work so as to avoid raising the base salary is a ridiculous plan*. The other competitive motivations will continue to drive some postdocs to work more. This will put Universities (and PIs) in the extremely distasteful position of creating this elaborate fiction about working hours.

One potential upside for the good PI, who is already maintaining postdocs at NRSA levels even when funded from the RPG, is that it will force the bad PIs into line. This should narrow the competitive disadvantage that comes with trying to treat your postdocs well.

Final point. This will take away jobs. Fewer postdocs will be hired. Whether this is good or bad....well, opinions vary. But the math is unmistakable.

[UPDATE: The modular budget grant limit of $250,000 was established for R01s in FY2000 and the NRSA 0 year postdoc salary in FY2000 was $26,916. This is 10.8% of the direct costs of a full modular R01. In FY2017 when this new NRSA adjustment, the 0 year postdoc will be 19% of the direct costs of a full modular R01. In short the postdoc is now 76% more expensive than the postdoc was in FY2000.]
__
*It is, however, a failed opportunity to attempt to normalize academic science's working conditions. I see no reason we shouldn't take a stab at enforcing a 40 h work week in academic science, personally. Particularly for the grad student and post-doc labor force who are realistically not very different from the technicians who do, btw, enjoy most labor protections.

117 responses so far

Older posts »