Archive for the 'NIH' category

Columbia University busted for taking too much overhead on NIH grants

Jul 15 2016 Published by under NIH, NIH Budgets and Economics

From the Manhattan branch of the US Attorney's Office charged:

The United States’ Complaint-In-Intervention (the “Complaint”) alleges that from July 1, 2003, through June 30, 2015, COLUMBIA impermissibly applied its “on-campus” indirect cost rate – instead of the much lower “off-campus” indirect cost rate – when seeking federal reimbursement for 423 NIH grants where the research was primarily performed at off-campus facilities owned and operated by the State of New York and New York City. The Complaint further alleges that COLUMBIA failed to disclose to NIH that it did not own or operate these facilities and that COLUMBIA did not pay for use of the space for most of the relevant period.

...and Columbia University admitted:

COLUMBIA has admitted that it applied the on-campus indirect cost rate to the 423 NIH grants even though the research was primarily performed in space not owned or operated by Columbia, and that it submitted to NIH certified reports that used the on-campus indirect cost rate to calculate the indirect cost amounts claimed by the university.

Ah, those tricky accountants.

Oh. cool. paging down on the complaint we get some specifics:

From July 1, 2003, through June 30, 2015, COLUMBIA’s On-Campus F&A Rate was approximately 61 percent, its Off-Campus F&A Rate was 26 percent, and its Modified Off-Campus F&A Rate was 29.4 percent. The Modified Off-Campus F&A Rate was to be applied to research conducted off-campus but within a certain proximity of the COLUMBIA campus.

Y'know. When I first read this my first thought was that I know some Columbia folks that work off campus at.... oh shit. It's them. It's the drug abuse folks.

COLUMBIA has a collaborative relationship with the New York State Psychiatric Institute (“NYSPI”), a clinical research facility administered by the New York State Office of Mental Health. COLUMBIA faculty perform research in two off-campus buildings owned by the State of New York and operated by NYSPI (the “NYSPI Buildings”). COLUMBIA faculty also perform research in another off-campus building owned and operated by the City of New York (the “City Building”).

For most of the relevant period, COLUMBIA did not pay the State of New York for use of the NYSPI Buildings, and therefore did not incur indirect “facilities-related” costs with respect to the medical research performed in these buildings. Similarly, COLUMBIA did not pay the City of New York for use of the City Building.

Presumably my friends who are the PIs on these grants had no idea. I have no idea what my institution actually charges the NIH as overhead on my grants, all I look at is my direct cost expenditures and balances. But still, sorry to see that it was their research grants that were involved. If nothing else it means that NIDA [Update: I found some details and 22/423 total grants were driect listings from NIDA, although there are what look like subaward identifiers (that may or may not involve other NIDA grants.)] was the entity being ripped off. The settlement was for $9.5 million. It doesn't say how much of this is direct recovery for the fraud and how much is court costs or punishment.

oh, wait. Damn. This looks bad.

COLUMBIA did not state on the applications for the NIH Grants that the research would be primarily performed off-campus, as required. Instead, Columbia frequently included the main address for the College of Physicians & Surgeons in the section of the application that was supposed to list the primary performance location. Even where the NYSPI Buildings or the City Building were listed in that section of the grant application, or mentioned elsewhere in the application, COLUMBIA failed to disclose that these buildings were not owned and operated by the university.

Starting in fiscal year 2009, in lieu of paying rent for use of one of the NYSPI Buildings, the Department of Neuroscience paid NYSPI a portion of the inflated indirect cost recoveries it received from NIH for research projects performed in that building.

This smells a lot more like highly intentional fraud and less like a mistake that someone should have caught. In the pre-award review of the grant, if you ask me. Especially when CU was clearly negotiating the rental arrangements with NYSPI. Someone pretty high up in the office of grants and contracts had to be doing this whole charade intentionally and with planning. There are a handful of other regulatory issues that I don't want to get into which very likely pointed a spotlight on the "performance location" too. This had to be intentional.

Turns out that this was a whistleblower case.

In connection with the filing of the lawsuit and settlement, the Government joined a private whistleblower lawsuit that had previously been filed under seal pursuant to the False Claims Act.

Good for that brave person for bringing this to light.

__
Final thought: I bet you that Columbia University is not the only NIH funded University out there that pulls some shenanigans like this. Now, you would think that there would be some sort of broad and universal alert sent to the Signing Officials of each University that has an on- and off-campus rate. Telling them to get their act together on this or any future investigation that busts them will automatically have the fines tripled. But going by at least one narrow similar area that I've followed over the past couple of decades (the anti-lobbying / grant writing thing) apparently this does not happen. So keep your eyes peeled for the next decade. I bet there will be more of these and that in each case it will again be figured out only via whistleblower.

46 responses so far

The R01 still doesn't pay for itself and reviewers are getting worse

Jul 11 2016 Published by under NIH, NIH Careerism, NIH funding

I pointed out some time ago that the full modular R01 grant from the NIH doesn't actually pay for itself.

In the sense that there is a certain expectation of productivity, progress, etc on the part of study sections and Program that requires more contribution than can be afforded (especially when you put it in terms of 40 hr work weeks) within the budget. Trainees on individual fellowships or training grants, undergrads working for free or work study discount, cross pollination with other grants in the lab (which often leads to whinging like your comment), pilot awards for small bits, faculty hard money time...all of these sources of extra effort are frequently poured into a one-R01 project. I think they are, in essence, necessary.

I had some additional thoughts on this recently.

It's getting worse.

Look, it has always been the case that reviewers want to see more in a grant proposal. More controls, usually. Extra groups to really nail down the full breadth of...whatever it is that you are studying. This really cool other line of converging evidence... anything is possible.

All I can reflect is my own experience in getting my proposals reviewed and in reviewing proposals that are somewhat in the same subfields.

What I see is a continuing spiral of both PI offerings and of reviewer demands.

It's inevitable, really. If you see a proposal chock full of nuts that maybe doesn't quite get over the line of funding because of whatever reason, how can you give a fundable score to a very awesome and tight proposal that is more limited?

Conversely, in the effort to put your best foot forward you, as applicant, are increasingly motivated to throw every possible tool at your disposal into the proposal, hoping to wow the reviewers into submission.

I have reviewed multiple proposals recently that cannot be done. Literally. They cannot be accomplished for the price of the budget proposed. Nobody blinks an eye about this. They might talk about "feasibility" in the sense of scientific outcomes or preliminary data or, occasionally, some perceived deficit of the investigators/environment. But I have not heard a reviewer say "nice but there is no way this can be accomplished for $250K direct". Years ago people used to crab about "overambitious" proposals but I can't say I've heard that in forever. In this day and age of tight NIH paylines, the promises of doing it all in one R01 full-modular 5 year interval are escalating.

These grants set a tone, btw. I'm here to tell you that I've seen subfield related proposals that do seem feasible, money-wise, get nailed because they are too limited in scope. In some cases there is enough study-section continuity involved for me to be certain that this is due to reviewer contamination from the aforementioned chock-full-o-nuts impossible proposals. Yes, some of this is due to SABV but not all of it. It ranges from "why you no include more co-investigators?" (a subtle spread-the-wealth knock on big labs? maybe) to "You really need to add X, Y and Z to be convincing" (mkay but... $250K dude) to "waaah, I just want to see more" (even though they don't really have a reason to list).

Maybe this is just me being stuck in the rut I was trained in. In my formative years, grant review seemed to expect you would propose a set of studies that you could actually accomplish within the time frame and budget proposed. I seem to remember study section members curbing each other with "Dude, the PI can't fit all that stuff into one proposal, back off.". I used to see revisions get improved scores when the PI stripped a bloated proposal down to a minimalist streamlined version.

Maybe we are just experiencing a meaningless sea change in grant review to where we propose the sky and nobody cares on competing renewal if we managed to accomplish all of that stuff.

37 responses so far

Where the NIGMS argument doesn't add up

Jul 08 2016 Published by under Fixing the NIH, NIH Budgets and Economics

The NIGMS continues its ongoing argument for funding more labs with ever decreasing amounts of grant funding in a new Feedback Loop post.

This one focuses, yet again, on "productivity" as assessed by publication counts and (this time) citations of those publications. It is, as always, significantly flawed by ignoring the effects of Glamour publications. I've done that before and it is starting to bore me. In short, you cannot compare apples to oranges because of the immense difference in the cost of generating your average Nature paper versus a Brain Research paper. And citations don't help because getting into a Glam journal does not mean your paper will get any particular number of citations. Furthermore, there is very little chance that papers that cost 10 or 20 times more will generate ten or twenty times the citations, on average, given the skew in citation distributions and the fact that Glam journals are only hitting means in the 30-40 range. Finally, their "efficiency" measures completely ignore the tremendous inefficiencies of interrupted funding, which is a reality under the current system and also not necessarily fixed with their spread-the-wealth schemes.

The real issue of the day is the opinion of the fans of NIGMS's "conclusion*", which reads:

Overall, however, the data suggest that supporting a greater number of investigators at moderate funding levels is a better investment strategy than concentrating high amounts of funding in a smaller number of researchers.

The Sally Rockey blog entry on "mythbusting" is relevant here. As of FY2009 about 72% of NIH funded investigators had one RPG. Another 20% had two and maybe 5% had three.

That's all.

The NIGMS data analyses are big on fitting productivity lines to about the single R01 level of direct costs (~$200K per year) and showing how the productivity/cost drops off as the grant funding increases. Take a good look at the most recent analysis. Linear productivity up to $300K direct costs with the 75%ile sustained all the way to $500K. The famous original 2010 analysis by Jeremy Berg at NIGMS is pretty similar in the sense that you don't get much change in the line fit to mean publications until you get to the $600-$700K direct costs range.

There is a critical point in lining up these two bits of information which is that the NIGMS policy intent is not supported by their analysis and it can't be. One or two RPG level from Rockey's post should be interpreted in full modular R01 terms ($250K direct, usually cut to $200K, $225K direct and in NIGMS' case to 4 years by default) with a little bit of float upwards for the rare cases. Consequently, it is obvious that most NIH awardees operate in the ~$200-250K part of NIGMS' dataset. Another 20% operate in the $400-$500K direct range. In other words, well within the linear part of the productivity/cost curve.

Mean publications as represented by the 2010 Berg analysis are increasing linearly well up to the three to four grant level of $750K direct costs.

In either case, the "inefficient" grant levels are being obtained by a vanishingly small number of investigators.

Fine, screw them, right?

Sure....but this does nothing to address either the stated goal of NIGMS in hedging their bets across many labs or the goal of the unfunded, i.e., to increase their chances substantially.

A recent Mike Lauer Blog post showed that about a third of those PI's who seek RPG funding over a rolling 5 year interval achieve funding. Obviously if you take all the multi-grant PIs and cut them down to one tomorrow, you'd be able to bump funded investigators up by 15-20%, assuming the FY2009 numbers are relatively good still**. It isn't precise because if you limit the big guys to one award then these are going to drift up to $499K direct at a minimum and a lot more will have special permission to crest the $500K threshold.

There will be a temporary sigh of relief and some folks will get funded at 26%ile. Sure. And then there will be even more PIs in the game seeking funding and it will continue to be a dogfight to retain that single grant award. And the next round of newbies will face the same steep odds of entry. Maybe even steeper.

So the ONLY way for NIGMS' plan to work is to cut per-PI awards way, way down into the front part of their productivity curves. Well below the point of inflection($300-500K or even $750K depending on measure) where papers-per-grant dollar drops off the linear trend. Even the lowest estimate of $300K direct is more than one full-modular grant. It will take a limit substantially below this level*** to improve perceptions of funding ease or to significantly increase the number of funded labs.

Which makes their argument based on those trends a lie, if they truly intend it to support their "better investment strategy". Changing the number of investigators they support in any fundamental way means limiting per-PI awards to the current full modular limit (with typical reductions) at the least, and very likely substantially below this level to produce anything like a phase change.

That's fine if they want to just assert "we think everyone should only have X amount of direct costs" but it is not so fine if they argue that they have some objective, productivity-based data analysis to support their plans. Because it does not.

__
*This is actually their long standing assertion that all of these seemingly objective analyses are designed to support.

**should be ballpark, given the way Program has been preserving unfunded labs at the expense of extra awards to funded labs these days.

***I think many people arguing in favor of the NIGMS type of "small grants for all" strategy operate from the position that they personally deserve funding. Furthermore that some grant award of full modular level or slightly below is sufficient for them. Any dishonest throwaway nod to other types of research that are more expensive (as NIGMS did "We recognize that some science is inherently more expensive, for example because of the costs associated with human and animal subjects.") is not really meant or considered. This is somewhat narrow and self-involved. Try assuming that all of the two-granters in Rockey's distribution really need that amount of funding (remember the erosion of purchasing power?) and that puts it at more like 92% of awardees that enjoy basic funding at present. Therefore the squeeze should be proportional. Maybe the bench jockeys should be limited to $100K or even $50K in this scenario? Doesn't seem so attractive if you consider taking the same proportional hit, does it?

22 responses so far

Power in the NIH review trenches

Jun 18 2016 Published by under Fixing the NIH, NIH, NIH Careerism

That extensive quote from a black PI who had participated in the ECR program is sticking with me.

Insider status isn't binary, of course. It is very fluid within the grant-funded science game. There are various spectra along multiple dimensions.

But make no mistake it is real. And Insider status is advantageous. It can be make-or-break crucial to a career at many stages.

I'm thinking about the benefits of being a full reviewer with occasional/repeated ad hoc status or full membership.

One of those benefits is that other reviewers in SEPs or closely related panels are less likely to mess with you.

Less likely.

It isn't any sort of quid pro quo guarantee. Of course not. But I guarantee that a reviewer who thinks this PI might be reviewing her own proposal in the near future has a bias. A review cant. An alerting response. Whatever.

It is different. And, I would submit, generally to the favor of the applicant that possesses this Mutually Assured Destruction power.

The Ginther finding arose from a thousand cuts, I argue. This is possibly one of them.

3 responses so far

Nakamura reports on the ECR program

Jun 17 2016 Published by under Fixing the NIH, Grant Review, NIH, NIH Careerism

If I stroke out today it is all the fault of MorganPhD.

Jeffery Mervis continues with coverage of the NIH review situation as it pertains to the disparity for African-American PIs identified in 2011 (that's five years and fifteen funding rounds ago, folks) by the Ginther report.

The main focus for this week is on the Early Career Reviewer program. As you will recall, this blog has advocated continually and consistently for the participation of more junior PIs on grant review panels.

The ECR program was created explicitly to deal with underrepresented groups. However, what happened is that there was immediate opposition which insisted that the ECR program had to be open to all junior faculty/applicants, regardless of representation in the NIH game.

One-quarter of researchers in ECR's first cohort were from minority groups, he notes. “But as we've gone along, there are fewer underrepresented minorities coming into the pool.”
...
Minorities comprise only 13% of the roughly 5100 researchers accepted into the program (6% African-American and 7% Hispanic), a percentage that roughly matches their current representation on study sections.

Ok, but how have the ECR participants fared?

[Nakamura] said ECR alumni have been more than twice as successful as the typical new investigator in winning an R01 grant.

NIIIIIICE. Except they didn't flog the data as hard as one might hope. This is against the entire NI (or ESI?) population.

The pool of successful ECR alumni includes those who revised their application, sometimes more than once, after getting feedback on a declined proposal. That extra step greatly improves the odds of winning a grant. In contrast, the researchers in the comparison group hadn't gone through the resubmission process.

Not sure if this really means "hadn't" or "hadn't necessarily". The latter makes more sense if they are just comparing to aggregate stats. CSR data miners would have had to work harder to get this isolated to those who hadn't revised yet, and I suspect if they had gone to that effort, they could have presented the ESIs who had at least one revision under their belt. But what about the underrepresented group of PIs that are the focus of all this effort?

It's also hard to interpret the fact that 18% of the successful ECRs were underrepresented minorities because NIH did not report the fraction of minorities among ECR alumni applicants. So it is not clear whether African-Americans participating in the program did any better than the cohort as a whole—suggesting that the program might begin to close the racial gap—or better than a comparable group of minority scientists who were not ECR alumni.

SERIOUSLY Richard Nakamura? You just didn't happen to request your data miners do the most important analysis? How is this even possible?

How on earth can you not be keeping track of applicants to ECR, direct requests from SROs, response rate and subsequent grant and reviewing behavior? It is almost as if you want to look like you are doing something but have no interest in it being informative or in generating actionable intelligence.

Moving along, we get a further insight into Richard Nakamura and his position in this situation.

Nakamura worries that asking minority scientists to play a bigger role in NIH's grantsmaking process could distract them from building up their lab, finding stable funding, and earning tenure. Serving on a study section, he says, means that “those individuals will have less time to write applications. So we need to strike the right balance.”

Paternalistic nonsense. The same thing that Scarpa tried to use to justify his purge of Assistant Professors from study sections. My answer is the same. Let them decide. For themselves. Assistant Professors and underrepresented PIs can decide for themselves if they are ready and able to take up a review opportunity when asked. Don't decide, paternalistically, that you know best and will refrain from asking for their own good, Director Nakamura!

Fascinatingly, Mervis secured an opinion that echoes this. So Nakamura will surely be reading it:

Riggs, the only African-American in his department, thinks the program is too brief to help minority scientists truly become part of the mainstream, and may even exacerbate their sense of being marginalized.

“After I sat on the panel, I realized there was a real network that exists, and I wasn't part of that network,” he says. “My comments as a reviewer weren't taken as seriously. And the people who serve on these panels get really nervous about having people … that they don't know, or who they think are not qualified, or who are not part of the establishment.”

If NIH “wants this to be real,” Riggs suggests having early-career researchers “serve as an ECR and then call them back in 2 years and have them serve a full cycle. I would have loved to do that.”

The person in the best position to decide what is good or bad for his or her career is the investigator themself.

This comment also speaks to my objection to the ECR as a baby-intro version of peer review. It isn't necessary. I first participated on study section in my Asst Prof years as a regular ad hoc with a load of about six grants, iirc. Might have been 2 less than the experienced folks had but it was not a baby-trainee experience in the least. I was treated as a new reviewer, but that was about the extent of it. I thought I was taken seriously and did not feel patronized.

__
Related Reading:
Toni Scarpa to leave CSR

More on one Scientific Society’s Response to the Scarpa Solicitation

Your Grant In Review: Junior Reviewers Are Too Focused on Details

The problem is not with review…

Peer Review: Opinions from our Elders

23 responses so far

NIH Director Collins and CSR Director Nakamura continue to kick the funding disparity can down the road

A News piece in Science by Jeffrey Mervis details the latest attempt of the NIH to kick the Ginther can down the road.

Armed with new data showing black applicants suffer a 35% lower chance of having a grant proposal funded than their white counterparts, NIH officials are gearing up to test whether reviewers in its study sections give lower scores to proposals from African-American applicants. They say it’s one of several possible explanations for a disparity in success rates first documented in a 2011 report by a team led by economist Donna Ginther of the University of Kansas, Lawrence.

Huh. 35%? I thought Ginther estimated more like a 13% difference? Oh wait. That's the award probability difference. About 16% versus 29% for white applicants which would be about a 45% lower chance. And this shows "78-90% the rate of white...applicants". And there was Nakamura quoted in another piece in Science:

At NIH, African-American researchers “receive awards at “55% to 60% the rate of white applicants,” Nakamura said. “That's a huge disparity that we have not yet been able to seriously budge,” despite special mentoring and networking programs, as well as an effort to boost the number of scientists from underrepresented minorities who evaluate proposals.

Difference vs rate vs lower chance.... Ugh. My head hurts. Anyway you spin it, African-American applicants are screwed. Substantially so.

Back to the Mervis piece for some factoids.

Ginther..noted...black researchers are more likely to have their applications for an R01 grant—the bread-and-butter NIH award that sustains academic labs—thrown out without any discussion...black scientists are less likely to resubmit a revised proposal ...whites submit at a higher rate than blacks...

So, what is CSR doing about it now? OK HOLD UP. LET ME REMIND YOU IT IS FIVE YEARS LATER. FIFTEEN FUNDING ROUNDS POST-GINTHER. Ahem.

The bias study would draw from a pool of recently rejected grant applications that have been anonymized to remove any hint of the applicant’s race, home institution, and training. Reviewers would be asked to score them on a one-to-nine scale using NIH’s normal rating system.

It's a start. Of course, this is unlikely to find anything. Why? Because the bias at grant review is a bias of identity. It isn't that reviewers are biased against black applicants, necessarily. It is that they are biased for white applicants. Or at the very least they are biased in favor of a category of PI ("established, very important") that just so happens to be disproportionately white. Also, there was this interesting simulation by Eugene Day that showed a bias that is smaller than the non-biased variability in a measurement can have large effects on something like a grant funding system [JournalLink].

Ok, so what else are they doing?

NIH continues to wrestle with the implications of the Ginther report. In 2014, in the first round of what NIH Director Francis Collins touted as a 10-year, $500 million initiative to increase the diversity of the scientific workforce, NIH gave out 5-year, $25 million awards to 10 institutions that enroll large numbers of minority students and created a national research mentoring network.

As you know, I am not a fan of these pipeline-enhancing responses. They say, in essence, that the current population of black applicant PIs is the problem. That they are inferior and deserve to get worse scores at peer review. Because what else does it mean to say the big money response of the NIH is to drum up more black PIs in the future by loading up the trainee cannon now?

This is Exhibit A of the case that the NIH officialdom simply cannot admit that there might be unfair biases at play that caused the disparity identified in Ginther and reinforced by the other mentioned analyses. The are bound and determined to prove that their system is working fine, nothing to see here.

So....what else ?

A second intervention starting later this year will tap that fledgling mentoring network to tutor two dozen minority scientists whose R01 applications were recently rejected. The goal of the intervention, which will last several months, is to prepare the scientists to have greater success on their next application. A third intervention will educate minority scientists on the importance of resubmitting a rejected proposal, because resubmitted proposals are three times more likely to be funded than a de novo application from a researcher who has never been funded by NIH.

Oh ff..... More of the same. Fix the victims.

Ah, here we go. Mervis finally gets around to explaining that 35% number

NIH officials recently updated the Ginther study, which examined a 2000–2006 cohort of applicants, and found that the racial disparity persists. The 35% lower chance of being funded comes from tracking the success rates of 1054 matched pairs of white and black applicants from 2008 to 2014. Black applicants continue to do less well at each stage of the process.

I wonder if they will be publishing that anywhere we can see it?

But here's the kicker. Even faced with the clear evidence from their own studies, the highest honchos still can't see it.

One issue that hung in the air was whether any of the disparity was self-inflicted. Specifically, council members and NIH officials pondered the tendency of African-American researchers to favor certain research areas, such as health disparities, women’s health, or hypertension and diabetes among minority populations, and wondered whether study sections might view the research questions in those areas as less compelling. Valantine called it a propensity “to work on issues that resonate with their core values.” At the same time, she said the data show minorities also do less well in competition with their white peers in those fields.

Collins offered another possibility. “I’ve heard stories that they might have been mentored to go into those areas as a better way to win funding,” he said. “The question is, to what extent is it their intrinsic interest in a topic, and to what extent have they been encouraged to go in that direction?”

Look, Ginther included a huge host of covariate analyses that they conducted to try to make the disparity go away. Now they've done a study with matched pairs of investigators. Valantine's quote may refer to this or to some other analysis I don't know but obviously the data are there. And Collins is STILL throwing up blame-the-victim chaff.

Dude, I have to say, this kind of denialist / crank behavior has a certain stench to it. The data are very clear and very consistent. There is a funding disparity.

This is a great time to remind everyone that the last time a major funding disparity came to the attention of the NIH it was the fate of the early career investigators. The NIH invented up the ESI designation, to distinguish it from the well established New Investigator population, and immediately started picking up grants out of the order of review. Establishing special quotas and paylines to redress the disparity. There was no talk of "real causes". There was not talk of strengthening the pipeline with better trainees so that one day, far off, they magically could better compete with the established. Oh no. They just picked up grants. And a LOT of them.

I wonder what it would take to fix the African-American PI disparity...

Ironically, because the pool of black applicants is so small, it wouldn’t take much to eliminate the disparity: Only 23 more R01 applications from black researchers would need to be funded each year to bring them to parity.

Are you KIDDING me? That's it?????

Oh right. I already figured this one out for them. And I didn't even have the real numbers.

In that 175 bin we'd need 3 more African-American PI apps funded to get to 100%. In the next higher (worse) scoring bin (200 score), about 56% of White PI apps were funded. Taking three from this bin and awarding three more AA PI awards in the next better scoring bin would plunge the White PI award probability from 56% to 55.7%. Whoa, belt up cowboy.

Moving down the curve with the same logic, we find in the 200 score bin that there are about 9 AA PI applications needed to put the 200 score bin to 100%. Looking down to the next worse scoring bin (225) and pulling these 9 apps from white PIs we end up changing the award probability for these apps from 22% to ..wait for it..... 20.8%.

Mere handfuls. I had probably overestimated how many black PIs were seeking funding. If this Mervis piece is to be trusted and it would only take 23 pickups across the entire NIH to fix the problem....

I DON'T UNDERSTAND WHAT FRANCIS COLLINS' PROBLEM IS.

Twenty three grants is practically rounding error. This is going to shake out to one or maybe three grants per year for the ICs, depending on size and what not.

Heck, I bet they fund this many grants every year by mistake. It's a big system. You think they don't have a few whoopsies sneak by every now and again? Of course they do.

But god forbid they should pick up 23 measly R01s to fix the funding disparity.

37 responses so far

NIH gets serious about the mouths-at-the-trough. Sortof.

Jun 06 2016 Published by under Fixing the NIH, NIH, NIH Careerism

The latest Open Mike blogpost from NIH Deputy Director for the OER, Mike Lauer, ventures into analysis of TheRealProblem at last.

The setup, in and of itself, is really good information.

We first looked at all research project grants (RPGs) funded between 2003 and 2015. For each year, we identified unique principal investigators who were named on at least one RPG award in that year. Figure 1 shows that the number of NIH-supported investigators has increased only slightly, and has remained fairly constant at about 27,500 over the past thirteen years.

Burn that one into your brain, people. There are about 27,500 unique PIs funded at any given time and this number has been rock steady for at least thirteen years. Sure, it is crazy-making stuff that they do not go back past the doubling interval to see what is really going on but hey, this is a significant improvement. At last the NIH is grappling with their enterprise by funded-investigators instead of funded-applications. This is a key addition and long, long overdue. I approve.

There are some related analyses from DataHound that lead into these considerations as well. I recommend you go back and read Longitudinal PI Analysis: Distributions, Mind the Gap and especially A longitudinal analysis of NIH R-Funded Investigators: 2006-2013. This latter one estimated a similar number of unique PIs but it also estimated the churn rate, that is, the number in each fiscal year that are new and the number who have left the funded-PI distribution (it was about 5,300 PIs per FY).

Back to Lauer's post for the supplicant information that DataHound couldn't get:

To determine how many unique researchers want to be funded, we identified unique applicants over 5-year windows. We chose to look at a multi-year window for two reasons: most research grants last for more than one year and most applicants submit applications over a period of time measured in years, not just 12 months, that may overlap with their periods of funding, if they are funded. Figure 2 shows our findings for applicants as well as awardees: the number of unique applicants has increased substantially, from about 60,000 investigators who had applied during the period from 1999 to 2003 to slightly less than 90,000 in who had applied during the period from 2011 to 2015.

via https://nexus.od.nih.gov/all/2016/05/31/how-many-researchers/

The too-many-mouths problem is illustrated. Simply. Cleanly. We can speculate about various factors until the cows come home but this is IT.

Too Many Mouths At The Trough And Not Enough Slops.

The blogpost then goes on to calculate a Cumulative Investigator Rate which is basically how many PIs get funded over a 5 year interval out of those who wish to be funded. In 2003 it was 43% and this declined to 31% in 2015. This was for RPGs. If you limit to R01 only, the CIR goes from 45% to 34% over this interval of time. For R21s, the CIR was at 20% in 2003 and is down around 11% for 2015. Newsbreak: Funding rates for R21s are terrible, despite what you would imagine should be the case for this mechanism.

Now we get to the hard part. Having reviewed these data the person responsible for the entire Extramural Research enterprise of NIH boots the obvious. Hard. First, he tries to off load the responsibility by citing Kimble et al and Pickett et al. Then he basically endorses their red herring distractions (when it comes to this particular issue).

NIH leadership is currently engaged in efforts to explore which policies or policy options best assure efficient and sustainable funding given the current hypercompetitive environment. These efforts include funding opportunity announcements for R35 awards which focus on programs, rather than highly specific projects; new models for training graduate students and postdoctoral fellows; establishment of an office of workforce diversity;

Right? It's right there in front of you, dude, and you can't even say it as one of a list of possible suggestions.

We need to stop producing so many PhD scientists.

This is the obvious solution. It is the only thing that will have sustained and systematic effect, while retaining some thin vestige of decency towards the people who have already devoted years and decades to the NIH extramural enterprise.

Oh and don't get me too wrong. From a personal perspective, clearly Lauer is not completely idiotic:

and even what we are doing here, namely drawing attention to numbers of unique investigators and applicants.

HAHAHHAHAHAAH. What a bureaucratic weasel. He sees it all right. He does. And he's trying to wink it into the conversation without taking any responsibility whatsoever. I see you, man. I see you. Okay. I'll take up the hard work for you.

We need to stop training so many PhDs. Now. Yesterday in fact. All of us. Stop pretending your high-falutin program gets to keep all their students and those inferior jerks, over there, need to close up shop. Significant reductions are called for.

Personally, I call for a complete moratorium on new PhD admits for 5 years.

Go.

123 responses so far

Your Grant in Review: Investigator Independence

May 27 2016 Published by under NIH, NIH Careerism, NIH funding

superkash started it:


and amplified it:

there was a bit of chatter and then eventually AListScientist asserted:

First, I addressed Independence of a NIH Grant PI to some extent way back in 2007, reposted in 2009.

I, as well as several other colleagues who review grants, have noticed a seemingly sharp uptick in the number of applications coming in from PIs who are more "transitioning" than "transitioned". PIs whose job titles might be something other than "Assistant Professor" and ones who are still in or around the same laboratory or research group in which they have done a big chunk of postdoctoral work. In extreme cases the PI might still be titled "Postdoc" or have trained in the same place essentially since graduate school!

Readers of this blog might conclude that this trend, which I've been noticing for at least the past 3-4 rounds, delights me. And to the extent that it represents a recognition of the problems with junior scientist's making the career transition to independence this does appear a positive step. To the extent that it opens up artificial barriers blocking the next generation of scientists- great.

The slightly more cynical view expressed by colleagues and, admittedly, myself is that this trend has been motivated by IC Program behavior both in capping per-PI award levels and in promoting grant success for New Investigators. In other words that the well-established PIs with very large research groups are thinking that grants for which they would otherwise be the PI will now be more successful with some convenient patsy long-term postdoc at the helm. The science, however, is just the same old stuff of the established group and PI.

I surmise that the tweeting of @superkash was related to this conundrum. I would suggest to newcomers to the NIH system that these issues are still alive and well and contribute in various ways to grant review outcome. We see very clearly in various grant/career related discussion on twitter, this blog and commentary to various NIH outlets that peer scientists have strong ideas on categories of PI that deserve or don't deserve funding. For example in the recent version on CSR's Peer Review website, comments suggest we should keel the yuge labs, keel the ESIs, keel the riffraff noobs and save the politically disconnected. The point being that peer reviewers come with biases for and against the PI(s) (and to lesser extent the other investigators).

The fact that the Investigator criterion is one of the five biggies (and there is no official suggestion that it is of any lesser importance than Approach, Significance or Innovation) permits (and one might say requires) the reviwers to exercise these biases. It also shows that AListScientist's apparent belief that Investigators are not to be evaluated because the applicant University has certified them is incorrect.

The official CSR Guidance on review of the Investigator criterion is posed as a series of questions:

Are the PD/PIs, collaborators, and other researchers well suited to the project? If Early Stage Investigators or New Investigators, or in the early stages of independent careers, do they have appropriate experience and training? If established, have they demonstrated an ongoing record of accomplishments that have advanced their field(s)? If the project is collaborative or multi-PD/PI, do the investigators have complementary and integrated expertise; are their leadership approach, governance and organizational structure appropriate for the project?

"well suited"
"appropriate experience"
Right there you can see where the independence of the PI might be of interest to the reviewer.

"have they demonstrated an ongoing record of accomplishments"
We what to know what they personally have accomplished. Or caused to have accomplished if you want to natter about PIs not really doing hands on science. The point is, can this PI make the proposed studies happen? Is there evidence that she has done so before? Or is there merely evidence that he has existed as glorified hands in the PIs lab up to this point in time?

"are their leadership approach, governance and organizational structure appropriate for the project?"

Can they lead? Can they boot all the tails hard enough to get this project accomplished? I say that this is an entirely appropriate consideration.

I hope you do as well and I would be interested to hear a counter argument.

I suspect that most of the pushback on this comes from the position of thinking about the Research Assistant Professor who IS good enough. Who HAS operated more or less independently and led projects in the SuperLab.

The question for grant review is, how are we to know? From the record and application in front of us.

__
I am unable to leave this part off: If you are a RAP or heading to be one as a mid to late stage postdoc, the exhortation to you is to lay down evidence of your independence as best you are able. Ask Associate Professor peers that you know about what possible steps you can take to enhance the optics of you-as-PI on this.

73 responses so far

Postdoc salaries and reinforcer value

One issue I've heard raised is that some PIs like to use salary differentials to reward the "good postdocs" with bonus pay.

Given the behaviorist education that lurks in my background, I am theoretically* in support of this notion.

The new salary rules may minimize such flexibility in the future.

Are you aware of labs in which merit of postdocs as interpreted by the PI leads to salary differentials?

Is this a legitimate complaint about the overtime rules?

Will PIs use the permission to work overtime (and be paid for it) as a workaround for merit pay?
__
*Given my distaste for workplace bias and desire to be a fair manager, I have never used merit to decide postdoc pay. I stick to NRSA schedules and to institutional adjustments as appropriate.

40 responses so far

Collins announces NRSA salaries will meet Obama's overtime rule

In a piece on HuffPo, NIH Director Francis Collins announces the NIH response to Obama's new rules on overtime for salaried employees. Collins:

Under the new rule, which was informed by 270,000 public comments, the threshold will be increased to $47,476 effective December 1, 2016. ....In response to the proposed FLSA revisions, NIH will increase the awards for postdoctoral NRSA recipients to levels above the threshold.

"levels". Meaning, presumably the entire scale will start around $47.5K and move upward with years of postdoctoral experience, as the NRSA scale usually does.

What about the larger population of postdocs that are paid from non-NRSA funds, Dr. Collins?

..we recognize that research institutions that employ postdocs will need to readjust the salaries they pay to postdocs that are supported through other means, including other types of NIH research grants. While supporting the increased salaries will no doubt present financial challenges to NIH and the rest of the U.S. biomedical research enterprise, we plan to work closely with leaders in the postdoc and research communities to find creative solutions to ensure a smooth transition.

Imprecise and highly disappointing when it comes to the postdocs supported on "other types of NIH research grants". This would have been a great opportunity to state that the NIH expects any postdocs paid from RPGs to be on the NRSA scale, wouldn't it? Most postdocs are supported on NIH grants. This Rock Talk post shows in FY2009 something like 11,000 basic biomed postdocs on Federal research grants and only 1,000 on Federal fellowships and training grants (and ~7,800 on nonFederal support). So Francis Collins is talking the happy talk about 10% of the postdocs who work for him and throwing 90% into the storm.

The OER head, Michael Lauer, has a few more interesting points on the Open Mike blog.

Institutions that employ postdocs through non-NRSA support can choose how to follow the new rule. They may choose to carefully track their postdocs’ hours and pay overtime. Or, keeping with the fact that biomedical research – as in many professional and scientific careers – does not fit into neatly defined hourly shifts, institutions can choose to raise salaries to the new FLSA salary threshold or above it, if they do not yet pay postdocs at or above that level.

This would imply that Dr. Collins' supposed plan to "work closely with" and "ensure a smooth transition" is more realistically interpreted as "hey, good luck with the new Obama regs, dudes".

Before we get at it in the comments, a few lead off points from me:

The current NRSA scale pays 0 year postdocs $43,692 so in December the brand new postdoc will see a $4,000 raise, roughly. There is currently something on the order of $1,800 increases for each successive year of experience, this estimate is close enough for discussion purposes. If this yearly raise interval is maintained we can expect to see that same $4,000 pay rise applied to every salary level. Remember to apply your local benefits rate for the cost to a grant, if you are paying your postdocs at NRSA scale from RPG funds. Could turn this into a $5,000-$6,000 cost to the grant.

Postdocs getting paid more is great. Everyone in science should be paid more but there is something specific here. Postdocs frequently work more than 40 h per week for their salaried positions. This is right down the middle of the intent of Obama's change for the overtime rules. He is right on this. Period.

With that said, there is a very real disconnect here between the need to pay postdocs more and the business model which funds them. As mentioned above, 90% of Federally funded postdocs are supported by research grants, and 10% on fellowships or traineeships. (A population almost 8 times as large as the latter are supported by nonFederal funds- the percentage of these working on Federal research projects is likely to be substantial.) A grant may have one or two postdocs on it so adding another $5,000-$10,000 per year isn't trivial. Especially since the research grant budgets are constrained in a number of ways.

First, in time. We propose grants in a maximum of 5 year intervals but often the budget is designed one or two years prior to funding. These grant budgets are not supplemented in the middle of a competitively-awarded interval just because NRSA salary levels are increased. Given the way NRSA rises have been coming down randomly over the years, it is already the case that budgets are stretched. Despite what people seem to think (including at NIH), we PIs do not pad the heck out of our proposed research budgets. We can't. Our peers would recognize it on review and ding us accordingly.

Second, grants are constrained by the modular budgeting process which limits direct costs to $250,000 per year. This a soft and nebulous limit which depends on the culture of grant design, review and award. Formally speaking, one can choose a traditional budget process at any time if one needs to request funds in excess of $250,000 per year. Practically speaking, a lot of people choose to use the modular budget process. For reasons. The purchasing power has been declining for 15 years and there is no sign of a change in the expectations for per-grant scientific output.

Third, grant budgets are often limited by reductions to the requested budget that are imposed by the NIH. This can be levied upon original funding of the award or upon the award of each of the annual non-competing intervals of funding. These can often range to 10%, for argument's sake let's keep that $25,000 figure in mind when assessing the impact of such a reduction on paying a salary for a staff member, such as a postdoc. Point being, it's a big fraction of a salary. This new postdoc policy isn't going to result in fewer cuts or shallower cuts. Believe me.

I will be watching the way that local Universities choose to deal with the new policy with curiosity. I think we all see that trying to limit postdocs to 40 h a week of work so as to avoid raising the base salary is a ridiculous plan*. The other competitive motivations will continue to drive some postdocs to work more. This will put Universities (and PIs) in the extremely distasteful position of creating this elaborate fiction about working hours.

One potential upside for the good PI, who is already maintaining postdocs at NRSA levels even when funded from the RPG, is that it will force the bad PIs into line. This should narrow the competitive disadvantage that comes with trying to treat your postdocs well.

Final point. This will take away jobs. Fewer postdocs will be hired. Whether this is good or bad....well, opinions vary. But the math is unmistakable.

[UPDATE: The modular budget grant limit of $250,000 was established for R01s in FY2000 and the NRSA 0 year postdoc salary in FY2000 was $26,916. This is 10.8% of the direct costs of a full modular R01. In FY2017 when this new NRSA adjustment, the 0 year postdoc will be 19% of the direct costs of a full modular R01. In short the postdoc is now 76% more expensive than the postdoc was in FY2000.]
__
*It is, however, a failed opportunity to attempt to normalize academic science's working conditions. I see no reason we shouldn't take a stab at enforcing a 40 h work week in academic science, personally. Particularly for the grad student and post-doc labor force who are realistically not very different from the technicians who do, btw, enjoy most labor protections.

112 responses so far

« Newer posts Older posts »