Archive for the 'NIH Careerism' category

The NIH Cull and the K99/R00 cohorts

May 28 2015 Published by under Fixing the NIH, NIH, NIH Careerism

DataHound posted two key analyses on the state of the NIH-funded extramural work force. In the first one he presents the number of unique PIs from 1985-2014. It looks to me, roughly, that there are about 18% fewer PIs than the peak and approximately 10% fewer PIs if we ignore the ARRA interval.

Most of the real drop (i.e., not postARRA) occurred between FY2011 and FY2012 but there has been a downward trend from 2012 to 2014 so this looks to be the new reality.

The Cull is in full view now.

Has it seemed like grants are getting funded slightly more easily lately? If so, you can thank the Cull. (No doubt the pressure is more about applications than funded awards to unique PIs. But if the applications are seeing similar drops, this explains the feeling of relief, if you have it.)

The second post at DataHound presents several graphs on the K99/R00 awardees by original award year.

Transition to the R00 phase did not vary much up through the 2010 cohort and the cohorts are on the same trajectory, given the time function. Importantly the 2007-2009 cohorts follow the exact same trajectory, 2010 cohorts have a little bit of drop-off at the far end, due to less time since original award. Six years after the K99 award is the hard ceiling on transition to R00 in the first three cohorts and 2010 K99ers aren't quite there yet.

Where the K99 awardee cohorts are not on the same trajectory is the transition to R01. DataHound's plots show a clear plateau for the 2007-2010 cohorts. The 2007 awardees topped out at about 58% transitioning to R01 funding and subsequent cohort success rates are lower, year over year. Success in gaining an R01 for the 2009 cohort is about 70% that of the 2008 grouup and about half that of the 2007 cohort. The 2010 cohort is at least 20% less-successful than the 2009 K99 awardees.

It is pretty clear the Cull described in the first linked post is falling harder on the K99/R00 awardees than on the general pool of NIH-funded PIs. Depending on whether you take the ARRA high water mark for unique PIs or something lower that adheres to the normal trend, the Cull is only about a 10-20% as of FY2014.

This is BACKWARDS!

All this talk about getting more new scientists over the hump to faculty level career status. All this whinging and moaning about eating our seed corn. All the handwringing over ESIs.

And the program that is the crown jewel in doing something about transition is....not working.

If history is any guide, it would have taken official NIHdom about 15 years to "suddenly realize" this is the case and to try something new.

Thank goodness for DataHound. I anticipate he has accelerated this process by posting these two key analyses.

56 responses so far

Thought on the Ginther report on NIH funding disparity

May 24 2015 Published by under Fixing the NIH, NIH, NIH Careerism, NIH funding

I had a thought about Ginther just after hearing a radio piece on the Asian-Americans that are suing Harvard over entrance discrimination. 

The charge is that Asian-American students need to have better grades and scores  than white students to receive an admissions bid. 

The discussion of the Ginther study revolved around the finding that African-American applicant PIs were less likely than PIs of other groups to receive NIH grant funding. This is because Asian-Americans, for example, did as well as white PIs. Our default stance, I assume, is that being a white PI is the best that it gets. So if another group does as well, this is evidence of a lack of bias. 

But what if Asian-American PIs submit higher quality applications as a group? 
How would we ever know if there was discrination against them in NIH grant award?

20 responses so far

Thoughts on NIH grant strategy from Associate Professor H. Solo

We spend a fair amount of time talking about grant strategy on this blog. Presumably, this is a reflection of an internal process many of us go through trying to decide how to distribute our grant writing effort so as to maximize our chances of getting funded. After all we have better things to do than to write grants.

So we scrutinize success rates for various ICs, various mechanisms, FOAs, etc as best we are able. We flog RePORTER for evidence of which study sections will be most sympathetic to our proposals and how to cast our applications so as to be attractive. We worry about how to construct our Biosketch and who to include as consultants or collaborators. We obsess over how much preliminary data is enough (and too much*).

This is all well and good and maybe, maybe....perhaps....it helps.

But at some level, you have to follow your gut, too. Even when the odds seem overwhelmingly bad, there are going to be times when dang it, you just feel like this is the right thing to do.

Submitting an R01 on very thin preliminary data because it just doesn't work as an R21 perhaps.

Proposing an R03 scope project even if the relevant study section has only one** of them funded on the RePORTER books.

Submitting your proposal when the PO who will likely be handling it has already told you she hates your Aims***.

Revising that application that has been triaged twice**** and sending it back in as a A2asA0 proposal.

I would just advise that you take a balanced approach. Make your riskier attempts, sure, but balance those with some less risky applications too.

I view it as....experimenting.

__
*Just got a question about presenting too much preliminary data the other day.

**of course you want to make sure there is not a structural issue at work, such as the section stopped reviewing this mechanism two years ago.

***1-2%ile scores have a way of softening the stony cold heart of a Program Officer. Within-payline skips are very, very rare beasts.

****one of my least strategic behaviors may be in revising grants that have been triaged. Not sure I've ever had one funded after initial triage and yet I persist. Less so now than I used to but.....I have a tendency. Hard headed and stupid, maybe.

13 responses so far

NIH Program Officers do not understand what happens during review

May 22 2015 Published by under Grant Review, Grantsmanship, NIH, NIH Careerism

It is one of the most perplexing things of my career and I still don't completely understand why this is the case. But it is important for PIs, especially those who have not yet experienced study section, to understand a simple fact of life.

The NIH Program Officers do not completely understand what contributes to the review and scoring of your grant application.

My examples are legion and I have mentioned some of them in prior blog posts over the years.

The recent advice from NIAID on how to get your grant to fit within a modular budget limit.

The advice from a PO that PIs (such as myself) just needed to "write better grants" when I was already through a stint on study section and had read many, many crappy and yet funded grants from more established investigators.

The observation that transitioning investigators "shouldn't take that job" because it was soft money and K grants were figuring heavily in the person's transition/launch plans.

Apparently honest wonder that reviewers do not read their precious Program Announcements and automatically award excellent scores to applications just because they align with the goals of the PA.

Ignorance of the revision queuing that was particularly endemic during the early part of my career (and pretend? ignorance that limiting applications to one revision round made no functional difference in this).

The "sudden discovery" that all of the New Investigator grants during the checkbox era were going to well-established investigators who simply happened not to have NIH funding before, instead of boosting the young / recently appointed investigators.

An almost comically naive belief that study section outcome for grants really is an unbiased reflection of grant merit.

I could go on.

The reason this is so perplexing to me is that this is their job. POs [eta: used to] sit in on study section meetings or listen in on the phone. At least three times a year but probably more often given various special emphasis panels and the assignment of grants that might be reviewed in any of several study sections. They even take notes and are supposed to give feedback to the applicant with respect to the tenor of the discussion. They read any and all summary statements that they care to. They read (or can read) a nearly dizzying array of successful and unsuccessful applications.

And yet they mostly seem so ignorant of dynamics that were apparent to me after one, two or at the most three study section meetings.

It is weird.

The takeaway message for less NIH-experienced applicants is that the PO doesn't know everything. I'm not saying they are never helpful....they are. Occasionally very helpful. Difference between funded and not-funded helpful. So I fully endorse the usual advice to talk to your POs early and often.

Do not take the PO word for gospel, however. Take it under advisement and integrate it with all of your other sources of information to try to decide how to advance your funding strategy.

24 responses so far

Crystal clear grant advice from NIAID

May 21 2015 Published by under Grant Review, Grantsmanship, NIH, NIH Careerism

from this Advice Corner on modular budgeting:

As you design your research proposal, tabulate a rough cost estimate. If you are above but near the $250,000 annual direct cost threshold, consider ways to lessen your expenses. Maybe you have a low-priority Specific Aim that can be dropped or a piece of equipment you could rent rather than buy new.

H/t: PhysioProf

Related Reading:

NIAID
Sample Grants

26 responses so far

That study of peer review perplexes me

Apr 24 2015 Published by under Grant Review, NIH, NIH Careerism, Peer Review

I just can't understand what is valuable about showing that a 1%ile difference in voted score leads to 2% difference in total citations of papers attributed to that grant award. All discussions of whether NIH peer review is working or broken center on the supposed failure to fund meritorious grants and the alleged funding of non-meritorious grants. 

Please show me one PI that is upset that her 4%ile funded grant really deserved a 2%ile and that shows that peer review is horribly broken. 

The real issue, how a grant overlooked by the system would fare *were it to be funded* is actually addressed to some extent by the graph on citations to clearly outlying grants funded by exception.

This is cast as Program rescuing those rare exception brilliant proposal. But again, how do we know the ones that Program fails to rescue wouldn't have performed well?

23 responses so far

On productivity and the "unfair" grant funding game

Apr 13 2015 Published by under Careerism, NIH, NIH Careerism

There is an article up on ASBMB Today by Andrew D. Hollenbach that laments the shut-down of his research program. In The reality that dare not speak its name we learn:

It was the day after my lab manager left, forced to find a new job by a vicious funding environment that took a trusted employee and friend from me and shut down my research program.

This is terrible, I will acknowledge. I have feared this outcome for my own research program, only briefly interrupted, for my entire independent career. The wolves are always near the door and winter is most certainly coming.

Hollenbach finds this to be unfair. And that assertion triggers slightly more thought than mere sympathy and empathetic butt clenching.

I spent 20 years studying the mechanisms underlying a childhood muscle tumor. I published more than 20 articles with a lab of no more than three people at one time, intentionally kept small so I could focus on mentoring. I established a new paradigm in my field, identified viable therapeutic targets and trained five students (three of whom went to Harvard University for postdocs). I am recognized worldwide for my research.

You would think that all of that would be enough to bring in money and continue my research. But it’s not.

My immediate thought was no, no I don't think that is enough in this day and age. 20 papers in 20 years of an independent career is not a fantastic publishing rate. Of course, yes, there are going to be field and model specifics that greatly affect publishing rate. There will be differences in publishing style and venue as well...if this had been 20 CNS publications, well, this would be pretty good productivity. But a search of PubMed seems to confirm that the pursuit of the very highest Glamour publications was not the issue. I am not an expert in this guy's field of study but glancing over his publication titles and journals I get the distinct impression of a regular-old Jane/Joe type of scientist here. Many people can claim to have established new paradigms, sent trainees off to impressive-sounding postdoctoral stints (or assistant professorships) and to have identified 'viable' therapeutic targets. I say this not to belittle the guy but to point out that this is not in any way special. It is not an immediately obvious compensation for a rather underwhelming rate of publication. For a PI, that is, who asserts he's had a long-term lab manager and up to three people in his group at a time.

Hollenbach's funding hasn't been overwhelmingly generous but he's had NIH grants. RePORTER shows that he started with a component of a P20 Center grant from 2004-2009 and an R01 from 2009-2013.

Wait. What "20 years"?

Hollenbach's bio claims he was made junior faculty in 2001 and won his first Assistant Professor job in 2003. This matches up better with his funding history so I think we'd better just focus on the past 10 years to really take home a message about careerism. One senior author publication in 2003 from that junior-faculty stint and then the next one is 2007 and then three in 2008. So far, so good. Pretty understandable for the startup launch of a new laboratory.

Then we note that there is only one paper in each of 2009 and 2010. Hmmm. Things can happen, sure. Sure. Two papers in 2011 but one is a middle authorship. One more publication in each of 2012, 2014 and 2015 (to date). The R01 grant lists 7 pubs as supported but two of those were published before the grant was awarded and one was published 9 months into the first funding interval. So 5 pubs supported by the R01 in this second phase. And an average as a faculty member that runs just under a publication per year.

Lord knows I haven't hit an overwhelming publication output rate across my entire career. I understand slowdowns. These are going to happen now and again. And for certain sure there are going to be chosen model systems that generate publishable datasets more slowly than others.

But.

But.....

One paper per year, sustained across 10 years, is not the kind of productivity rate that people view as normal and average and unremarkable. Particularly when it comes to grant review at the NIH level.

I would be very surprised if the grant applications this PI has submitted did not receive a few comments questioning his publication output.

Look at my picture, and you will not see a failure. You will see someone who worked hard, excelled at what he did, held true to himself and maintained his integrity. However, you also will see someone whose work was brought to a halt by an unfair system.

Something else occurs to me. The R01 was funded up to March 2013. So this presumably means that this recent dismissal of the long-term lab manager comes after a substantial interval of grant submission deadlines? I do wonder how many grant applications the guy submitted and what the outcomes were. This would seem highly pertinent to the "unfair system" comment. You know my attitude, Dear Reader. If one is supported on a single grant, bets the farm on a competing continuation hitting right on schedule and is disappointed...this is not evidence of the system being unfair. If a PI is unfunded and submits a grant, waits for the reviews, skips a round, submits the revision, waits for the reviews, skips another round, writes a new proposal..... well, THIS IS NOT ENOUGH! This is not trying. And if you are not trying, you have no right to talk about the "unfair system" as it applies to your specific outcome.

I close, as I often do, with career advice. Don't do this people. Don't let yourself publish on the lower bound on what is considered an acceptable rate for your field, approaches, models and, most importantly, funding agency's review panels.

PS: This particular assertion regarding what surely must be necessary to survive as a grant-funded is grotesquely inaccurate.

Some may say that I did not do enough. Maybe I didn’t. I could have been a slave-driving mentor to get more publications in journals with higher impact factors. I could have worked 80-hour weeks, ignoring my family and friends. I could have given in to unfettered ambition, rolling over anyone who got in my way.

98 responses so far

A new Stock Criticism for NIH Grants

Apr 11 2015 Published by under Fixing the NIH, NIH Careerism

I have decided to deploy a brand new Stock Criticism.

 "No effort for a staff scientist is described, which may limit progress."

Feel free to borrow it.

5 responses so far

More data to explain.....attitudes. (UPDATED)

Mar 25 2015 Published by under NIH, NIH Budgets and Economics, NIH Careerism

via: http://twitter.com/MHendr1cks/status/580831188820090880

  

Rockey had posted on the amount of grant money going to age groups, this Tweep divided by the number of PIs in each group. 

I had two immediate thoughts.

When the end of the doubling hit, if you were 50 or under you felt it immediately. 

If you were 56 or older at that point, you didn't feel anything until 2012.

Funny how nicely this maps onto attitudes. We've seen the older types get vocal only in the last 2-3 years and we have been bemused.

My response has been "welcome to the reality the rest of us have been under for a decade." 

Nice to see some actual data confirming that the Boomers really have been insulated from pain until recently. 

UPDATED: More from @MHendr1cks
NIHGrantIncreaseByAgeThe piechart really brings it home, doesn't it?

23 responses so far

NCI pilots a staff scientist award

Mar 19 2015 Published by under Fixing the NIH, NIH Careerism

http://news.sciencemag.org/biology/2015/03/cancer-institute-plans-new-award-staff-scientists

Wow.
In July, 2007 I wrote:

Create career awards (not fellowships because of the way institutions use this to screw fellows out of the usual employment benefits) for that category of doctoral research scientist who is happy to labor away in someone else’s lab without being a PI. These people already exist, in great numbers and often work through to retirement in nebulous job categories. Let’s recognize that these people are an essential fuel for the NIH engine. It can be on the 5 yr cycle so that productivity is assessed and individuals are accountable to produce. This will create a great deal of independence in these individuals so that they are not beholden to one PI. Think of the side bennies on scientific fraud!

In Aug, 2008 I wrote:

I have a modest suggestion, of course. The K05 mechanism. Or rather, something much like the K05......Suppose something like this were made available for career Ph.D. scientists as essentially a fellowship. Without any requirement for a professorial appointment and minimal actual research component. The important point being that it is applied for, awarded to and evaluated for renewal by the career scientist with every expectation that this is a career award. There would be details of course. You'd have to have a host lab at most times- but allow for transition if one lab loses grant support or something. Nice and easy for the supported career scientist to find a new lab, don't you think? "Hey, PI Smith, I have my salary supported and I'd like to come play in your lab..." would go over quite nicely. Progress could be evaluated just as with any other award, keeping the pressure on for the individual to publish.

I commented at Rock Talking in Feb 2011:

Returning to the OP question about workforce, one of the most profound changes over 30 years is the length of time, sometimes career length, spent in the dark twilight of postdoc/superpostdoc/research scientist/etc in traditional academic settings.

Some could be perfectly happy in such a role if there were a little more career certainty, benefits and insulation from exploitative PIs.

One thing the NIH could do is create a K mech sort of like the K05 but intended for the staff scientist level. Career level benefits required. Has to be renewable too. It could be tied to Rmechs of a lab head (for the primary research support) but it should be easily switched to a different lab w/in the University if necessary. Competitive review would focus on productivity rather than the *specific* project.

I think you can see why I am so excited about what NCI is proposing [video link, start at 2:20] to do as described by Jocelyn Kaiser At ScienceInsider:

The K05 “research specialist award,” as NCI is calling it, would be aimed at scientists with a master’s, Ph.D., M.D., or other advanced degree holding positions such as lab research scientist, core facility manager, or data scientist.

Applicants would need to be sponsored by a PI and their institution. The award could cover up to 100% of their salary, but not research expenses. The 5-year, renewable award also would be “portable” if the recipient moved to another lab or institution.

Notes from the NCI presentation:
at 2:26:30 it is emphasized that this award has to be independent from a PI's grant.

slide at 2:26 notes it would be for individuals including but not limited to : lab research scientists, core facility managers and data scientists.

2:27 slide emphasizes "only to individuals who have made significant contribution to a cancer research program".

2:28 only for that portion of salary devoted to cancer research, expected to be at least 50% effort

[DM- this taps into a sticky point I've mentioned before which is how this is supposed to work for cross-IC scientists. I think they need to work this out better, maybe do it from the OD if necessary. It's all for the good of NIH, right? So they need to work out how to have a scientist be able to jump from a NCI lab to a NIGMS lab if necessary]

2:29 -the research proposal is to be written jointly by the applicant and the sponsoring PI, describing the research.

[DM- I think this is workable even though my eye started to twitch. There is going to be some slippage here with respect to the goals of making this award portable and not tied to the fate of one lab's research grant]

2:29:55 -Initially the Research Specialist to apply while supported on an existing research grant. Once the K05 is awarded, it would be expected to be 50/50 support with the grant and then continuing on the K05 100% once the grant ended.

2:30:30 - Review criteria. Accomplishment of applicant individually and within the nominating lab's program. Accomplishment of the PI and Uni. Importance of the applicant to the research program of the PI.

[DM- Welp. This is certainly going down a road of contributing to the rich getting richer which is not something I support. Unless "importance to the research program of the PI" means helping to stabilize the science of a have-not type of PI who struggles to maintain consistent funding.]

2:31- They are going to launch this via RFA as a pilot program. 50-60 awards planned over an 18 month period.

[DM- NICE!]

2:32: slide on portability of the award - possible but requires PO approval if PI and K05 move together, if the PI leaves and K05 stays, if the grant is lost, etc.

if K05 Specialist chooses on her/his own hook to leave old lab, it will require a new PI, approval, etc. The old PI is eligible for 2 year administrative supplement because they are "suddenly missing a critical support component".

[DM- ugh, this last part. Why should the original grant be compensated for the K05 person deciding to leave? It will already have benefited from that 50% free effort. Rich get richer, one. and a reward for that scenario where the PI is such a jerkface that the K05 leaves him/her? no. and regarding "critical support component", dude, what about when any postdoc chooses to leave? happens all the time. can I get some free money for suddenly missing an awesome postdoc?]

2:36 on assessment of the pilot. "critical to get input from the PI about how well their needs have been served"

[DM- well sure. but...... grrrr. this should be about the K05 awardee's perspective. The whole point is that the existing system puts these people's careers into the hands of the big cheese PI. That is what the focus should be on here. The K05 Research Specialist. Not on whether the PI's loss of control has allowed him or her to continue to exploit or whether this is just a way to shield the haves of the world from the grant game a little bit more.]

Q/A:
Bar-Sagi: Restrict the applicants to PhDs? Should Core Directors be excluded (because business model of the U makes security different)? 2:39:20- situation where it "backfires" on the lab

Golub: "but, the quid pro quo is that they (the staff sci type) exchange the lack of obligation to raise their own salary for the lack of independence". wants to know if somehow this is a bait and switch

[DM- well yeah, but that ship has sailed. the goal here is to fix the part where staff scientists can no longer rely on the BigCheese just being endlessly funded forever with out interruption]

2:46 Gray: " a mechanism by which one could survive a hiccup in funding".

[DM ugh- the "one" here is clearly meant as the PI. Sooooooo focused on the PI and not the K05 person.....]

83 responses so far

Older posts »