Archive for the 'NIH' category

On productivity and the "unfair" grant funding game

Apr 13 2015 Published by under Careerism, NIH, NIH Careerism

There is an article up on ASBMB Today by Andrew D. Hollenbach that laments the shut-down of his research program. In The reality that dare not speak its name we learn:

It was the day after my lab manager left, forced to find a new job by a vicious funding environment that took a trusted employee and friend from me and shut down my research program.

This is terrible, I will acknowledge. I have feared this outcome for my own research program, only briefly interrupted, for my entire independent career. The wolves are always near the door and winter is most certainly coming.

Hollenbach finds this to be unfair. And that assertion triggers slightly more thought than mere sympathy and empathetic butt clenching.

I spent 20 years studying the mechanisms underlying a childhood muscle tumor. I published more than 20 articles with a lab of no more than three people at one time, intentionally kept small so I could focus on mentoring. I established a new paradigm in my field, identified viable therapeutic targets and trained five students (three of whom went to Harvard University for postdocs). I am recognized worldwide for my research.

You would think that all of that would be enough to bring in money and continue my research. But it’s not.

My immediate thought was no, no I don't think that is enough in this day and age. 20 papers in 20 years of an independent career is not a fantastic publishing rate. Of course, yes, there are going to be field and model specifics that greatly affect publishing rate. There will be differences in publishing style and venue as well...if this had been 20 CNS publications, well, this would be pretty good productivity. But a search of PubMed seems to confirm that the pursuit of the very highest Glamour publications was not the issue. I am not an expert in this guy's field of study but glancing over his publication titles and journals I get the distinct impression of a regular-old Jane/Joe type of scientist here. Many people can claim to have established new paradigms, sent trainees off to impressive-sounding postdoctoral stints (or assistant professorships) and to have identified 'viable' therapeutic targets. I say this not to belittle the guy but to point out that this is not in any way special. It is not an immediately obvious compensation for a rather underwhelming rate of publication. For a PI, that is, who asserts he's had a long-term lab manager and up to three people in his group at a time.

Hollenbach's funding hasn't been overwhelmingly generous but he's had NIH grants. RePORTER shows that he started with a component of a P20 Center grant from 2004-2009 and an R01 from 2009-2013.

Wait. What "20 years"?

Hollenbach's bio claims he was made junior faculty in 2001 and won his first Assistant Professor job in 2003. This matches up better with his funding history so I think we'd better just focus on the past 10 years to really take home a message about careerism. One senior author publication in 2003 from that junior-faculty stint and then the next one is 2007 and then three in 2008. So far, so good. Pretty understandable for the startup launch of a new laboratory.

Then we note that there is only one paper in each of 2009 and 2010. Hmmm. Things can happen, sure. Sure. Two papers in 2011 but one is a middle authorship. One more publication in each of 2012, 2014 and 2015 (to date). The R01 grant lists 7 pubs as supported but two of those were published before the grant was awarded and one was published 9 months into the first funding interval. So 5 pubs supported by the R01 in this second phase. And an average as a faculty member that runs just under a publication per year.

Lord knows I haven't hit an overwhelming publication output rate across my entire career. I understand slowdowns. These are going to happen now and again. And for certain sure there are going to be chosen model systems that generate publishable datasets more slowly than others.

But.

But.....

One paper per year, sustained across 10 years, is not the kind of productivity rate that people view as normal and average and unremarkable. Particularly when it comes to grant review at the NIH level.

I would be very surprised if the grant applications this PI has submitted did not receive a few comments questioning his publication output.

Look at my picture, and you will not see a failure. You will see someone who worked hard, excelled at what he did, held true to himself and maintained his integrity. However, you also will see someone whose work was brought to a halt by an unfair system.

Something else occurs to me. The R01 was funded up to March 2013. So this presumably means that this recent dismissal of the long-term lab manager comes after a substantial interval of grant submission deadlines? I do wonder how many grant applications the guy submitted and what the outcomes were. This would seem highly pertinent to the "unfair system" comment. You know my attitude, Dear Reader. If one is supported on a single grant, bets the farm on a competing continuation hitting right on schedule and is disappointed...this is not evidence of the system being unfair. If a PI is unfunded and submits a grant, waits for the reviews, skips a round, submits the revision, waits for the reviews, skips another round, writes a new proposal..... well, THIS IS NOT ENOUGH! This is not trying. And if you are not trying, you have no right to talk about the "unfair system" as it applies to your specific outcome.

I close, as I often do, with career advice. Don't do this people. Don't let yourself publish on the lower bound on what is considered an acceptable rate for your field, approaches, models and, most importantly, funding agency's review panels.

PS: This particular assertion regarding what surely must be necessary to survive as a grant-funded is grotesquely inaccurate.

Some may say that I did not do enough. Maybe I didn’t. I could have been a slave-driving mentor to get more publications in journals with higher impact factors. I could have worked 80-hour weeks, ignoring my family and friends. I could have given in to unfettered ambition, rolling over anyone who got in my way.

98 responses so far

A new Stock Criticism for NIH Grants

Apr 11 2015 Published by under Fixing the NIH, NIH Careerism

I have decided to deploy a brand new Stock Criticism.

 "No effort for a staff scientist is described, which may limit progress."

Feel free to borrow it.

5 responses so far

McKnight posts an analysis of NIH peer review

Apr 08 2015 Published by under NIH, NIH Budgets and Economics, NIH funding, Peer Review

Sortof.

In his latest column at ASBMB Today, Steve McKnight attempts to further his assertion that peer review of NIH grants needs to be revamped so that more qualified reviewers are doing the deciding about what gets funded.

He starts off with a comment that further reveals his naivete and noobitude when it comes to these issues.

Reviewers judge the application using five criteria: significance, investigator, innovation, approach and environment. Although study sections may weigh the importance of these criteria to differing degrees, it seems to me that feasibility of success of the proposed research plan (approach) tends to dominate. I will endeavor to provide a quantitative assessment of this in next month’s essay.

The NIH, led by then-NIGMS Director Berg, already provided this assessment. Ages ago. Try to keep up. I mention this because it is becoming an obvious trend that McKnight (and, keep in mind, many of his co-travelers that don't reveal their ignorance quite so publicly) spouts off his ill-informed opinions without the benefit of the data that you, Dear Reader, have been grappling with for several years now .

As reported last month, 72 percent of reviewers serving the HHMI are members of the National Academy of Sciences. How do things compare at the NIH? Data kindly provided by the CSR indicate that there were 7,886 reviewers on its standing study sections in 2014. Evaluation of these data reveals the following:

48 out of 324 HHMI investigators (15 percent) participated in at least one study section meeting.
47 out of 488 NIH-funded NAS members (10 percent) participated in at least one study section meeting.
11 of these reviewers are both funded by HHMI and NAS members.

These 84 scientists constituted roughly 1.1 percent of the reviewer cadre utilized by the CSR.

This tells us nearly nothing of importance. How many investigators from other pertinent slices of the distribution serve? ASBMB members, for example? PIs from the top 20, 50, 100 funded Universities and Medical Schools? How many applications do NAS / HHMI investigators submit each year? In short, are they over- or under-represented in the NIH review system?

Anyway, why focus on these folks?

I have focused on the HHMI investigators and NAS members because it is straightforward to identify them and quantify their participation in the review process. It is my belief that HHMI investigators and NIH-funded members of the NAS are substantively accomplished. I readily admit that scientific accomplishment does not necessarily equate to effective capacity to review. I do, however, believe that a reasonable correlation exists between past scientific accomplishment and capacity to choose effectively between good and poor bets. This contention is open for debate and is — to me — of significant importance.

So confused. First, the supposed rationale that these elite scientists are readily discernible folks amongst a host of well qualified so that's why he has used them for his example, aka the Street Lamp excuse. Next we get a ready admission that his entire thesis he's been pursuing since the riff-raff column is flawed, followed immediately by a restatement of his position based on..."belief". While admitting it is open to debate.

So how has he moved the discussion forward? All that we have at this point is his continued assertion of his position. The data on study section participation do exactly nothing to address his point.


Third, it is clear that HHMI investigators and NIH-funded members of the NAS participate in study sections charged with the review of basic research to a far greater extent than clinical research. It is my belief that study sections involving HHMI investigators and NAS members benefit from the involvement of highly accomplished scientists. If that is correct, the quality of certain basic science study sections may be high.

Without additional information this could be an entirely circular argument. If HHMI and NAS folks are selected disproportionally for their pursuit of basic science (I believe they are, Professor McKnight. Shall you accept my "belief" as we are expected to credit yours? or perhaps should you have looked into this?) they of course they would be disproportioanlly on "basic" study sections. If only there were a clinically focused organization of elite good-old-backslappers-club folks to provide a suitable comparison of more clinically-focused scientists.

McKnight closes with this:

I assume that it is a common desire of our biomedical community that all sources of funding, be they private or public, find their way to the support of our most qualified scientists — irrespective of age, gender, ethnicity, geographical location or any other variable. In subsequent essays, I will offer ideas as to how the NIH system of grant award distribution might be altered to meet this goal.

Nope. We want the funding to go to the most important science. Within those constraints we want the funding to go to highly qualified scientists but we recognize that "the most qualified" is a fool's errand. Other factors come in to play. Such as "the most qualified who are not overloaded with other research projects at the moment". Or, "the most qualified who are not essentially carbon copies of the three other folks funded in similar research at the moment".

This is even before we get into the very thorny argument over qualifications and how we identify the "most" qualified for any particular purpose.

McKnight himself admits to this when he claims that there are lots of other qualified people but he selected HHMI/NAS out of mere convenience. I wonder if it will eventually trickle into his understanding that this mere convenience pollutes his entire thinking on this matter?

h/t: philapodia

48 responses so far

Do try to keep up

Mar 26 2015 Published by under Fixing the NIH, NIH, NIH Budgets and Economics

I hope you all have read through the Bridges to Independence (2005) report. Yes? It's freely downloadable and told us a lot about the state of NIH extramural funding, age cohorts and demographic disparities....a DECADE ago.

So when Rockey posts abbreviated data sets.....yeah.

3 responses so far

More data to explain.....attitudes. (UPDATED)

Mar 25 2015 Published by under NIH, NIH Budgets and Economics, NIH Careerism

via: http://twitter.com/MHendr1cks/status/580831188820090880

  

Rockey had posted on the amount of grant money going to age groups, this Tweep divided by the number of PIs in each group. 

I had two immediate thoughts.

When the end of the doubling hit, if you were 50 or under you felt it immediately. 

If you were 56 or older at that point, you didn't feel anything until 2012.

Funny how nicely this maps onto attitudes. We've seen the older types get vocal only in the last 2-3 years and we have been bemused.

My response has been "welcome to the reality the rest of us have been under for a decade." 

Nice to see some actual data confirming that the Boomers really have been insulated from pain until recently. 

UPDATED: More from @MHendr1cks
NIHGrantIncreaseByAgeThe piechart really brings it home, doesn't it?

23 responses so far

NCI pilots a staff scientist award

Mar 19 2015 Published by under Fixing the NIH, NIH Careerism

http://news.sciencemag.org/biology/2015/03/cancer-institute-plans-new-award-staff-scientists

Wow.
In July, 2007 I wrote:

Create career awards (not fellowships because of the way institutions use this to screw fellows out of the usual employment benefits) for that category of doctoral research scientist who is happy to labor away in someone else’s lab without being a PI. These people already exist, in great numbers and often work through to retirement in nebulous job categories. Let’s recognize that these people are an essential fuel for the NIH engine. It can be on the 5 yr cycle so that productivity is assessed and individuals are accountable to produce. This will create a great deal of independence in these individuals so that they are not beholden to one PI. Think of the side bennies on scientific fraud!

In Aug, 2008 I wrote:

I have a modest suggestion, of course. The K05 mechanism. Or rather, something much like the K05......Suppose something like this were made available for career Ph.D. scientists as essentially a fellowship. Without any requirement for a professorial appointment and minimal actual research component. The important point being that it is applied for, awarded to and evaluated for renewal by the career scientist with every expectation that this is a career award. There would be details of course. You'd have to have a host lab at most times- but allow for transition if one lab loses grant support or something. Nice and easy for the supported career scientist to find a new lab, don't you think? "Hey, PI Smith, I have my salary supported and I'd like to come play in your lab..." would go over quite nicely. Progress could be evaluated just as with any other award, keeping the pressure on for the individual to publish.

I commented at Rock Talking in Feb 2011:

Returning to the OP question about workforce, one of the most profound changes over 30 years is the length of time, sometimes career length, spent in the dark twilight of postdoc/superpostdoc/research scientist/etc in traditional academic settings.

Some could be perfectly happy in such a role if there were a little more career certainty, benefits and insulation from exploitative PIs.

One thing the NIH could do is create a K mech sort of like the K05 but intended for the staff scientist level. Career level benefits required. Has to be renewable too. It could be tied to Rmechs of a lab head (for the primary research support) but it should be easily switched to a different lab w/in the University if necessary. Competitive review would focus on productivity rather than the *specific* project.

I think you can see why I am so excited about what NCI is proposing [video link, start at 2:20] to do as described by Jocelyn Kaiser At ScienceInsider:

The K05 “research specialist award,” as NCI is calling it, would be aimed at scientists with a master’s, Ph.D., M.D., or other advanced degree holding positions such as lab research scientist, core facility manager, or data scientist.

Applicants would need to be sponsored by a PI and their institution. The award could cover up to 100% of their salary, but not research expenses. The 5-year, renewable award also would be “portable” if the recipient moved to another lab or institution.

Notes from the NCI presentation:
at 2:26:30 it is emphasized that this award has to be independent from a PI's grant.

slide at 2:26 notes it would be for individuals including but not limited to : lab research scientists, core facility managers and data scientists.

2:27 slide emphasizes "only to individuals who have made significant contribution to a cancer research program".

2:28 only for that portion of salary devoted to cancer research, expected to be at least 50% effort

[DM- this taps into a sticky point I've mentioned before which is how this is supposed to work for cross-IC scientists. I think they need to work this out better, maybe do it from the OD if necessary. It's all for the good of NIH, right? So they need to work out how to have a scientist be able to jump from a NCI lab to a NIGMS lab if necessary]

2:29 -the research proposal is to be written jointly by the applicant and the sponsoring PI, describing the research.

[DM- I think this is workable even though my eye started to twitch. There is going to be some slippage here with respect to the goals of making this award portable and not tied to the fate of one lab's research grant]

2:29:55 -Initially the Research Specialist to apply while supported on an existing research grant. Once the K05 is awarded, it would be expected to be 50/50 support with the grant and then continuing on the K05 100% once the grant ended.

2:30:30 - Review criteria. Accomplishment of applicant individually and within the nominating lab's program. Accomplishment of the PI and Uni. Importance of the applicant to the research program of the PI.

[DM- Welp. This is certainly going down a road of contributing to the rich getting richer which is not something I support. Unless "importance to the research program of the PI" means helping to stabilize the science of a have-not type of PI who struggles to maintain consistent funding.]

2:31- They are going to launch this via RFA as a pilot program. 50-60 awards planned over an 18 month period.

[DM- NICE!]

2:32: slide on portability of the award - possible but requires PO approval if PI and K05 move together, if the PI leaves and K05 stays, if the grant is lost, etc.

if K05 Specialist chooses on her/his own hook to leave old lab, it will require a new PI, approval, etc. The old PI is eligible for 2 year administrative supplement because they are "suddenly missing a critical support component".

[DM- ugh, this last part. Why should the original grant be compensated for the K05 person deciding to leave? It will already have benefited from that 50% free effort. Rich get richer, one. and a reward for that scenario where the PI is such a jerkface that the K05 leaves him/her? no. and regarding "critical support component", dude, what about when any postdoc chooses to leave? happens all the time. can I get some free money for suddenly missing an awesome postdoc?]

2:36 on assessment of the pilot. "critical to get input from the PI about how well their needs have been served"

[DM- well sure. but...... grrrr. this should be about the K05 awardee's perspective. The whole point is that the existing system puts these people's careers into the hands of the big cheese PI. That is what the focus should be on here. The K05 Research Specialist. Not on whether the PI's loss of control has allowed him or her to continue to exploit or whether this is just a way to shield the haves of the world from the grant game a little bit more.]

Q/A:
Bar-Sagi: Restrict the applicants to PhDs? Should Core Directors be excluded (because business model of the U makes security different)? 2:39:20- situation where it "backfires" on the lab

Golub: "but, the quid pro quo is that they (the staff sci type) exchange the lack of obligation to raise their own salary for the lack of independence". wants to know if somehow this is a bait and switch

[DM- well yeah, but that ship has sailed. the goal here is to fix the part where staff scientists can no longer rely on the BigCheese just being endlessly funded forever with out interruption]

2:46 Gray: " a mechanism by which one could survive a hiccup in funding".

[DM ugh- the "one" here is clearly meant as the PI. Sooooooo focused on the PI and not the K05 person.....]

83 responses so far

Your Grant in Review: Effort and systems designed for amateur scientists

Mar 11 2015 Published by under Grant Review, Grantsmanship, NIH, NIH Careerism

Since we're discussing the amount of PI salary that should be rightfully paid by the NIH versus a local University lately, I have a grant review scenario to mention.

It is not uncommon to see R01 proposals come in from PIs who say that they will charge the grant for "three months summer salary". As we know, this is likely a scenario where the Professor in question has a 9 month salary from his or her University and is permitted to supplement that with up to three months of salary from extramural support funds.

Let us assume we're talking a normal research plan for an R01 that involves research effort pretty much around the calendar year. We're not talking about something that requires focal field work for a few summer months and then can subside into a much lower level of activity for the rest of the year.

On first glance the reviewer can only assume that the PI's remaining 9 months are being paid by the University to DO SOMETHING. Despite comment from Neuro-conservative about situations that seem very strange and unique, my experience is that Universities put some expectation of non-research activity on that 9 month of salary*.

Unless the PI has specified an expectation of research in their official job description, the reviewer can only assume that the effort on the grant will only be available during the summer.

Such a proposal should be met with the utmost skepticism since the conduct of the research requires ongoing supervision of the staff**, at the very least. Right?

So the grantsmithing advice part of this post is that if you are in this sort of situation, be sure to make very clear what your University explicitly expects in terms of your nine-month-hard-salary time.

From the perspective of our ongoing discussion, how is this all supposed to work? What true amount of brain-second-cycles are available to the project at any given time throughout the year?

Teaching duties tend to be rather inelastic and research duties tend to be highly elastic. I can always put off working on a paper or data analysis for another day. I can pick and choose when to work on a poster or oral presentation. I can't really put off lecture at 8am just because I have some exciting results in the laboratory that I want to write up right now. Grading may be a teeensy bit more flexible but there are clear deadlines...unlike paper submissions and most unlike designing new research projects and/or collaborations. Also very unlike meeting with your grad students and postdocs about various things.

I would suggest that under the 50/50 time scenario proposed by Neuro-conservative, one of the two task demands is going to receive short-shrift in a large number of cases. This will mostly be determined by what type of University the PI is employed within. Those that lean towards research? Well, we all know about how the tenure stool really only has one leg. Research. Conversely, there are very high teaching load institutions that inevitably push research toward the background during the active instructional school year.

In these situations either the NIH is being fleeced to support undergraduate instruction or the undergraduate instruction support system (State general funds and tuition, the latter includes scholarships and the like btw, another interested party) is being fleeced to pay for the NIH's business.

The only ethical situation is when there is perfect balance between the expectations of the respective sources of financial support and the PIs actual distribution of work.

I do wonder how many NIH PIs that have nine month salary support actually achieve the appropriate balance of brain effort devoted to their respective tasks. I bet not many.

__
*I would like to hear some specific language from people's job descriptions that specify that their hard money effort is supposed to be devoted X amount to research, btw. I know these do exist. How commonly?

**Naturally these sorts of proposals are often coupled with 12 mo of full time effort from trainees or techs which supports the notion that the project is not limited to the summer months.

77 responses so far

A hint about staff scientist awards from Varmus

Mar 06 2015 Published by under Fixing the NIH, NIH, NIH Careerism

One ray of possible sunshine. Jocelyn Kaiser noted something that I had originally missed in Varmus' letter to the NCI community.


In his resignation letter, Varmus introduces a new award for staff scientists. He says the idea is to offer salary support and independence to scientists who hold less high-profile but essential jobs, such as managing a core facility or doing informatics within a principal investigator’s lab.

The part I missed was in Varmus' bullet points on accomplishments.

In efforts to provide greater stability for investigators in these difficult times, we have established a new seven year Outstanding Investigator Award; are discussing new awards to accelerate graduate and post-doctoral training; and are planning to provide individual support for so-called "staff scientists" at extramural institutions.

Excellent. I have repeatedly suggested that some sort of K mechanism could be used to pay the salary of the people who want to stay in science somewhere below the PI level. They should be limited to salary support, perhaps with a little bit of travel money, full benefits and be competitively renewable.

This would mean that the person would have to be linked up to a NIH funded research mechanism but it wouldn't have to be the same one all of the time. This approach would allow this person some flexibility if the primary PI runs out of money or becomes unduly exploitative. It puts the staff scientist a little bit more in charge of their own destiny, which has to be a plus for most people. And it ensures NIH programmatic, and periodically competitive, review of productivity to continue, thereby keeping the staff scientist from slacking off entirely.

The only real problems here are the overhead rates (if adopted in any large numbers) of K mechanisms and how to deal with a staff scientist who jumps from a primary research grant funded by one IC to another.

I am eager to see what NCI has planned for their version of a staff scientist support initiative.

23 responses so far

Poll: Fixing the NIH Action Items

Mar 06 2015 Published by under Fixing the NIH, NIH

Long time commenter Neuro-conservative offered up some specific action items for fixing the NIH. Which of these is of highest priority to you?

68 responses so far

Thought of the Day

Mar 05 2015 Published by under Anger, NIH Careerism, Tribe of Science

This is about the cognitive dissonance involved with realizing your part in a collective action.

It is about the result of a slowly evolving cultural hegemony.

It is about the tragedy of the commons and the emergent properties of systems that depend on the decisions of individual self-interested actors.

It is not all about Snidely Whiplash figures intentionally committing egregious, knowing acts of thievery and sabotage.

We need to be very clear about this or it devolves into useless fighting about personal responsibility for things that are not the direct result of highly specific individual acts.

20 responses so far

« Newer posts Older posts »