Archive for the 'NIH' category

Your Grant in Review: Credible

Jan 30 2015 Published by under NIH, NIH Careerism, NIH funding, Peer Review

I am motivated to once again point something out.

In ALL of my advice to submit grant applications to the NIH frequently and on a diversity of topic angles, there is one fundamental assumption.

That you always, always, always send in a credible application.

That is all.

17 responses so far

Data faker happily employed by the US Patent Office

via retraction watch we learn:

A Bijan Ahvazi has been working at the USPTO since at least 2008, and today a source confirmed that it was the same person who was the subject of last October’s ORI report. Ahvazi was found to have faked five different images in three different papers, two of which have been retracted.

The Notice of ORI finding appeared in October of 2014.

Based on the report of an investigation conducted by the National Institutes of Health (NIH) and additional analysis by ORI in its oversight review, ORI found that Dr. Bijan Ahvazi, former Director of the Laboratory of X-ray Crystallography, National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS), NIH, engaged in research misconduct in research supported by the Intramural Program at NIAMS, NIH.

The Notice shows that the offenses for which Ahvazi was convicted date to 2004 and 2006. One doesn't have to assume that much to figure out that he was busted and then had to look for a new job somewhere between 2006 and 2008. It took until 2014 for his fraud to come to light via the official ORI mechanisms. Presumably, although we don't know for sure, the investigation was confidential up until it reached its formal conclusions which may have permitted him to avoid telling the US Patent and Trade Office about his little whoopsie? I dunno, do you think the USPTO would hire a data fraud as a patent examiner if they knew about it? One thinks not.

__
p.s. apparently a co-author of this data faker died under bizarre circumstances in 2003.

11 responses so far

These ILAF types just can't help sounding selfishly elitist, can they?

Good Gravy.

One David Korn of the Massachusetts General Hospital and Harvard Medical School has written a letter to Nature defending the indirect cost (IDC; "overhead") rates associated with NIH grants. It was submitted in response to a prior piece in Nature on IDC which was, to my eye, actually fairly good and tended to support the notion that IDC rates are not exorbitant.

But overall, the data support administrators’ assertions that their actual recovery of indirect costs often falls well below their negotiated rates. Overall, the average negotiated rate is 53%, and the average reimbursed rate is 34%.

The original article also pointed out why the larger private Universities have been heard from loudly, while the frequent punching-bag smaller research institutes with larger IDC rates are silent.

Although non-profit institutes command high rates, together they got just $611 million of the NIH’s money for indirect costs. The higher-learning institutes for which Nature obtained data received $3.9 billion, with more than $1 billion of that going to just nine institutions, including Johns Hopkins University in Baltimore, Maryland, and Stanford (see ‘Top 10 earners’).

Clearly Dr. Korn felt that this piece needed correction:

Aspects of your report on US federal funding of direct research costs and the indirect costs of facilities and administration are misleading (Nature 515, 326–329; 2014).

Contrary to your claim, no one is benefiting from federal largesse. Rather, the US government is partially reimbursing research universities for audit-verified indirect costs that they have already incurred.

Ok, ok. Fair enough. At the very least it is fine to underline this point if it doesn't come across in the original Nature article to every reader.

The biomedical sciences depend on powerful technologies that require special housing, considerable energy consumption, and maintenance. Administration is being bloated by federal regulations, many of which dictate how scientists conduct and disseminate their research. It is therefore all the more remarkable that the share of extramural research spending on indirect costs by the US National Institutes of Health (NIH) has been stable at around 30% for several decades.

Pretty good point.

But then Korn goes on to step right in a pile.

Negotiated and actual recovery rates for indirect costs vary across the academic community because federal research funding is merit-based, not a welfare programme.

You will recognize this theme from a prior complaint from Boston-area institutions.

“There’s a battle between merit and egalitarianism,” said Dr. David Page, director of the Whitehead Institute, a prestigious research institution in Cambridge affiliated with MIT.

Tone deaf, guys. Totally tone deaf. Absolutely counter-productive to the effort to get a majority of Congress Critters on board with support for the NIH mission. Hint: Your various Massachusetts Critters get to vote once, just like the Critters from North and South Dakota, Alabama and everywhere else that doesn't have a huge NIH-funded research enterprise.

And why Korn chooses to use a comment about IDC rates to advance this agenda is baffling. The takeaway message is that he thinks that higher IDC rates are awarded because His Awesome University deserves it due to the merit of their research. This totally undercuts the point he is trying to make, which is presumably "institutions may be private or public, urban or rural, with different structures, sizes, missions and financial anatomies.".

I just don't understand people who are this clueless and selfish when it comes to basic politics.

23 responses so far

Excellent observation on only funding the absolutely most amazing science

Jan 21 2015 Published by under Fixing the NIH, NIH

Over at Rock Talk, by a Joel MacAuslan:

It isn’t about whether to fund only the “best” science: I really DON’T want only Isaac Newtons and Louis Pasteurs to be competitive, and to be able to spend their careers on this research. That’s because I don’t want to wait 200 years for all that “great” science to trickle through society. Fund lots and lots of very good science, and cure heart disease in 40 years, instead!

5 responses so far

NIGMS will now consider PIs' "substantial unrestricted research support"

According to the policy on this webpage, the NIGMS will now restrict award of its grants when the applicant PI has substantial other research support. It is effective as of new grants submitted on or after 2 Jan, 2015.

The clear statement of purpose:

Investigators with substantial, long-term, unrestricted research support may generally hold no more than one NIGMS research grant.

The detail:

For the purposes of these guidelines, investigators with substantial, long-term, unrestricted support (“unrestricted investigators”) would have at least $400,000 in unrestricted support (direct costs excluding the principal investigator’s salary and direct support of widely shared institutional resources, such as NMR facilities) extending at least 2 years from the time of funding the NIGMS grant. As in all cases, if NIGMS funding of a grant to an investigator with substantial, long-term, unrestricted support would result in total direct costs from all sources exceeding $750,000, National Advisory General Medical Sciences Council approval would be required

This $400,000 limit, extending for two years would appear to mean $200,000 per year in direct costs? So basically the equivalent of a single additional R01-worth of direct cost funding?

I guess they are serious about the notion that two grants is fine but three-R01-level funding means you are a greedy commons-spoiling so-and-so.

51 responses so far

More on NIGMS's call for "shared responsibility"

Jan 07 2015 Published by under Fixing the NIH, NIH, NIH Careerism, NIH funding

The post from NIGMS Director Lorsch on "shared responsibility" (my blog post) has been racking up the comments, which you should go read.

As a spoiler, it is mostly a lot of the usual, i.e., Do it to Julia!

But two of the comments are fantastic. This one from anonymous really nails it down to the floor.

More efficient? My NOA for my R01 came in a few weeks ago for this year, and as usual, it has been cut. I will get ~$181,000 this year. Let’s break down the costs of running a typical (my) lab to illustrate that which is not being considered. I have a fairly normal sized animal colony for my field, because in immunology, nothing gets published well without knockouts and such. That’s $75,000 a year in cage per diem costs. Let’s cover 20% of my salary (with fringe, at 28.5%), one student, and one postdoc (2.20 FTE total). Total salary costs are then $119,800. See, I haven’t done a single experiment and my R01 is gone. How MORE efficient could I possibly be? Even if we cut the animals in half, I have only about $20,000 for the entire year for my reagents. Oh no, you need a single ELISA kit? That’s $800. That doesn’t include plates? Hell, that’s another $300. You need magnetic beads to sort cells, that’s $800 for ONE vial of beads. Wait, that doesn’t include the separation tubes? Another $700 for a pack. You need FACS sort time? That’s $100 an hour. Oh no, it takes 4 hours to sort cells for a single experiment? Another $400. It’s easy to spend $1500 on a single experiment given the extreme costs of technology and reagents, especially when using mice. Then, after 4 years of work, you submit your study (packed into a single manuscript) for publication and the reviewers complain that you didn’t ALSO use these 4 other knockout mice, and that the study just isn’t complete enough for their beloved journal. And you (the NIH) want me to be MORE efficient? I can’t do much of anything as it is.

Anyone running an academic research laboratory should laugh (or vomit) at the mere suggestion that most are not already stretching every penny to its breaking point and beyond.

This is what is so phenomenally out of touch with Lorsch's concentration on the number of grants a PI holds. Most of us play in the full-modular space. Even for people with multiple grants that have one that managed to get funded with a substantial upgrade from full-mod, they are going to have other ones at the modular limit. And even the above-modular grants often get cut extra compared with the reductions that are put on the modular-limit awards.

The full-modular has not been adjusted with inflation. And the purchasing power is substantially eroded compared with a mere 15 years ago when they started the new budgeting approach.


[this graph depicts the erosion of purchasing power of the $250K/yr full-modular award in red and the amount necessary to maintain purchasing power in black. Inflation adjustment used was the BRDPI one]

Commenter Pinko Punko also has a great observation for Director Lorsch.

The greatest and most massive inefficiency in the system is the high probability of a funding gap for NIGMS (and all other Institute) PIs. Given that gaps in funding almost always necessitate laying off staff, and prevent long-term retention of expertise, the great inefficiency here is that expertise cannot possibly be “on demand”. I know that you are also aware that given inflation, the NIH budget never actually doubled. There has likely been a PI bubble, but it is massively deflating with a huge cost.

The lowest quantum for funding units in labs is 1. Paylines are so low, it seems the only way to attempt to prevent a gap in funding is to have an overlap at some point, because going to zero is a massive hit when research needs to grind to a halt. It is difficult to imagine that there is a large number of excessively funded labs.

While I try to put a positive spin on the Datahound analysis showing the probability of a PI becoming re-funded after losing her last NIH award, the fact is that 60% of PIs do not return to funding. A prior DataHound post showed that something between 14-30% of PIs are approximately continuously-funded (extrapolating generously here from only 8 years of data). Within these two charts there is a HUGE story of the inefficiency of trying to maintain that funding for the people who will, in the career-long run, fall into that "continuously funded" category.

This brings me to the Discussion point of the day. Lorsch's blog post is obsessed with efficiency. which he asserts comes with modestly sized research operations, indexed approximately by the number of grant awards. Three R01s being his stated threshold for excessive grants even though he cites data showing that $700K per year in direct costs is the most productive* amount of funding- i.e., three grants at a minimum.

I have a tale for you, Dear Reader. The greatly abridged version, anyway.

Once upon a time the Program Staff of ICx decided that they were interested in studies on Topic Y and so they funded some grants. Two were R01s obtained without revision. They sailed on for their full duration of funding. To my eye, there was not one single paper that resulted that was specific to the goals of Topic Y and damn little published at all. Interestingly there were other projects also funded on Topic Y. One of them required a total of 5 grant applications and was awarded a starter grant, followed by R01 level funding. This latter project actually produced papers directly relevant to Topic Y.

Which was efficient, Director Lorsch?
How could this process have been made more efficient?

Could we predict PI #3 was the one that was going to come through with the goods? Maybe we should have loaded her up with the cash and screw the other two? Could we really argue that funding all three on a shoestring was more efficient? What if the reason that the first two failed is that they just didn't have enough cash at the start to make a good effort on what was, obviously, a far from easy problem to attack.

Would it be efficient to take this scenario and give PI #3 a bunch of "people-not-projects" largesse at this point in time because she's proved able to move the scientific ball on this? Do we look at the overall picture and say "in for a penny, in for a pound"? Or do we fail to learn a damn thing and let the productive PI keep fighting against the funding cycles, the triage line and what not to keep the program going under our current approaches?

It may sound like I am leaning in one direction on this but really, I'm not. I don't know what the answer is. The distribution of success/failure across these three PIs could have been entirely different. As it happens, all three are pretty dang decent scientists. The degree to which they looked like they could kick butt on Topic Y at the point of funding their respective projects definitely didn't support the supremacy of PI#3 in the end analysis. But noobs can fail too. Sometimes spectacularly. Sometimes, as may have been the case in this little tale, people can fail because they simply haven't grown their lab operations large enough, fast enough to sustain a real program, particularly when one of the projects is difficult.

I assume, as usual, that this narrow little anecdote is worth relating because these are typical scenarios. Maybe not hugely common but not all that rare either. Common enough that a Director of an IC should be well aware.

When you have an unhealthy interest in the grant game, as do I, you notice this stuff. You can see it play out in RePORTER and PubMed. You can see it play out as you try to review competing-continuation proposals on study section. You see it play out in your sub-fields of interest and with your closer colleagues.

It makes you shake your head in dismay when someone makes assertions that they know how to make the NIH-funded research enterprise more efficient.

UPDATE: I realized that I should really should say that the third project required at least five applications since I'm going by the amended status of the *funded* awards. It is unknown if there were unfunded apps submitted. It is also unknown if either of the first two PIs tried to renew the awards and couldn't get a fundable score. I think I should also note that the third project was awarded funding in a context that triggers on at least three of the major "this is the real problem" complaints being bandied in Lorsch's comments section. The project that produced nothing at all, relevant or not, was awarded in a context that I think would align with these complainants "good" prescription. FWIW.

__
*there are huge problems even with this assessment of productivity but we'll let that slide for now.

66 responses so far

Your Grant In Review: Thought of the Day

Jan 07 2015 Published by under Grant Review, NIH Careerism

I've said it repeatedly on this blog and it is true, true, true people.

In NIH grant review, the worm turns very rapidly.

The pool of individual PIs who are appropriate to apply for, and review, NIH grants in a narrow subfield is a lot smaller than most people seem to think. Or maybe this is just my field.

My guiding belief is that the reviewer of a given grant is going to have one of her own grants reviewed by the PI of the proposal she just reviewed  in very short order. Or maybe it takes a half a decade, even more. But it will happen.

And PIs do not take kindly to jackholish reviews of their proposals.

As we all know, in this day and age it takes very little in the way of reviewer behavior to totally torpedo a grant's chances. You don't even have to be obvious about it*.

This is why I try as hard as I possibly can to ground my grant reviewing in concrete reasons for criticism.

Because I want the reviewers of my proposals to do the same. And it is the right thing to do.

We have a system of grant review that is at all times precariously balanced on a knife's edge that could slide off into Mutually Assured Destruction cycles of retaliation** at any time. And I am sure it happens in some study sections and amongst some reviewers.

Mutual Professional Respect is better. It is supported one review at a time by engaging our firmest professionalism to override the biases that we cannot help but have.

 

illustration from here.

__

*This is very likely the second hardest decision I have to make about registering a Conflict of Interest in reviewing grants. I have reviewed a lot of grants of PIs who have been on the study section panels reviewing my grants. I am pretty confident this is the case for just about anyone who has served a full term appointed on a study section and probably anyone who has reviewed with full loads in over about 3 panels as ad hoc. This in and of itself cannot be a reason to recuse yourself or they would never get anything reviewed. And as my Readers know, I am very firm in my belief that it is a fool's errand to try to game out which reviewers were on your proposals and which ones were...critical.

**And, gods above, pre-emptive counter-striking.

 

2 responses so far

The new normal

Jan 05 2015 Published by under Careerism, NIH, NIH Careerism

An interesting comment from Anonymous :

A friend of mine is getting his PhD this year. His mentor was awarded tenure last year. She had an RO1 and an R21 and enough pubs, reputation, etc., so that she was a "slam dunk," according to some of her colleagues in the field, at least. When my friend joined her lab, she had 7 students -- a mix of masters and doctoral students. When he graduates this year, there will be only 1 doctoral student left in the lab. His mentor hasn't succeeded in getting more grants, so I guess she hasn't hired anyone else because of that?

What I'm wondering is this: given the current climate, is this normal? Or is she really in trouble? Has she done it wrong? I always assumed that by the time you got tenure, your lab would be humming along.

 

I think this is pretty normal these days:

We are in an era of boom and bust instability when it comes to NIH funding. It is the very rare flower indeed, in my estimation, that will be completely free of the cycle in the coming decade or two.

As always, my view is quite possibly colored by my experiences. But I have seen the boom and bust cycle play out across a large number of labs. Some of my close acquaintance. Some labs that I know only through the grant review process. Some labs that happen to make it to the scuttlebutt news channel for some reason or other.

It usually plays out like this. "Yeah, Dr. So-and-so is really well funded.....what? What do you mean they are on the ropes? [checks RePORTER]..how in the hell did THAT happen". ....Two years later "Oh phew, glad to see So-and-so got another grant. ....what? TWO grants? and an R21? how in the hell did THAT happen?"...

Repeat.

Normal, but the PI is still in trouble. How could she not be? Has she done it wrong? Probably not. Most likely she's just experiencing the variance of grant fortune as it currently exists.

This part is painful though: " I always assumed that by the time you got tenure, your lab would be humming along."

Yeah, so did we. Because when people of my approximate scientific generation were coming up through postdoc we saw the generation of Assistant Professors just above us struggling. But then as we were finishing postdocs and starting our own Assistant Professor stints, we saw the next-older generation transition to a cruise mode. A time where they got their renewals without too much hassle*. They got their second or even third grants and maybe a few got an R37 extension. This made the struggles we went through as newbie applicants to the NIH a bit easier to stomach. Hazing ritual. Sure, we can stand this, and then we'll REALLY get stuff cranking in the lab later.

Instead the budget went stagnant just as we were reaching that stage of our careers. And then the powers that be went and invented the ESI boost to give affirmative action to those juuuuuust behind us.

So....we ride the roller coaster. And as things keep going in the wrong direction with NIH funding, more and more of us from all scientific cohorts/generations will experience the thrill.

__

*Yes, I realize it is all relative. I certainly had it easier than the kids these days. And the next-older generation did plenty of complaining about how hard they had it compared to the really established folks.

70 responses so far

NIGMS blog post on "shared responsibility"

Jan 05 2015 Published by under NIH, NIH Careerism, NIH funding

This is a fascinating read.

Jon Lorsch, current NIGMS Director, spreads around so much total nonsense that I just can't even deal.

And journals, professional societies and private funding organizations should examine the roles they can play in helping to rewire the unproductive incentive systems that encourage researchers to focus on getting more funding than they actually need.

Riiiight. We PIs out here in extramural land are focused on getting more grant money than we feel that we need. Because what? We enjoy grant writing? Is this guy nuts? When I feel like I have enough grant support to keep what is really a very modest operation afloat I quit writing grants! The problem is that Director Lorsch is really, really out of touch with what a PI in today's climate actually needs. "Unproductive incentive systems"? Dude, when NIGMS stops giving grants to anyone who publishes in Science, Nature or Cell, and starts beancounting Supplementary Data for reduced publication output, and punishes PIs for failing to publish data that is "scooped" or "not hot enough" etc, then maybe I will take you seriously. Jeepers. LOOK IN THE MIRROR, NIH!!!!!!!

But to achieve this increase, we must all be willing to share the responsibility and focus on efficiency as much as we have always focused on efficacy. In the current zero-sum funding environment, the tradeoffs are stark: If one investigator gets a third R01, it means that another productive scientist loses his only grant or a promising new investigator can’t get her lab off the ground. Which outcome should we choose?

better to have everyone funded at $50K and sitting around doing nothing, right?

Although certain kinds of research projects—particularly those with an applied outcome, such as clinical trials—can require large teams, a 2010 analysis by NIGMS and a number of subsequent studies of other funding systems (Fortin and Currie, 2013; Gallo et al., 2014) have shown that, on average, large budgets do not give us the best returns on our investments in basic science.

The "2010 analysis" has been discussed here, I recall. It's flawed. It fails to recognize the cost of a Glamour Pub- love or hate, we have to admit that it takes a rich lab to play in that arena. One pub to the accountants has like 6-10 pubs worth of time/effort and probably data (buried in the Supplemental Materials). It fails to recognize there are going to be some scientific advances that simply cannot be accomplished for less. It fails to recognize the "efficiencies" and lack thereof associated with continued funding versus the scary roller coaster of a funding gap.

and Lorsch does a little neat ju-jitsu with this post. Berg's analysis concluded that $700K was the peak of productivity. That is three concurrent full modular R01s. Even with a traditional budget award the PI has to have two of them awarded at $350K per year (and we know it really means more than that because of cuts) to hit this level. So the finger pointing at the investigator who "gets a third R01" doesn't even square with his own citation on "efficiency", now does it?

Furthermore, the larger a lab gets, the more time the principal investigator must devote to writing grants and performing administrative tasks, further reducing the time available for actually doing science.

Good GRAVY man! Do you have any idea what the hell time it is on PI street? A one-grant lab PI is constantly on the edge of disaster and the PI is always writing furiously to increase that funding level to where she can finally breathe for six months. Protocols and registrations most often are one per lab, so that "administrative tasks" claim is also nonsense. It is the smaller PI who spends more effort per-grant on keeping the approvals in place.

These and other lines of evidence indicate that funding smaller, more efficient research groups will increase the net impact of fundamental biomedical research: valuable scientific output per taxpayer dollar invested.

[citation needed]

It may sound truthy but it is not at all obvious that this is the case. "More efficient" is tautological here but the smaller=efficient is not proven. At all. Especially when you are talking about the longer term- 30 years of a career. There is also the strong whiff of Magic Unicorn Leprechaun money about this comment.

My main motivation for writing this post is to ask the biomedical research community to think carefully about these issues.

You know what I would really like to ask? For the NIH to actually think carefully about these issues. Starting with Director Lorsch.

But reshaping the system will require everyone involved to share the responsibility.

Somehow I doubt he means this. Is there any evidence that NIGMS actually denies the super awesome PIs their extra R01s? Is there any evidence they do anything more than handwring about HHMI investigators with NIGMS grants? Is there any evidence they mean to face down the powerful first and make them equal to the rest of the drones before they take it out of the hide of the less-powerful? HA!

h/t: PhysioProf

23 responses so far

The proof is in the budgeting

Dec 17 2014 Published by under NIH, NIH Budgets and Economics

When we last discussed Representative Andy Harris it was in the wake of an editorial he published in the NYT. It consisted of a call to put hard targets on the NIH for reducing the average age of the first R01, standard Golden Fleece style carping about frivolous research projects and a $700M "tap" of the NIH budget. I speculated that this last was the real target because the "tap" is money appropriated to the NIH that then goes to "program evaluation" and the AHRQ. There is the possibility that this is a covert attack on the ACA ("Obamacare").

The recent appropriation to the NIH passed by the Congress is interesting because it addresses these three issues. According to Jocelyn Kaiser at ScienceInsider:

The report also directs NIH to pay more attention to the age at which new NIH investigators receive their first research grant... but lacks that specific target.

So toothless verbiage, but no more.


Lawmakers also address a perennial concern: that the amount NIH spends on specific diseases doesn’t take into account the burden that disease creates or death rates. The report “urges NIH to ensure research dollars are invested in areas in which American lives may be improved.” It also tells NIH “to prioritize Federal funds for medical research over outreach and education,”

"urges". Again, this is totally impotent. Two strikes on Rep Harris.

One recent concern about NIH’s budget—that each year some money is skimmed off for other Department of Health and Human Services (HHS) agencies—is remedied in the bill. It says that the $700 million that NIH is set to contribute to the “tap” this year will come back as $715 million for the agency.

Well that seems like Rep Harris got a win on the tap, no? And his goal was what again?

For one thing, we need to eliminate a budget gimmick, known as the “tap,” that allows the Department of Health and Human Services to shift money out of the N.I.H. budget into other department efforts. The N.I.H. lost $700 million to the “tap” in 2013 alone. Instead, the money should be placed under the control of the N.I.H. director, with an explicit instruction that it go to young investigators as a supplement to money already being spent. If we don’t force the N.I.H. to spend it on young investigators, history has shown that the agency won’t.

"lost". "supplement to money already being spent". This creates the strong impression that Rep Harris was trying to increase the NIH budget by $700M. And yet. The overall NIH appropriation only increased by $150M.

So in point of fact Rep Harris took three strikes.

Or so it appears.

Of course, if his agenda was to go after those agencies that received their support from the tap, perhaps he didn't strike out after all. We'll have to see if those agencies got all their money in this budget and, more importantly, if they remain this way in subsequent years. It is not impossible that breaking the previous recipients of the "tap" down into individual line items in the budget will allow them to be eliminated one by one.

One thing is for sure, Rep Harris didn't do anything concrete to help out the young investigator issue at the NIH in this budget appropriation.

UPDATE: Actually I screwed this up. If there is no net decrease in the budget and the NIH no longer loses $700M to the tap obligations, I guess this is a net gain. My bad.

15 responses so far

« Newer posts Older posts »