Archive for the 'NIH Careerism' category

Entitled to a Grant: What is fair?

May 02 2016 Published by under Fixing the NIH, NIH, NIH Careerism, NIH funding

I am genuinely curious as to how you people see this. Is there any particular difference between people arguing that that acquisition of the first major grant award should be protected versus multiple award and the people arguing that acquisition of the first and third concurrent awards should be on an equal footing?

If we agree that NIH (or NSF or CIHR or whatever) grants are competitively awarded, it follows that nobody is actually entitled to a grant. And as far as I am aware, all major funding agencies operate in a way that states and demonstrates the truth of this statement.

Specifically in the NIH system, it is possible for the NIH officials to choose not to fund a grant proposal that gets the best possible score and glowing reviews during peer review. Heck, this could happen repeatedly for approximately the same project and the NIH could still choose not to fund it.

Nobody is entitled to a grant from the NIH. Nobody.

It is also the case that the NIH works very hard to ensure a certain amount of equal representation in their awarded grants. By geography (State and Congressional district), by PI characteristics of sex and prior NIH PIness, by topic domain (see the 28 ICs) or subdomain (see Division, Branches of the ICs. also RFAs), etc.

Does a lean to prioritize the award of a grant to those with no other major NIH support (and we're not just talking the newcomers- plenty of well-experienced folks are getting special treatment because they have run out of other NIH grant support) have a justification?

Does the following graph, posted by Sally Rockey, the previous head of Extramural Research at the NIH make a difference?

This shows the percentage of all PIs in the NIH system for Fiscal Years 1986, 1998, 2004 (end of doubling) and 2009 who serve as PI on 1-8 Research Project Grants. In the latest data, 72.3% had only one R01 and 93% had 1 or 2 concurrent RPGs. There were 5.4% of the PIs that held 3 grants and 1.2% that held 4 grants. I just don't see where shifting the 7% of 3+ concurrent awards into the 1-2 grant population is going to budge the needle on the perceived grant chances of those without any major NIH award. Yes, obviously there will be some folks funded who would otherwise not have been. Obviously. But if this is put through in a systematic way*, the first thing the current 3+ grant holders are going to do is stop putting in modular grants and max out their allowable 2 at $499,999 direct costs. Maybe some will even get Program permission to breach the $500,000 DC / y threshold. So there won't be a direct shift of 7% of grants back into the 1-2 grant PI population.

There has been a small trend for PIs holding more grants concurrently from 1986 to the late naughties but this is undoubtedly down to the decreasing purchasing power of the modular-budget grant.

BRDPI.
I"ve taken their table of yearly adjustments and used those to calculate the increase necessary to keep pace with inflation (black bars) and the decrement in purchasing power (red bars). The starting point was the 2001 fiscal year (and the BRDPI spreadsheet is older so the 2011 BRDPI adjustment is predicted, rather than actual). As you can see, a full modular $250,000 year in 2011 has 69% of the purchasing power of that same award in 2001.

Without that factor, I'd say the relative proportions of PIs holding 1, 2, 3 etc grants would be even more similar across time than it already is.

So I come back to my original question. What is fair? What policies should the NIH or any broad governmental funding body adopt when it comes to distributing the grant wealth across laboratories? On what basis should they do this?

Fairness? Diversity of grant effort? PR/optics?

__
*and let us face it, it is hugely unlikely that the entire NIH will put through a 2-grant cap without any exceptions. Even with considerable force and authority behind it, any such initiative is likely to be only partially successful in preventing 3+ grant PIs.

DISCLAIMER: As always, I am an interested party in these discussions. My lab's grant fortunes are affected by broad sweeping policies that the NIH might choose to adopt or fail to adopt. You should always read my comments about the NIH grant game with this in mind.

100 responses so far

Open Grantsmanship

Apr 27 2016 Published by under Careerism, NIH, NIH Careerism

The Ramirez Group is practicing open grantsmanship by posting "R01 Style" documents on a website. This is certainly a courageous move and one that is unusual for scientists. It is not so long ago that mid-to-senior level Principal Investigator types were absolutely dismayed to learn that CRISP, the forerunner to RePORTER, would hand over their funded grants' abstract to anyone who wished to see it.

There are a number of interesting things here to consider. On the face of it, this responds to a plea that I've heard now and again for real actual sample grant materials. Those who are less well-surrounded by grant-writing types can obviously benefit from seeing how the rather dry instructions from NIH translate into actual working documents. Good stuff.

As we move through certain changes put in place by the NIH, even the well experienced folks can benefit from seeing how one person chooses to deal with the Authentication of Resources requirement or some such. Budgeting may be helpful for others. Ditto the Vertebrate Animals section.

There is the chance that this will work as Open Pre-Submission Peer Review for the Ramirez group as well. For example, I might observe that referring to Santa Cruz as the authoritative proof of authentic antibodies may not have the desired effect in all reviewers. This might then allow them to take a different approach to this section of the grant, avoiding the dangers of a reviewer that "heard SC antibodies are crap".

But there are also drawbacks to this type of Open Science. In this case I might note that posting a Vertebrate Animals statement (or certain types of research protocol description) is just begging the AR wackaloons to make your life hell.

But there is another issue here that I think the Readers of this blog might want to dig into.

Priority claiming.

As I am wont to observe, the chances are high in the empirical sciences that if you have a good idea, someone else has had it as well. And if the ideas are good enough to shape into a grant proposal, someone else might think these thoughts too. And if the resulting application is a plan that will be competitive, well, it will have been shaped into a certain format space by the acquired wisdom that is poured into a grant proposal. So again, you are likely to have company.

Finally, we all know that the current NIH game means that each PI is submitting a LOT of proposals for research to the NIH.

All of this means that it is likely that if you have proposed a 5 year plan of research to the NIH someone else has already, or will soon, propose something that is a lot like it.

This is known.

It is also known that your chances of bringing your ideas to fruition (published papers) are a lot higher if you have grant support than if you do not. The other way to say this is that if you do not happen to get funded for this grant application, the chances that someone else will publish papers related to your shared ideas is higher.

In the broader sense this means that if you do not get the grant, the record will be less likely to credit you for having those ideas and brilliant insights that were key to the proposal.

So what to do? Well, you could always write Medical Hypotheses and review papers, sure. But these can be imprecise. They describe general hypotheses and predictions but....that's about all.

It would be of more credit to you to lay out the way that you would actually test those hypotheses, is it not? In all of the brilliant experimental design elegance, key controls and fancy scientific approaches that are respected after the fact as amazing work. Maybe even with a little bit of preliminary evidence that you are on the right track, even if that evidence is far too limited to ever be published.

Enter the Open Grantsmanship ploy.

It is genius.

For two reasons.

First, of course, is pure priority claiming. If someone else gets "your" grant and publishes papers, you get to go around whining that you had the idea first. Sure, many people do this but you will have evidence.

Second, there is the subtle attempt to poison the waters for those other competitors' applications. If you can get enough people in your subfield reading your Open Grant proposals then just maaaaaybe someone on a grant panel will remember this. And when a competing proposal is under review just maaaaaaybe they will say "hey, didn't Ramirez Group propose this? maybe it isn't so unique.". Or maybe they will be predisposed to see that your approach is better and downgrade the proposal that is actually under review* accordingly. Perhaps your thin skin of preliminary data will be helpful in making that other proposal look bad. Etc.

__
*oh, it happens. I have had review comments on my proposals that seemed weird until I became aware of other grant proposals that I know for certain sure couldn't have been in the same round of review. It becomes clear in some cases that "why didn't you do things this way" comments are because that other proposal did indeed do things that way.

21 responses so far

On removing grant deadlines

Apr 20 2016 Published by under NIH, NIH Careerism, NIH funding

Eric Hand reported in Science that one NSF pilot program found that allowing for any-time submission reduced applications numbers.

Assistant Director for Geosciences Roger Wakimoto revealed the preliminary results from a pilot program that got rid of grant proposal deadlines in favor of an anytime submission. The numbers were staggering. Across four grant programs, proposals dropped by 59% after deadlines were eliminated.

I have been bombarded with links to this article/finding and queries as to what I think.

Pretty much nothing.

I do know that NIH has been increasingly liberal with allowing past-deadline submissions from PIs who have served on study section. So there is probably a data source to draw upon inside CSR if they care to examine it.

I do not know if this would do anything similar if applied to the NIH.

The NSF pilot was for

geobiology and low-temperature geochemistry, geomorphology and land-use dynamics, hydrological sciences, and sedimentary geology and paleobiology.

According to the article these are fields in which

"many scientists do field work, having no deadline makes it easier for collaborators to schedule time when they can work on a proposal".

This field work bit is not generally true of the NIH extramural community. I think it obvious that continual-submission helps to schedule time but I would note that it also eliminates a stick for the more proactive members of a collaboration to beat the slaggards into line. As a guy who hits his deadlines for grant submission, it's probably in my interest to further lower the encouragements the lower-energy folks require.

According to a geologist familiar with reviewing these grants

The switch is “going to filter for the most highly motivated people, and the ideas for which you feel the most passion,” he predicts. When he sits on merit review panels, he finds that he can usually reject half of the proposals right away as being hasty or ill-considered. “My hope is that this has taken off the bottom 50%,” he says. “Those are the ones you read and say, ‘Did they have their heart in this?’”

Personally I see very few NIH grant proposals that appear to me to be "hasty or ill-considered" or cause me to doubt the PI has her heart in it. And you know how I feel about the proposition that the RealProblem with NIH grant success hinges on whether or not PIs refine and hone and polish their applications into some shining gem of a document. Applications are down so therefore success rates go up is the only thing we need to take away from this pilot, if you ask me. Any method by which you could decrease NIH applications would likewise seem to improve success rates.

Would it work for NIH types? I tend to doubt it. That program at NSF started with only two submission rounds per year. NIH has three rounds for funding per year, but this results from a multitude of deadlines including new R01, new R21/R03, two more for the revised apps, special ones for AIDS-related, RFAs and assorted other mechanisms. As I mentioned above, if you review for the NIH (including Advisory Council service) you get an extra extension to submit for a given decision round.

The pressure for most of us to hit any specific NIH deadline during the year is, I would argue, much lower at baseline. So if the theory is that NSF types were pressured to submit junky applications because their next opportunity was so far away....this doesn't apply to NIH folks.

6 responses so far

Thought of the Day

Apr 09 2016 Published by under Fixing the NIH, NIH, NIH Careerism

"Uppity" is a fascinating concept when it comes to NIH Grant award. 

We know the sentiment applies to newer and younger investigators. I've heard countless review and Program Officer comments which amount to "let's not get too big for your britches, young un!" in my day. 

I wonder how much of the Ginther effect is related to sentiments similar to "damn uppity [insert subvocalization]"? 

17 responses so far

MIRA Moaners

Mar 31 2016 Published by under Fixing the NIH, NIH, NIH Careerism

Jocelyn Kaiser reports that some people who applied for MIRA person-not-project support from NIGMS are now complaining. 

I have no* comment.

____
*printable

25 responses so far

Reviewer mindset 

Mar 22 2016 Published by under NIH Careerism, Peer Review

I was just observing that I'd far rather my grants were reviewed by someone who had just received a new grant (or fundable score) than someone who had been denied a few times recently. 

It strikes me that this may not be universal logic.

Thoughts? 

 Is the disgruntled-applicant reviewer going to be sympathetic? Or will he do unto you as he has been done to?

Will the recently-awarded reviewer be in a generous mood? Or will she pull up the ladder? 

25 responses so far

It's the pig-dog field scientists that are the problem

Mar 10 2016 Published by under Anger, Fixing the NIH, NIH, NIH Careerism

But clearly the laboratory based male scientists would never harass their female subordinates.

Field science is bad.

Lab science is good.

This is what the head of the Office of Extramural Research at the NIH seems to think.

8 responses so far

Never Ever Trust a Dec 1 NIH Grant Start Date: The Sickening

Mar 09 2016 Published by under NIH, NIH Budgets and Economics, NIH Careerism

As I noted in a prior post, the Cycle I NIH Grant awards (submitted in Feb-Mar, Reviewed Jun-July, Council Aug) with a first possible funding date of December 1 hardly ever are funded on time. This is due to Congress never passing a budget for the Fiscal Year that starts in October on time. The Congress sometimes goes into a stop-gap measure, like Continuing Resolution, which theoretically permits Federal agencies to spend along the parameters of the past year's budget. I find that NIH ICs of my greatest interest are highly conservative and never* fund new grants in December. The ICs that I follow almost inevitably wait until late Jan when Congress returns from their winter recess to see if they will do something more permanent.

New Cycle I grants then start trickling out in Feb, again, typically.

This year one of my favorite ICs, namely NIDA, has only just issued new Cycle I grants** this week, they hit RePORTER today.

March friggin 9th.

Six new R01 awards. Three K01, three K99s, one R15, one "planning grant" and three SBIR.

Even this is just a trickle, compared to what they should be funding for one of their major Cycles. I anticipate there will be a lot more coming out over the next couple of weeks so that they can (hopefully?) clear the decks for the Cycle II awards that are supposed to fund April 1.

I pity all those poor PIs out their waiting, just waiting, for their awards to fund. I cannot imagine why NIDA chooses to do this instead of at least trickling out the best score awards and the stuff they KNOW they are going to fund, way back in December***.
__
*Statistically undifferentiable from never

**You can tell by clicking on the individual awards and you'll see that they (R01s anyway) end Nov 30 or Dec 31 for the initial round of funding. These are Cycle I, not upjumped Cycle II.

***Some ICs do tend to fund a few new awards in December, no matter what the status of Congress' activity on a budget.

7 responses so far

On sending trainees to conferences that lack gender balance

Neuroscientist Bita Moghaddam asked a very interesting question on Twitter but it didn't get much discussion yet. I thought I'd raise it up for the blog audience.

My immediate thought was that we should first talk about the R13 Support for Scientific Conferences mechanism. These are often used to provide some funding for Gordon Research Conference meetings, for the smaller society meetings and even some very small local(ish) conferences. Examples from NIDA, NIMH, NIGMS. I say first because this would seem to be the very easy case.

NIH should absolutely keep a tight eye on gender distribution of the meetings supported by such grant awards.The FOA reads, in part:

Additionally, the Conference Plan should describe strategies for:

Involving the appropriate representation of women, minorities, and persons with disabilities in the planning and implementation of, and participation in, the proposed conference.
Identifying and publicizing resources for child care and other types of family care at the conference site to allow individuals with family care responsibilities to attend.

so it is a no-brainer there, although as we know from other aspects of NIH the actual review can depart from the FOA. I don't have any experience with these mechanisms personally so I can't say how well this particular aspect is respected when it comes to awarding good (fundable) scores.

Obviously, I think any failure to address representation should be a huge demerit. Any failure to achieve representation at the same, or similar meeting ("The application should identify related conferences held on the subject during the past 3 years and describe how the proposed conference is similar to, and/or different from these."), should also be a huge demerit.

At least as far as this FOA for this scientific conference support mechanism goes, the NIH would appear to be firmly behind the idea that scientific meetings should be diverse.

By extension, we can move on to the actual question from Professor Moghaddam. Should we use the additional power of travel funds to address diversity?

Of course, right off, I think of the ACNP annual meeting because it is hands down the least diverse meeting I have ever attended. By some significant margin. Perhaps not in gender representation but hey, let us not stand only on our pet issue of representation, eh?

As far as trainees go, I think heck no. If my trainee wants to go to any particular meeting because it will help her or him in their careers, I can't say no just to advance my own agenda with respect to diversity. Like it or not, I can't expect any of them to pay any sort of price for my tender sensibilities.

Myself? Maybe. But probably not. See the aforementioned ACNP. When I attend that meeting it is because I think it will be advantageous for me, my lab or my understanding of science. I may carp and complain to certain ears that may matter about representation at the ACNP, but I'm not going on strike about it.

Other, smaller meetings? Like a GRC? I don't know. I really don't.

I thank Professor Moghaddam for making me think about it though. This is the start of a ponder for me and I hope it is for you as well.

16 responses so far

CSR Head Nakamura Makes Bizarre Pronouncement

Feb 02 2016 Published by under Careerism, NIH Careerism

An email from the CSR of the NIH hit late yesterday a few days ago, pointing to a number of their Peer Review Notes including one on the budget bump that we are about to enjoy.
Actually that should be "some selected few of us will enjoy" because

“While $2 billion is a big increase, it is less than a 10 percent increase, and a large portion of it is set aside for specific areas and initiatives,” said Dr. Nakamura. “Competition for funding is still going to be intense, and paylines will not return to historic averages . . .

Yeah, as suspected, that money is already accounted for.

The part that has me fired up is the continuation after that ellipsis and a continuing header item.

So make sure you put your best effort into your application before you apply.”

Counterproductive Efforts
“We know some research deans have quotas and force their PIs to submit applications regularly,” said Dr. Nakamura. “It’s important for them to know that university submission rates are not correlated with grant funding. Therefore, PIs should be encouraged to develop and submit applications as their research and ideas justify the effort to write them and have other scientists review them.”

As usual I do not know if this is coming from ignorance or calculated strategy to make their numbers look better. I fear both possibilities. I'm going from memory here because I can't seem to rapidly find the related blog post or data analysis but I think I recall an illustration that University-total grant submission rates did not predict University-total success rates.

At a very basic level Nakamura is using the lie of the truncated distribution. If you don't submit any grant applications, your success rate is going to be zero. I'm sure he's excluding those because seemingly that would make a nice correlation.

But more importantly, he is trying to use university-wide measures to convince the individual PI what is best for her to do.

Wrong. Wrong. Wrong.

Not everyone's chances at that institution are the same. The more established investigators will probably, on average, enjoy a higher success rate. They can therefore submit fewer applications. Lesser folk enjoy lower success rates so therefore they have to keep pounding out the apps to get their grants.

By extension, it takes very little imagination to understand that depending on your ratio of big important established scientists to noobs, and based somewhat on subfields, the apparent University-wide numbers are going to swamp out the information that is needed for each individual PI.

In short, this is just another version of the advice to young faculty to "write better grants, just like the greybeards do".

The trick is, the greybeards DO NOT WRITE BETTER GRANTS! I mean sure, yes, there is a small experience factor there. But the major driver is not the objective quality but rather the established track record of the big-deal scientist. This gives them little benefits of the doubt all over the place as we have discussed on this blog endlessly.

I believe I have yet to hear from a new-comer to NIH grant review that has not had the experience within 1-2 rounds of a reviewer ending his/her review of a clearly lower-quality grant proposal with "....but it's Dr. BigShot and we know she does great work and can pull this off". Or similar.

I have been on a study section round or two in my day and I am here to tell you. My experience is not at all consistent with the idea that the "best" grants win out. Merit scores are not a perfect function of objective grant quality at all. Imperfectly prepared or boring grants get funded all the time. Really exciting and nearly-perfect grants get unfundable scores or triaged. Frequently.

This is because grant review hinges on the excitement of the assigned reviewers for the essence of the project. All else is detail.

You cannot beat this system by writing a "perfect" grant. Because it may not be perfect for all three reviewers no matter how well it has been prepared and how well vetted by whatever colleagues you have rounded up to advise you.

Nakamura should know this. He probably does. Which makes his "advice" a cynical ploy to decrease submissions so that his success rate will look better.

One caveat: I could simply be out of touch with all of these alleged Dean-motivated crap apps. It is true that I have occasionally seen people throw up grant applications that really aren't very credible from my perspective. They are very rare. And it has occasionally been the case that at least one other reviewer liked something about an application I thought was embarrassingly crappy. So go figure.

I also understand that there are indeed Deans or Chairs that encourage high submission rates and maybe this leads to PIs writing garbage now and again. But this does not account for the dismal success rates we are enjoying. I bet that magically disappearing all apps that a PI submitted to meet institutional vigor requirements (but didn't really mean to make a serious play for an award) would have no perceptible effect on success rates for the rest of us. I just haven't ever seen enough non-credible apps for this to make a difference. Perhaps you have another experience on study section, DearReaders?

Finally, I really hate this blame-the-victim attitude on the part of the CSR and indeed many POs. There are readily apparent and demonstrable problems with how some categories of PIs' grants are reviewed. Newer and less experienced applicants. African-American PIs. Women. Perhaps, although this is less well-explicated lately, those from the wrong Universities.

For the NIH to avoid fixing their own problems with review (for example the vicious cycle of study sections punishing ESI apps with ever-worsening scores when the NIH used special paylines to boost success rates) and then blame victims of these problems by suggesting they must be writing bad grants takes chutzpah. But it is wrong. And demoralizing to so many who are taking it on the chin in the grant review game.

And it makes the problems worse. How so? Well, as you know, Dear Reader I am firmly convinced that the only way to succeed in the long term is to keep rolling the reviewer dice, hoping to get three individuals who really get what you are proposing. And to take advantage of the various little features of the system that respond to frequent submissions (reviewer sympathy, PO interest, extra end of year money, ARRA, sudden IC initiatives/directions, etc). Always, always you have to send in credible proposals. But perfect vs really good guarantees you nothing. And when perfect keeps you from submitting another really good grant? You are not helping your chances. So for Nakamura to tell people to sacrifice the really good for the perfect he is worsening their chances. Particularly when the people are in those groups who are already at a disadvantage and need to work even harder* to make up for it.

__
*Remember, Ginther showed that African-American PIs had to submit more revisions to get funded.

25 responses so far

Older posts »