Archive for the 'Grant Review' category

Thoughts on NIH grant strategy from Associate Professor H. Solo

We spend a fair amount of time talking about grant strategy on this blog. Presumably, this is a reflection of an internal process many of us go through trying to decide how to distribute our grant writing effort so as to maximize our chances of getting funded. After all we have better things to do than to write grants.

So we scrutinize success rates for various ICs, various mechanisms, FOAs, etc as best we are able. We flog RePORTER for evidence of which study sections will be most sympathetic to our proposals and how to cast our applications so as to be attractive. We worry about how to construct our Biosketch and who to include as consultants or collaborators. We obsess over how much preliminary data is enough (and too much*).

This is all well and good and maybe, maybe....perhaps....it helps.

But at some level, you have to follow your gut, too. Even when the odds seem overwhelmingly bad, there are going to be times when dang it, you just feel like this is the right thing to do.

Submitting an R01 on very thin preliminary data because it just doesn't work as an R21 perhaps.

Proposing an R03 scope project even if the relevant study section has only one** of them funded on the RePORTER books.

Submitting your proposal when the PO who will likely be handling it has already told you she hates your Aims***.

Revising that application that has been triaged twice**** and sending it back in as a A2asA0 proposal.

I would just advise that you take a balanced approach. Make your riskier attempts, sure, but balance those with some less risky applications too.

I view it as....experimenting.

__
*Just got a question about presenting too much preliminary data the other day.

**of course you want to make sure there is not a structural issue at work, such as the section stopped reviewing this mechanism two years ago.

***1-2%ile scores have a way of softening the stony cold heart of a Program Officer. Within-payline skips are very, very rare beasts.

****one of my least strategic behaviors may be in revising grants that have been triaged. Not sure I've ever had one funded after initial triage and yet I persist. Less so now than I used to but.....I have a tendency. Hard headed and stupid, maybe.

12 responses so far

NIH Program Officers do not understand what happens during review

May 22 2015 Published by under Grant Review, Grantsmanship, NIH, NIH Careerism

It is one of the most perplexing things of my career and I still don't completely understand why this is the case. But it is important for PIs, especially those who have not yet experienced study section, to understand a simple fact of life.

The NIH Program Officers do not completely understand what contributes to the review and scoring of your grant application.

My examples are legion and I have mentioned some of them in prior blog posts over the years.

The recent advice from NIAID on how to get your grant to fit within a modular budget limit.

The advice from a PO that PIs (such as myself) just needed to "write better grants" when I was already through a stint on study section and had read many, many crappy and yet funded grants from more established investigators.

The observation that transitioning investigators "shouldn't take that job" because it was soft money and K grants were figuring heavily in the person's transition/launch plans.

Apparently honest wonder that reviewers do not read their precious Program Announcements and automatically award excellent scores to applications just because they align with the goals of the PA.

Ignorance of the revision queuing that was particularly endemic during the early part of my career (and pretend? ignorance that limiting applications to one revision round made no functional difference in this).

The "sudden discovery" that all of the New Investigator grants during the checkbox era were going to well-established investigators who simply happened not to have NIH funding before, instead of boosting the young / recently appointed investigators.

An almost comically naive belief that study section outcome for grants really is an unbiased reflection of grant merit.

I could go on.

The reason this is so perplexing to me is that this is their job. POs [eta: used to] sit in on study section meetings or listen in on the phone. At least three times a year but probably more often given various special emphasis panels and the assignment of grants that might be reviewed in any of several study sections. They even take notes and are supposed to give feedback to the applicant with respect to the tenor of the discussion. They read any and all summary statements that they care to. They read (or can read) a nearly dizzying array of successful and unsuccessful applications.

And yet they mostly seem so ignorant of dynamics that were apparent to me after one, two or at the most three study section meetings.

It is weird.

The takeaway message for less NIH-experienced applicants is that the PO doesn't know everything. I'm not saying they are never helpful....they are. Occasionally very helpful. Difference between funded and not-funded helpful. So I fully endorse the usual advice to talk to your POs early and often.

Do not take the PO word for gospel, however. Take it under advisement and integrate it with all of your other sources of information to try to decide how to advance your funding strategy.

23 responses so far

Crystal clear grant advice from NIAID

May 21 2015 Published by under Grant Review, Grantsmanship, NIH, NIH Careerism

from this Advice Corner on modular budgeting:

As you design your research proposal, tabulate a rough cost estimate. If you are above but near the $250,000 annual direct cost threshold, consider ways to lessen your expenses. Maybe you have a low-priority Specific Aim that can be dropped or a piece of equipment you could rent rather than buy new.

H/t: PhysioProf

Related Reading:

NIAID
Sample Grants

26 responses so far

Fighting with the New Biosketch format

I have been flailing around, of and on for a few months, trying to write my Biosketch into the new format [Word doc Instructions and Sample].

 

I am not someone who likes to prance around bragging about "discoveries" and unique contributions and how my lab's work is I am so awesomely unique because, let's face it, I don't do that kind of work. I am much more of a work-a-day type of scientist who likes to demonstrate stuff that has never been shown before. I like to answer what are seemingly obvious questions for which there should be lots of literature but then it turns out that there is not. I like to work on what interests me about the world and I am mostly uninterested in what some gang of screechy monkey GlamourHumpers think is the latest and greatest.

Ahem.

This is getting in the way of my ability to:

Briefly describe up to five of your most significant contributions to science. For each contribution, indicate the historical background that frames the scientific problem; the central finding(s); the influence of the finding(s) on the progress of science or the application of those finding(s) to health or technology; and your specific role in the described work.

Now interestingly, it was someone who works in a way most unlike the way I do that showed me the light. Actually, he gave me the courage to think about ignoring this supposed charge in the sample / instruction document. This person recommended just writing a brief sentence or two about the area of work without trying to contextualize the importance or significance of the "contribution". I believe I actually saw one of the five permitted subheadings on his version that was more or less "And here's some other stuff we work on that wasn't easily categorized with the rest of it."

I am at least starting from this minimalist standpoint. I don't know if I will have the courage to actually submit it like this, but I'm leaning towards doing so.

I have been hearing from quite a number of you that you are struggling with creating this new version of the NIH Biosketch. So I thought I'd open it up to comment and observation. Anyone have any brilliant solutions / approaches to recommend?

UPDATE:
One of the things that has been bothering me most about this is that it takes the focus off of your work that is specific to the particular application in question. In the most recent version of the Biosketch, you selected 15 pubs that were most directly relevant to the topic at hand. These may not be your "most significant contributions" but they are the ones that are most significant for the newly proposed studies.

If one is now to list "your most significant contributions", well, presumably some of these may not have much to do with the current application. And if you take the five sections seriously, it is hard to parse the subset of your work that is relevant to one focal R01 sized project into multiple headings and still show now those particular aspects are a significant contribution.

I still think it is ridiculous that they didn't simply make this an optional way to do the Biosketch so as to accommodate those people that needed to talk about non-published scholarly works.

56 responses so far

That study of peer review perplexes me

Apr 24 2015 Published by under Grant Review, NIH, NIH Careerism, Peer Review

I just can't understand what is valuable about showing that a 1%ile difference in voted score leads to 2% difference in total citations of papers attributed to that grant award. All discussions of whether NIH peer review is working or broken center on the supposed failure to fund meritorious grants and the alleged funding of non-meritorious grants. 

Please show me one PI that is upset that her 4%ile funded grant really deserved a 2%ile and that shows that peer review is horribly broken. 

The real issue, how a grant overlooked by the system would fare *were it to be funded* is actually addressed to some extent by the graph on citations to clearly outlying grants funded by exception.

This is cast as Program rescuing those rare exception brilliant proposal. But again, how do we know the ones that Program fails to rescue wouldn't have performed well?

23 responses so far

Your Grant in Review: Future Directions

Mar 17 2015 Published by under Grant Review, Grantsmanship

One of the most perplexing thing I have learned about the review of 5 year R01 NIH grant proposals is a species of reviewer that is obsessed with Future Directions.

It was a revelation to me in one of my first few study section meetings that some reviewers really want to see extensive comment on where the project might be heading after the completion of 5 years of work. As in, a whole subheaded paragraph at the end of the Research Plan. This is insane to me.

Right?

For the most part, we all recognize that ongoing results in your own lab and in the field at large are going to dictate what is important to pursue five years from now. So speculation about what is coming next is silly.

And especially when I was a relatively inexperienced grant writer who had been getting beat up for "over ambitious" plans contained in a single 5 year plan, well.... I was amazed that people wanted to see even more in a speculative, hand wavey paragraph.

Consequently, I struggle with this. But I have tried to include something about Future Directions in my proposals. Yes, even now that we have only 12 precious pages to describe the actual plans for the current proposal.

I have recently seen a summary statement that describes insufficient attention paid to the Future Directions as the "primary weakness" of the proposal. I cannot even imagine what this reviewer was thinking. How can this be the primary weakness? Unless there is literally nothing else to complain about. And we know that never happens.

70 responses so far

Your Grant in Review: Effort and systems designed for amateur scientists

Mar 11 2015 Published by under Grant Review, Grantsmanship, NIH, NIH Careerism

Since we're discussing the amount of PI salary that should be rightfully paid by the NIH versus a local University lately, I have a grant review scenario to mention.

It is not uncommon to see R01 proposals come in from PIs who say that they will charge the grant for "three months summer salary". As we know, this is likely a scenario where the Professor in question has a 9 month salary from his or her University and is permitted to supplement that with up to three months of salary from extramural support funds.

Let us assume we're talking a normal research plan for an R01 that involves research effort pretty much around the calendar year. We're not talking about something that requires focal field work for a few summer months and then can subside into a much lower level of activity for the rest of the year.

On first glance the reviewer can only assume that the PI's remaining 9 months are being paid by the University to DO SOMETHING. Despite comment from Neuro-conservative about situations that seem very strange and unique, my experience is that Universities put some expectation of non-research activity on that 9 month of salary*.

Unless the PI has specified an expectation of research in their official job description, the reviewer can only assume that the effort on the grant will only be available during the summer.

Such a proposal should be met with the utmost skepticism since the conduct of the research requires ongoing supervision of the staff**, at the very least. Right?

So the grantsmithing advice part of this post is that if you are in this sort of situation, be sure to make very clear what your University explicitly expects in terms of your nine-month-hard-salary time.

From the perspective of our ongoing discussion, how is this all supposed to work? What true amount of brain-second-cycles are available to the project at any given time throughout the year?

Teaching duties tend to be rather inelastic and research duties tend to be highly elastic. I can always put off working on a paper or data analysis for another day. I can pick and choose when to work on a poster or oral presentation. I can't really put off lecture at 8am just because I have some exciting results in the laboratory that I want to write up right now. Grading may be a teeensy bit more flexible but there are clear deadlines...unlike paper submissions and most unlike designing new research projects and/or collaborations. Also very unlike meeting with your grad students and postdocs about various things.

I would suggest that under the 50/50 time scenario proposed by Neuro-conservative, one of the two task demands is going to receive short-shrift in a large number of cases. This will mostly be determined by what type of University the PI is employed within. Those that lean towards research? Well, we all know about how the tenure stool really only has one leg. Research. Conversely, there are very high teaching load institutions that inevitably push research toward the background during the active instructional school year.

In these situations either the NIH is being fleeced to support undergraduate instruction or the undergraduate instruction support system (State general funds and tuition, the latter includes scholarships and the like btw, another interested party) is being fleeced to pay for the NIH's business.

The only ethical situation is when there is perfect balance between the expectations of the respective sources of financial support and the PIs actual distribution of work.

I do wonder how many NIH PIs that have nine month salary support actually achieve the appropriate balance of brain effort devoted to their respective tasks. I bet not many.

__
*I would like to hear some specific language from people's job descriptions that specify that their hard money effort is supposed to be devoted X amount to research, btw. I know these do exist. How commonly?

**Naturally these sorts of proposals are often coupled with 12 mo of full time effort from trainees or techs which supports the notion that the project is not limited to the summer months.

77 responses so far

Thought of the Day

Feb 11 2015 Published by under Grant Review, Grantsmanship, NIH Careerism

I hate when I review grant proposals that are good, but clearly have been made pedestrian and conservative through the school of hard knocks. There is so much awesome that could be done by these people. It is so clear to me what the really high impact version of this grant should look like. (Not having any illusions about my own unique brilliance, I assume they could see it too. )

But the review realities batter PIs down into a defensive crouch, worried that if they step too far past their Preliminary Data or established expertise they will get crushed.

Because, of course, they would get crushed.

Sometimes I wish I were the Boss of Science more than other times.

28 responses so far

Your Grant In Review: Thought of the Day

Jan 07 2015 Published by under Grant Review, NIH Careerism

I've said it repeatedly on this blog and it is true, true, true people.

In NIH grant review, the worm turns very rapidly.

The pool of individual PIs who are appropriate to apply for, and review, NIH grants in a narrow subfield is a lot smaller than most people seem to think. Or maybe this is just my field.

My guiding belief is that the reviewer of a given grant is going to have one of her own grants reviewed by the PI of the proposal she just reviewed  in very short order. Or maybe it takes a half a decade, even more. But it will happen.

And PIs do not take kindly to jackholish reviews of their proposals.

As we all know, in this day and age it takes very little in the way of reviewer behavior to totally torpedo a grant's chances. You don't even have to be obvious about it*.

This is why I try as hard as I possibly can to ground my grant reviewing in concrete reasons for criticism.

Because I want the reviewers of my proposals to do the same. And it is the right thing to do.

We have a system of grant review that is at all times precariously balanced on a knife's edge that could slide off into Mutually Assured Destruction cycles of retaliation** at any time. And I am sure it happens in some study sections and amongst some reviewers.

Mutual Professional Respect is better. It is supported one review at a time by engaging our firmest professionalism to override the biases that we cannot help but have.

 

illustration from here.

__

*This is very likely the second hardest decision I have to make about registering a Conflict of Interest in reviewing grants. I have reviewed a lot of grants of PIs who have been on the study section panels reviewing my grants. I am pretty confident this is the case for just about anyone who has served a full term appointed on a study section and probably anyone who has reviewed with full loads in over about 3 panels as ad hoc. This in and of itself cannot be a reason to recuse yourself or they would never get anything reviewed. And as my Readers know, I am very firm in my belief that it is a fool's errand to try to game out which reviewers were on your proposals and which ones were...critical.

**And, gods above, pre-emptive counter-striking.

 

2 responses so far

More in "NIH responds to a non-problem by creating a problem"

Dec 05 2014 Published by under Grant Review, Grantsmanship, NIH, NIH Careerism

I can't even imagine what they are thinking.

This Notice informs the applicant community of a modification for how NIH would like applicants to mark changes in their Resubmission applications. NIH has removed the requirement to identify 'substantial scientific changes' in the text of a Resubmission application by 'bracketing, indenting, or change of typography'.

Effective immediately, it is sufficient to outline the changes made to the Resubmission application in the Introduction attachment. The Introduction must include a summary of substantial additions, deletions, and changes to the application. It must also include a response to weaknesses raised in the Summary Statement. The page limit for the Introduction may not exceed one page unless indicated otherwise in the Table of Page Limits.

First of all "would like" and "removed the requirement" do not align with each other. If the NIH "would like" that means this is not just a "we don't care whether you do it or not". So why not make it a mandate?

Next up...WHY?

Finally: How in all that is holy do they really expect the applicant to ("must") summarize "substantial additions, deletions, and changes" and to "include a response to weaknesses" in just one page?

I am starting to suspect Rockey is planning on burning the OER down to the ground before leaving for greener pastures.

18 responses so far

Older posts »