Search Results for "grantsmanship"

Apr 27 2016

Open Grantsmanship

Published by under Careerism,NIH,NIH Careerism

The Ramirez Group is practicing open grantsmanship by posting "R01 Style" documents on a website. This is certainly a courageous move and one that is unusual for scientists. It is not so long ago that mid-to-senior level Principal Investigator types were absolutely dismayed to learn that CRISP, the forerunner to RePORTER, would hand over their funded grants' abstract to anyone who wished to see it.

There are a number of interesting things here to consider. On the face of it, this responds to a plea that I've heard now and again for real actual sample grant materials. Those who are less well-surrounded by grant-writing types can obviously benefit from seeing how the rather dry instructions from NIH translate into actual working documents. Good stuff.

As we move through certain changes put in place by the NIH, even the well experienced folks can benefit from seeing how one person chooses to deal with the Authentication of Resources requirement or some such. Budgeting may be helpful for others. Ditto the Vertebrate Animals section.

There is the chance that this will work as Open Pre-Submission Peer Review for the Ramirez group as well. For example, I might observe that referring to Santa Cruz as the authoritative proof of authentic antibodies may not have the desired effect in all reviewers. This might then allow them to take a different approach to this section of the grant, avoiding the dangers of a reviewer that "heard SC antibodies are crap".

But there are also drawbacks to this type of Open Science. In this case I might note that posting a Vertebrate Animals statement (or certain types of research protocol description) is just begging the AR wackaloons to make your life hell.

But there is another issue here that I think the Readers of this blog might want to dig into.

Priority claiming.

As I am wont to observe, the chances are high in the empirical sciences that if you have a good idea, someone else has had it as well. And if the ideas are good enough to shape into a grant proposal, someone else might think these thoughts too. And if the resulting application is a plan that will be competitive, well, it will have been shaped into a certain format space by the acquired wisdom that is poured into a grant proposal. So again, you are likely to have company.

Finally, we all know that the current NIH game means that each PI is submitting a LOT of proposals for research to the NIH.

All of this means that it is likely that if you have proposed a 5 year plan of research to the NIH someone else has already, or will soon, propose something that is a lot like it.

This is known.

It is also known that your chances of bringing your ideas to fruition (published papers) are a lot higher if you have grant support than if you do not. The other way to say this is that if you do not happen to get funded for this grant application, the chances that someone else will publish papers related to your shared ideas is higher.

In the broader sense this means that if you do not get the grant, the record will be less likely to credit you for having those ideas and brilliant insights that were key to the proposal.

So what to do? Well, you could always write Medical Hypotheses and review papers, sure. But these can be imprecise. They describe general hypotheses and predictions but....that's about all.

It would be of more credit to you to lay out the way that you would actually test those hypotheses, is it not? In all of the brilliant experimental design elegance, key controls and fancy scientific approaches that are respected after the fact as amazing work. Maybe even with a little bit of preliminary evidence that you are on the right track, even if that evidence is far too limited to ever be published.

Enter the Open Grantsmanship ploy.

It is genius.

For two reasons.

First, of course, is pure priority claiming. If someone else gets "your" grant and publishes papers, you get to go around whining that you had the idea first. Sure, many people do this but you will have evidence.

Second, there is the subtle attempt to poison the waters for those other competitors' applications. If you can get enough people in your subfield reading your Open Grant proposals then just maaaaaybe someone on a grant panel will remember this. And when a competing proposal is under review just maaaaaaybe they will say "hey, didn't Ramirez Group propose this? maybe it isn't so unique.". Or maybe they will be predisposed to see that your approach is better and downgrade the proposal that is actually under review* accordingly. Perhaps your thin skin of preliminary data will be helpful in making that other proposal look bad. Etc.

*oh, it happens. I have had review comments on my proposals that seemed weird until I became aware of other grant proposals that I know for certain sure couldn't have been in the same round of review. It becomes clear in some cases that "why didn't you do things this way" comments are because that other proposal did indeed do things that way.

23 responses so far

Oct 12 2010

Reprint: Open Access Grantsmanship

This originally appeared on October 17, 2007.

BikeMonkey Re-Post
I was reading one of the summaries of the CSR Peer Review open house roundtable things, from the "Neuroscience" one. The thing that struck me was that over 50% of participants had never had a grant triaged.
Now the first thing that comes to mind is the people that NIH usually drags in to give them an "authoritative" opinion on various topics of interest. The opinions are most frequently sought from research luminaries, heads of institutions and society officialdom, i.e. (very) senior scientists.

These PIs are utterly unrepresentative of the pool of NIH applicants (and potential applicants).

In this particular case, however, there is a possible alternate explanation which is that triage is not in fact the norm for "good" scientists and those of us getting streamlined with any regularity are just writing bad grants. Or are bad scientists. etc. This is the sort of thing that keeps junior (and not so junior) scientists awake at night. It would be nice to have some current and specific information on triage rates across the NIH. Trouble is, this is not a data set that is easily obtained. The specific questions of the day being, "What proportion of scientists (as opposed to applications) applying to the NIH are triaged? Have these numbers changed over time? What do the per-PI triage rate distributions look like?".

I was discussing this a little bit with a colleague and came to the realization that this is the sort of information we don't even get a clear bead on with our usual collegial chit chat. Mostly because it is hard to keep track in a bunch of casual conversations how many grants someone has put in, how many times they've bitched about a triage, etc. I realize also because it is ever so slightly taboo to really ask someone these sorts of things. I, for one, wouldn't feel comfortable asking "So, how many grants have you put in and how many triages?" The few (two?) times someone has asked me for similar specifics on how many grants I've put in, I recall having a slight negative reaction. Like "Hey, that's private dude!". There's another issue which I realized after quashing the first sentiment which is that even I don't keep track of the numbers very well. I know this sounds strange to someone in year 2 of grant writing but after a while...

So, for today's lesson and in the spirit of OpenScience, I've bothered to pull my grant submission data from Commons for your entertainment and derision. Be kind. YMMV, of course. And naturally, this only counts the stuff where I have been the PI for the submission.

I've been putting proposals in since early 2000. I count up 20 Type 1 and 2 Type 2 applications submitted. Yikes, has it really been that many? In that list I have 8 A1s and 2 A2s. Fifteen of the applications were scored and 7 were streamlined. Three were funded so far. (I will note that this is not sufficient in soft-money land, of course and the answer to "how?" is that I've acquired at least an equivalent portfolio through sub-components, pilots and the like.)

Of course, the question of individual "success rate" beyond simple definitional purposes is complicated. I've abandoned my Type 2 after two triages so there would theoretically have been a third chance that I've chosen not to take. I've lost interest in pursuing a particular line with two grants because a competitor in the field (and now at least two I note) has gotten funded to do very similar stuff. I have at least three that are in active revision mode, although these will trickle across several rounds as I get to them. I also have a few more that are somewhat promising after the -01 or A1 review but are a bit down my priority list. Never gone, of course. Especially the ideas. Nevertheless it is hard to determine what the denominator should be. As the DM is fond of pointing out, if you haven't revised you haven't really submitted an application.

I note something a little more subtle which is that I have, for the most part, taken at least two lines of attack on a given research area of interest to me. This means that one of the two usually gets abandoned if the other is funded or sidelined if the other has a more promising score. Sometimes a sidelined one later gets dropped because my own work has moved on, the field indicates different directions or a competitor gets a related grant funded.

What other fun things does this sort of review reveal?

  • So far, I've never had a grant triaged in an SEP review even though a couple of 250-280 range scores show that it was a close thing.
  • In the salad days of the early noughties a 170 was a 19%-ile and fundable, in this most recent round a 28.6%-ile and not even close.
  • I have 5 scores in the 160-175 range over the years. I might encourage people to view this range as a good indicator that you are being taken seriously as a scientist, you have good ideas and can write a grant. It may be a matter of luck to improve from this range. For example it is hard to show where my one 120/1.6% scored application is this dramatically/categorically different, say, from my 160-170 scored ones.
  • Of the 7 applications that have been reviewed in the most-frequent study section, I have 4 triages as well as a personal best score. This is relevant to theories that "that particular study section hates me".
  • None of my proposals funded to date have gotten there after an initial triage. This stat is contaminated by the reduced chance that I will have revised a triaged proposal, of course.

So, nothing too surprising here since I had a pretty decent seat-of-the-pants recollection of how I've been doing. I think the most interesting thing is the fate of triaged applications. Mostly because a very common question from new applicants is "Should I abandon a triaged application?". My default response to this is "No way", mostly because of the way revisions are treated; DM has similar attitudes posted here and there. Also because of a sort of back-of-the-head suspicion that we've had applications triaged in our section which then are revised into highly competitive applications. But are they? Again, this is an area where some hard CSR data could be useful. The anecdote of me suggests at this point that perhaps it is wise to abandon a triaged application. This counters my gut feeling but the data are what they are.

The link regarding the 50% never-triaged also suggests that 23% of respondents had a previously triaged grant eventually win funding.

Looking again at Commons, it would appear that I have submitted four additional grants with myself as the PI since writing this- one new, one A1 and two A2. All were scored and two have been funded. I have also put in roughly a grants' worth of effort on about 5 additional submissions over the time since the original post.

10 responses so far

Sep 22 2008

Interpreting Grantsmanship Advice, Take Eleventy

Published by under Grantsmanship

Your local Office of Grants and Contracts staff only get a look at one part of the grant puzzle and are therefore just as bad as everyone else in accounting for career stage when providing grant advice. This thought struck me recently when I was speaking with a colleague who is preparing to submit her first NIH R01 application. Now the degree of involvement of your Grants & Contracts people varies quite a bit from place to place so YMMV. In this particular case we're talking about the type of person who assists the PIs with the final assembly of the grant and therefore has some experience with respect to the parts which are not the core scientific parts (e.g., the biosketches, the environment and equipment descriptions, the Vertebrate Animals or Human Subjects, etc.). A person who in their Pre-Award role can answer Institutional information questions, provide guidance on how this particular University handles particular details, etc. The type of person who might helpfully tell the newbie on her first grant to take a look at the BigCheez's sections and use those, or that might say it looks okay if the n00b does this on her own. You might be inclined to take this person's word since they are so experienced in seeing what gets funded and what gets triaged at your institution.
Bad Idea.

Continue Reading »

15 responses so far

Dec 22 2016

Twelve Months of Drugmonkey (2016)

Published by under BlogBlather

I've been doing these year-end summaries for quite some time now. Previously I've posted a link to the first post of every month. For this year I'm going to shake it up and post the last entry of the month.

Jan: In the NIH extramural grant funding world the maximum duration for a project is 5 years.

Feb: There are these moments in science where you face a decision...Am I going to be the selfish asshole here?

Mar: Jocelyn Kaiser reports that some people who applied for MIRA person-not-project support from NIGMS are now complaining.

Apr: The Ramirez Group is practicing open grantsmanship by posting "R01 Style" documents on a website.

May: By now most of you are familiar with the huge plume of vapor emitted by a user of an e-cigarette device on the streets.

Jun: A Daniel Sarewitz wrote an opinion piece in Nature awhile back to argue that the pressure to publish regularly has driven down the quality of science.

Jul: The other lesson to be drawn from recent political events and applied to science careers is not to let toxic personalities drive the ship.

Aug: From the NYT account of the shooting of Dennis Charney:

Sep: The NIH FOAs come in many flavors of specificity.

Oct: Imagine that the New Investigator status (no prior service as PI of major NIH grant) required an extra timeline document?

Nov: So. A federal judge* managed to put a hold on Obama's move to increase the threshold for overtime exemption.

Dec: If you love the NIH and its mission, your mantra for the next four years is a simple one.


3 responses so far

Aug 12 2015

Repost: An Honor Codes' Second Component and Research Science

Published by under Uncategorized

This was originally posted October 4, 2007.

Many academic honor codes boil down to two essential statements, namely "I will not cheat and I will not tolerate those who do". For "cheat" you may read any number of disreputable activities including plagiarism and research fraud. My alma mater had this sort of thing, I know the US military academies have this. Interestingly a random Google brings up some which include both components (Davidson College, Notre Dames, Florida State Univ (which as been in the academic cheating news lately), and some which do not (CU Boulder, Baylor); Wikipedia entry has a bunch of snippet Honor Codes. The first component, i.e. "don't cheat" is easily comprehended and followed. The second component, the " I will not tolerate those who do" part is the tricky one. Continue Reading »

12 responses so far

Aug 13 2014

On "transactional" science

Published by under NIH,NIH Careerism

Important questions from Paul Knoepfler:

In today’s transactional dominated world, scientists are spending an increasing proportion of their time basically fundraising. Writing grants. Honing grantsmanship. Doing experiments specifically for grant preliminary data rather than driven by transformative ideas. Working the philanthropy side of things.

By contrast, transformative activities would include these kinds of things: reading, thinking, teaching, mentoring, model building, listening to others, doing risky pilot experiments, etc.

So are you as transformative a scientist as you think or has transactional science become a dominant vein in your daily professional life? How is this playing out more generally in science?

Can you have the best of both worlds to be transformative and transactional?

I think the answer to the last question is that sure, one can be both transformative and transactional...and even still have fun in the lab. It is possible.

Is it better to spend less time raising funds? Better to spend less time working for preliminary data and more time working to get the paper closed out?

of course.

Nobody is in this merely to raise support for their lab.

But to ask this question is to be in denial.

The question has a bit of the upraised nose sniff to it. A bit of a slap at those who are in a situation in which raising laboratory funding looms large right now. It is a pat on the back for those who happen to be flush with cash and can go back to thinking about fun science for a little while.

My problem is that rewarding the people who don't have to work for their support very hard with more easy support just hardens the silo around a lucky few and makes it even harder for the rest of those poor chumps.

We (and here I mean Francis Collins and his comments on HHMI-like support for a select few) continue to think that success is the province of the brilliant deserving few. This gets in the way of recognizing that it is the outcome of giving any of a number of deserving someones the chance to succeed. It therefore, has the potential to give us even less bang for our funding buck since a select few are unlikely over the long haul to be as creative as a crowd would be.

11 responses so far

Nov 25 2013

Writedit and the Bergster publish a book on NIH grant strategy

I cannot wait until my copy of this book arrives.

How the NIH Can Help You Get Funded An Insider's Guide to Grant Strategy
Michelle L. Kienholz and Jeremy M. Berg
Oxford University Press
ISBN: 9780199989645

Kienholz is, of course, our longstanding blog friend writedit

Michelle Kienholz has partnered with scientists, clinicians, and public health researchers from all disciplines at dozens of universities to develop grant applications for almost every federal agency, including most grant mechanisms for each of the institutes and centers at the NIH. She volunteers her knowledge and experience on her popular blog, Medical Writing, Editing and Grantsmanship (as writedit), through which she has learned the most common and vexing concerns of researchers who interact with the NIH and how best to foster a partnership between investigators and NIH personnel.

and Jeremy Berg, PhD who

joined the University of Pittsburgh in June 2011 as the associate senior vice chancellor for science strategy and planning in the health sciences and a faculty member in the Department of Computational and Systems Biology. Prior, Dr. Berg became director of the National Institute of General Medical Sciences (NIGMS) at the National Institutes of Health (NIH) in November 2003.

is, well, familiar to our readership as the prior head of NIGMS, blogger and provider of much grant-funding data.

Berg recently twitted a teaser graph from the book which finally coughs up a comparison of funding policy for several ICs. According to the Twitter comment it refers to FY 2012 trends.
KienholzBerg-Funding Curves-2012

Nine ICs were willing to cough up data on the percentage of grants funded by the percentile they achieved at study section review. Lower is better, in NIH parlance so you can see that almost everything in the top 7-10% is getting funded across the ICs. Once you get to the top 35th percentile, your chances of funding are almost (but not quite) nil.

What is of best interest here is that we can finally see contrasting IC styles. There are 28 total ICs so this is just a subset but the NCI is huge and the NIMH is no slouch either. The topic domains range from cancer to the brain to metabolic to infectious disease to basic science so there is some breadth there too. I like this as a representative picture although we must always remain suspicious that those who chose not to send the authors their data might have done so for.....reasons.

Anyhow, what jumps out at me first is that NINDS has the sharpest dropoff past their apparent payline. If I am not mistaken, this is precisely the IC that is rumoured to assert their strictness with respect to payline. Strictness involves two choice points of the Program Staff. Whether to skip over grants that fall below (better than) the payline and whether to pick up grants that fall above the payline. Although I do seem to spot some skips under the payline for NINDS, NIA and NIAMS do not appear to have a similar skips. All the other graphs do appear to show skipping behavior. On the other side of "strict payline" behavior, clearly NINDS has funded some grants above their readily apparent payline. It's just that the distribution drops off much more steeply for them.

I note that NIGMS, NIDA and NIAID seem to have the smoothest curves of pickups away from the apparent payline. The reason I say "apparent" payline is that some institutes, of which NIDA and NIMH are two iirc, insist they do not have a payline. What I have asserted since I noticed Berg's posting of NIGMS' funding decisions is that published payline or not, ICs follow roughly the same behavior. These charts demonstrate that. All that differs is the slope of the curve defining above-apparent-payline pickups.

I'm hoping I'll have more to discuss once my copy of the book arrives.

14 responses so far

Mar 14 2011

An update on NIH grant success rates

A query at PhysioProf's blog from cackleofrad asks:

What has been the % chance of funding over the course of your career? Does your strategy of 2 RO1′s, on average, per year mean that you spend a great deal of time tweaking each one? I ask since my contemporaries are sending in ~4-8 RO1′s per year with the chance of funding ~10% or less.

As a reminder, the payline is not exactly equal to the "chance of funding". Reason being, in simple terms, that the paylines (where published for various ICs) are conservative. They tend to be the level at which the IC has overwhelming confidence they are ok if they fund all the grants that are reviewed as being at that percentile or better. They end up funding more grants than fall within the payline simply because of this conservativeness. If you want to know why they would be conservative, well it is one heck of a lot easier to deal with PIs who get funded absent an expectation of funding than to deal with PIs who do not get funded who have a good expectation that they would.

There is also the consideration that Program staff are intentionally conservative with the payline to facilitate funding proposals out of order- aka "exceptions" or "pickups". As demonstrated by the NIGMS funding data, the grant funded above an apparent (or published) hard payline are not selected randomly, the correlation with overall impact score is still pretty good. Nevertheless, you can see that in the margin between "everything gets funded" and "essentially nothing gets funded" there are quite a number of awards being made. So this is where the success rate gets much higher than the payline.

So in terms of your realistic chance of funding, the success rate is a better answer. The NIH almost always trumpets the success rate which is (almost) the number of grants funded in a given Fiscal Year divided by applications received for that Fiscal Year of funding. (I say "almost" because there is a bit of mumbo with applications that are revised within the fiscal year.) It is good enough for gov'mint work, as they say, when it comes to addressing cackleofrad's query.

This slide is from FASEB, via writedit. It gives a recent slice of the "success rate" picture. (For more see this, as well as this and this.)

Getting back to cackleofrad's question in the context of PhysioProf's observations, I note that the success rate for those without prior NIH awards during the real salad days of the doubling (~98-03) did not benefit as much as did the success rate of the experienced investigators. About 21-22% versus 25-26%. Four percentage points might not seem like much, but it is a 15% hit on the success rate.

Now go back to the origin of the data series here, 1995, when the doubling was just having an effect. New investigators enjoyed only a 19% success rate. In the most recent years, this number is anywhere from 17-19%. Yes, thanks to a number of efforts that have raised the newbs onto the trendline enjoyed by experienced investigators. Which is a good thing, true. But it sure looks like the success rates for new investigators right now compare favorably with the situation pre-doubling.

All I'm trying to point out with this is that if you want to know "how hard Prof X had it" when she started her career compared to how hard the new investigators have it today, you need to consider the newb success rates then and now, not just the overall NIH omnibus success rates.

17 responses so far

Oct 21 2010

Should you mention your grant score in your tenure track job application?

This question has now arisen in two places so it seemed worth a brief mention.

First, over at Medical Writing, Editing & Grantsmanship, mumbercycle asked:

I just recieved my K22-A1 priority score (20). This seems very safe with FY10 payline at 26. Any thoughts on the possibility of my app not getting funded? I am applying to positions and want to share the score and my potential funding, but don’t want to shove my foot in my mouth if something goes wrong. Thanks for any input!

and over at LabSpaces, Dr. O posted a similar query:

So I finally have it - the much-anticipated score on my K grant - and I have no idea what to think. It's a 31 - not a great score, but certainly fundable some years at certain institutes...I have several job applications due before the council meeting, a couple of which require some sort of funding for consideration. Should I include this score in the cover letters for these job applications? I don't have a good feel for how impressive/pathetic this will look, but I don't want to pass up including a potential "positive" in my applications.

I've been mulling this over and concluded first that yes, you want the knowledge that you have applied for a K-mech grant (or other) in the minds of the search committee. Of course. Although in this day and age you will be competing with many people who have also submitted grant applications, there will be those who have not. This puts you ahead of the game. If you have been scored, even better. It puts you into the "competitive for funding" category in my view. Now, if you had your application triaged, my suggestion is that you may not want to lead with this information.

Which brings us to the "how to communicate" question. I don't have any firm answers here, however, I would think not in the cover letter, unless that is one and the same with the research plan. Personally I think grant applications prepared and reviewed are best mentioned as part of your plans. It is pretty natural to conclude the part about what great science you want to accomplish by pointing out that "some of these experiments have been proposed in a K99/R00 application that received a priority score of XX. While not competitive for funding at this time, this shows the considerable enthusiasm of NIH reviewers for these studies".

An alternative to this is just to put it in your CV. Presumably you will have a section about various fellowships and travel awards and whatnot funding you have obtained competitively. Seems perfectly fine to me to have a "Pending" subsection, this would be the place.

Now, the discussion at LabSpaces seems to be leaning toward not including the specific score in your job application and just describe it as "competitive". I think this is a mistake. That's a little too vapor-ware for my taste and I think there is no way in hell you are going to get any reaction other than annoyance that you didn't just put in the actual score. Unless the particular search committee member doesn't give a rat's patootie about grant scores...but you aren't putting that information in there for this person, are you?

The reader at writedit's place is in a slightly different situation than is Dr. O*, given that the score is within the payline from the prior Fiscal Year. Writedit advises that the applicant address the chances of funding by mentioning the payline. I think you have to tread carefully here. You don't want to act like you expect it to fund, even if you have great reason to anticipate that happy event. Because things can happen. I would think at best you might describe your score as "highly competitive given the prior fiscal year payline for this IC was...".

My bottom line answer is that I think yes, you should include the information. The only trick is to do it in a way that is natural, places your score in the best possible light and yet does not oversell the chances of the award actually funding.
*Dr. O referred to some jobs being only open to those with funding. If so, this is a no brainer. You HAVE to include a lot of happy talk about your pending funding so this should not even be a question.

8 responses so far

Mar 16 2010

A survey on "science blogs"

After I read the now-infamous paper by I. Kouper, entitled "Science blogs and public engagement with science: practices, challenges, and opportunities", I was left in some confusion as to how the author selected 11 blogs to study. I was also curious about what my readers thought of when asked to generate a list of "science blogs" so I asked them. I left the request as general as possible because I was interested in what "science blog" meant as much as in specific examples.
For your entertainment and edification, I tabulated* the results from the 31 answers supplied as of this writing.

Continue Reading »

32 responses so far

Older posts »