Grant Reviewer Burnout

Oct 06 2008 Published by under Grant Review

The fall study section meetings are upon us and many colleagues are finishing up their huge pile of reviews, flying off for study section meeting and / or returning exhausted. I've been considering a more than usual level of reviewer behavior and attitude that speaks of burnout. I'm sure applicants will be thrilled.

As always I have the anecdotes of myself, the other reviewers on one specific panel (what they say and write) and random colleagues on other panels. I suppose that is a disclaimer. Also, I rarely go back and read summary statements during the post-meeting Edit phase. In my own case, I can say for sure that I frequently use the Edit phase to clean up some of my written critique that might be a little less-sympathetic (ok, nastier) than is really warranted. So it may simply be the case that I am seeing a little less polished pre-meeting writing than is usually the case. Again, a disclaimer.
Still. Things are a bit ugly out there. Yes, I'm including myself. But my impression is for much briefer summary statements as a general trend (no, I don't read all the pre-meeting critiques in the "reading" phase or during the actual meeting but I do spotcheck. plus I listen super attentively to each and every reviewer during the meeting :-)). A good deal of shortcutting in the description of the projects and a tendency to focus on a smaller set of pluses and minuses. Considerably less patience with *ThingsThatTickReviewersOFF!
I don't know quite what to think. Reviewer burdens do seem to be higher than the recent past, unsurprising since the push to decrease the number of total reviewers has been gaining steam. I had a weird impression this round, as well, that we had a great deal of new stuff to review with less from the usual suspects we've been seeing for years. Fewer revisions (hmm, can this be true?). Anyway, if this is generally true that reviewers are less likely to get revised versions of applications they just reviewed a few rounds back...well, the load is increased.
Is it that people that I know best are now pushing deep into their review commitment (of four years service)? Is there an exhaustion that sets in? This is actually pretty important to know because changing the duration of service is an easy fix. If people are just losing their review edge after 2-3 years of the grind, we need to deal with this I would think.
Perhaps grant proposals are changing in some way, for better or worse, and becoming harder to review? I would think not but it is possible that desperate investigators are so tied up in knots trying to get funded that the are writing CrazyQuilt grants, just throwing the kitchen sink in there trying to appeal to everyone, anyone? People are trying to adopt whatever they've heard of as the "best" style and struggling to overcome their natural grant-writing voice?
What think you?
Those of you reviewing... are you seeing signs of burnout?
Those of you receiving summary statements... are they getting grouchier?
UPDATE 10/08/08: drdrA has a post on grant writer burnout.
*I may get to some of this when I calm down but short version: When I advocate sending the proposal in sooner rather than later because of my belief that you can't write your way (or prelim data or publish your way) into guarantee of funding....? There's a caveat- don't write such a piece of garbage that it really irritates and insults the reviewers!

31 responses so far

  • BugDoc says:

    Did you perceive signs of favoring highly feasible (but perhaps, shall we say, non-innovative) proposals vs. somewhat more risky but higher payoff proposals? I just put in a submission and was wavering between including an aim supported by really exciting preliminary data that took the grant in a bit of a different direction OR sticking with the current consensus model and just doing further mechanistic analysis (safe). Being a glutton for punishment, I opted for the former, so we'll see how crushing the summary statement is!

  • Becca says:

    DM and PP should totally write a list of *ThingsThatTickReviewersOFF!

  • DrugMonkey says:

    BugDoc, I would think that you have to get a feel for study section by study section. We have a little bit of each perspective come up each and every meeting to my recollection. There will be whinging about feasibility in some cases and bragging on innovation and excitement in others.
    I have very little doubt that there would be some sections almost entirely filled with "tried-and-true" fans and others full of "latest SuperzExcitementELEVENTY!!!" fans.

  • river tam says:

    I totally agree with Becca. Top 10 things that tick reviewers off!

  • PhysioProf says:

    #10: Using numbered citations instead of (Author, Year). Each time I have to page back to the references list to figure out what the fuck paper you are citing, I hate you a little more.

  • DrugMonkey says:

    #9: When I look at a page and see solid uninterrupted text from upper left to lower right... I go crack another beer

  • Can't see how I wouldn't be burnt out after 4 years of reading that stuff. 2-3 years seems a lot more appropriate. I bet it would help the grouchiness, either way.

  • Art says:

    "#10: Using numbered citations instead of (Author, Year). Each time I have to page back to the references list to figure out what the fuck paper you are citing, I hate you a little more."
    In a proposal with 200 references, this practice can save a page or more of space.
    Which suggests a #8 - a bajillion references (most of marginal relevance) intended to cover one's butt against the criticism that the PI doesn't know the field.

  • River Tam says:

    LOL. Please don't stop! Think of the poor young people who need this advice! By stopping now you are hindering our education!

  • Barn Owl says:

    I've noticed a lot more sarcasm and snark in the review meetings, and a lot more picking at the budgets (especially percent(s) effort). Not that most of the snark makes it to the summary statements ... I think in part this reflects the frustration of knowing that some really deserving proposals will not get funded right now. Or maybe never, given Bush and his Most Excellent Adventure Fucking Up the Country.
    I get ticked off when the PI can't be arsed to calculate and describe the numbers of rats' arses, the age of the rats' arses, and the genotypes/strains of the rats' arses which will be used in the control and experimental groups for the proposed aims. Make a table. It isn't difficult. That's why they pay you the big bucks, arsehole. And when you are proposing to breed double or triple knockout mice, please include the chromosome locations of the targeted genes. It makes a difference, and I shouldn't have to look up the genes on the Jackson Labs website.

  • Toughen up, Buttercups.

  • crystaldoc says:

    "a lot more picking at the budgets (especially percent(s) effort)."
    Barn Owl, please expand. For a new investigator going with a 250K modular budget, what are the dos and don'ts?

  • Nat says:

    #10: Using numbered citations instead of (Author, Year).
    I hate that in papers too, and with everything going electronic, there's no reason that this space saving anathema should be allowed to continue.

  • Nat says:

    #10: Using numbered citations instead of (Author, Year).
    I hate that in papers too, and with everything going electronic, there's no reason that this space saving anathema should be allowed to continue.

  • Nat says:

    Sorry, the comments are hiccuping.

  • River Tam says:

    I would find it INSANELY useful if someone could expand on issues related to percent effort. NSF is starting to care about this, but I can't seem to get anybody to either tell me what it is (is it really the percent of you time you will devote to the project or how much of your paid time?) or tell me what level of percent effort would be considered too low. I know this might not translate across fields, but I'd really love to know how it works in any field.

  • DrugMonkey says:

    hmm, I tried to respond to crystaldoc before and it got aced. OK, so for crystaldoc and RT...
    Slippery topic and I'm not sure it "works" in any field.
    The idea is that you have an amount of work which you pour into your "job" (100% effort or, now in NIHspeak, 12 calendar months) however that may be defined. Within that job you can allocate proportions of your time or effort to various thinks like teaching, advising/mentoring, committee work and research.
    Simple enough, right? Except scientists' time and effort is notoriously fluid. How many hours do you work per year- more than 2080 of the 40 hr work week / 52 week-year, right? One wackaloon puts in 80 hrs a week with regularity, you work 60. Yet both of you have the same 100%/12 ca month allocation...
    So it is a bit of a crock from the outset and we all recognize that.
    Practically speaking, and getting back to crystaldoc's query, grant reviewers think the more effort allocated to a given proposal, the stronger the proposal. This has a tendency to grade with experience in that less experienced investigators are somewhat expected to devote more effort to a project than would Prof Bluehair. As with all grant review criteria it is hard to specify hard targets because reviewer behavior varies (even within-reviewer!).
    If forced, I'd say a newish investigator who proposes less than 20% effort is risking fire.
    Then we get into the issue of the PI who is on 9mo hard salary. I'm confused on these and there is no seeming standard for description of the school year effort. This can sometimes lead to problems in the review I will note....We all know that the teaching PI doesn't close down the lab during the school year but when only "summer months" are listed on the application...

  • juniorprof says:

    I would find it INSANELY useful if someone could expand on issues related to percent effort
    Correct me if I'm giving (getting) bad advice (fairly confident I am not). Our department administrators suggest that us new investigators go in for all grants at the percent effort that the department will eventually expect us to bring in terms of salary support (for me, 50%). This way we can at least have our butts covered when the startup funds run out. As percent efforts eventually start to make life complicated from the aspect of submitting and getting more grants funded they can be revised down to take care of exceeding whatever your max number is according to your institutional responsibilities (some number not to exceed 100%).

  • DrugMonkey says:

    Do keep in mind jp that while there are some minor hassles if you dip more than 25% below your proposed effort on a funded NIH grant, there is absolutely no problem* increasing your effort. So yes, in your own budget you need to be aware of your effort-covering needs. As far as the application goes you should be focused on grantsmanship issues. Issues, which, btw, include the initial funding of a proposal should you be so lucky. You need to make your numbers add up when a new grant is about to be funded and if you've proposed too much effort and happen to be fortunate with some other funding it can get sticky.
    What I mean is suppose you start off with one proposal at 50% effort. But by the time you get it funded, you've picked up collaboration here, small grants there, maybe gotten a lucky surprise with another big project. All of a sudden you have a hard time carving out 50% effort and you have to juggle things around. Far better to propose 30%, say, even if when that's the only award you actually have to cover 50% effort...
    *NIH-wise. do check with you local institution to make sure that they don't have a problem.

  • A full $250,000 per year modular budget R01 should generally have a maximum of 35% PI effort allocated to it. Anything more than that, and the additional financial burden will impair your ability to actually get the science done.

  • DrugMonkey says:

    Anything more than that, and the additional financial burden will impair your ability to actually get the science done.
    YMMV. Being out on your ass because you can't cover a soft-money salary also impairs your ability to get science done. Even if you are at bloody NIH cap it is better to spend the money and limp along rather than quit science...

  • You are correct, DoucheMonkey. The 35% number would be applicable only to those of us who are hard.

  • Becca says:

    I've been meaning to ask this, and this is as good an opening as any...
    My PI has an R01 that took multiple resubmissions to get renewed last time around. I think he realized, particularly near the end of the resubmissions, that he should have asked for more money (realistically what we're doing costs more like $250k/year than the $200k/year we're getting). But he was afraid that reviewers would see that and use it as an excuse to refuse the grant- something he really couldn't afford on resub #3. How realisitc is/was his fear? Do NIH reviewers really toss out grants based on the dollar number of the budget?
    In at least one respect, it may have hurt us (there's an internally-administered grad-student supplement funding thing that takes into account how much the R01 funding the project was cut by when deciding who to give awards to).

  • River Tam says:

    Thanks, DM, PP, and JuniorProf (aren't you in Helsinki right now?) I appreciate the feedback. Unfortunately, I'm one of those pesky 9-monthers. It gets all confusing because for my 9 mo appointment I am 50% research, go only a goddess or a good calculator know how much percentage effort I can put into a grant and not end up over 100%.
    Sorry Becca, didn't mean to distract. I'm really interested in how they answer your question too because I have friends dealing with similar budget perception issues.

  • Barn Owl says:

    Sorry to rant and then disappear, but I think DrugMonkey covered the percent effort basics quite well, and I just want to add a few variations/special cases that may arise or elicit discussion in grant review meetings.
    1. Keep track of your collaborations and collaborators, especially the ones that fall under, say 10% effort (1.2 calendar months, or whatever), and make sure that the duties for each person listed are described appropriately in the budget justification. It's easy to ax Dr. Grosskopf's 5% effort if all he seems to contribute are pontification and reputation. The reviewers aren't going to suggest that you collaborate with Dr. Manos-Milagros instead; they'll just cut that part of the budget. Also, if you are a collaborator on multiple grants already, it might look as if you're spreading yourself too thin.
    2. Some grant mechanisms (not through NIH) are specifically targeted at new investigators, and percent effort expected for the PI will be higher than the usual 25-35%. Try 50% or higher, especially if you're still doing benchwork and training students or postdocs yourself. If you are funded through such a mechanism, then your institution may release you from some teaching or administrative duties, especially if you were only expected to pay 25-35% of your salary from grants.
    3. If you are a BDS department chair, service core director, or administrator who still submits grant proposals, you cannot realistically ask for, say, 20% effort. Well, you can ask, but no one will believe that you can devote that much real effort to the proposed research. Plus, your salary is already so bloated that 20% of yours is more than my entire salary. Some postdoctoral fellow or senior scientist in your lab is writing your proposals anyway, so why not give her a raise?

  • Back to the burnout issue, I'd argue that it's no so much being on a 4-year term as much as it is spending three or four years knowing that you'll be lucky if one grant from your pile (or PDFs, more accurately) of assignments will actually be funded. One looks at the time one spends on carefully parsing these proposals and its easy to get discouraged knowing how much good science is going unfunded and undone. The sustained funding climate combined with the fact that things won't get better for at least a couple of years even if the Democrats are elected.
    Scientists serving on study section are by definition not afraid of hard work and long hours. However, one would feel more valued and fulfilled if this less-than-minimum wage professional service led to more than one project being funded. Yes, yes, dear SROs, we are to judge the science solely, but seeing 14%ile grants coming back as A2s breaks my heart.

  • neurolover says:

    So the directive about only one revision went out, effective after the January 25th deadline. Discussions Drugmonkey?
    I don't see how this is going to be fixed. As Abel Pharmboy says, what's the fun of critiquing the 85% of grants that are simply not going to get funded? I have a pile in my plate right now, and if I'm supposed to pick the *1* that will be funded, I have no confidence in my ability to pick it. And, what's the point of giving detailed criticism to all the others?
    Also, we're seeing an effect of over-extending, too, just like the financial monster that's collapsing. We've bubbled up science funding and are seeing it collapse while people wildly throw hail mary passes in an attempt to keep their funding alive. No wonder people who are judging those hail marys are cranky.
    Major suckage.

  • me says:

    Does a high budget cloud scientific review and the merit score?
    It shouldn't and it normally doesn't. But every once in a while, some self-righteous reviewer let's it happen and starts digging into the budget during scientific review.

  • Full $250,000 modular budgets are completely ignored by study sections, and approved without discussion.

  • whimple says:

    Full $250,000 modular budgets are completely ignored by study sections, and approved without discussion.
    This is not true for my study section.

  • Dr. Isis is spending her day finishing up a grant. She wants DrugMonkey to know tht, just for him, she has used a 10 point font, 1/4" margins, and has typed the entire thing single spaced. She shall send along a 6-pack for him to enjoy while he reviews it.

Leave a Reply