NIH should involve the not-yet-funded in review

Nov 09 2015 Published by under NIH, NIH Careerism

Following up my post on RFAs and the inherent self-reinforcing conservatism of NIH grant review.

35 responses so far

  • Philapodia says:

    You want to put riff-raff on study section so they can "learn"? This will lead to chaos, I tell you, Chaos!!! Only those with decades of experience can effectively choose winners, just like Marian Koshland.

    Why, the riff-raff will probably score grants based on "merit" (whatever the hell that means) rather than funding history and productivity. I mean, those who currently have grants have them because they have done a lot of NIH funded research in the past and obviously deserve them.

    Damn socialist!

  • rs says:

    Yep, it will democratize the whole process.

  • Michael says:

    Not sure I understand - the non-funded are already involved in review, via the Early Career Reviewer program http://public.csr.nih.gov/ReviewerResources/BecomeAReviewer/ECR/Pages/default.aspx

  • drugmonkey says:

    Philapodia- in this context, my interest in having not-yet-successful applicants funded is to oppose the inherent conservativeness of having grants reviewed only by the successful.

    Michael- this ECR program is halting and limited. I think they are only supposed to have one on any given panel?

    I want to see SROs freed up and encouraged to recruit more noobs. not just as a baby-table intro with only two grants to review. as more of a regular thing.

  • Microscientist says:

    IT is also nearly impossible to get on a panel as an ECR. You better have your post-doc advisor or PhD advisor know someone who can strong arm whoever sets up the panel. So it further emphasizes coming from a big name lab.

  • Mikka says:

    The ECR program is not mandatory for SROs AFAIK. I could be wrong. I begged and groveled to several SROs to be included in a study section but only sensed eye-rolling disdain and annoyance at the irksome program.

    It could also be that I'm male and I'm white and from a research-intensive institution though. I'd be OK with it if this is the reason I never got considered.

  • docack says:

    I've been continuously funded by NIH since 2011 and still my requests to serve on a study section are never answered /acknowledged. Is there some secret here?

  • Noncoding Arenay says:

    Wait, so there's all this complaining about too many grants and increasing pressure on the reviewers, but from anecdata in these comments such as the one above from docack, it seems as if there are (funded)folks really wanting to review but not getting a chance. Gotta prove one's non-riffraffiness first I suppose...?

  • Newbie PI says:

    The ECR program involves heavy instruction from the SRO who will also often read your reviews and comment on them in advance before you submit them. You then get STRONG encouragement during the Read Phase to adjust your scores and comments to be more like the other two reviewers if there is any disagreement. Thus, the ECRs don't really get to offer much of a different perspective even if they are allowed on the panel. (That was my experience anyway. Though it was an enormously educational experience in terms of learning what gets criticized.)

  • Noncoding Arenay says:

    @Newbie PI - it always seemed to me that the ECR program was designed not as much to obtain critical input from the ECR on the applications that they have been assigned, but rather to provide an opportunity to exercise/experience SS and use that knowledge toward improving their approach to grant writing and enhance their chances of success. Your experience aligns perfectly with that.

  • Comradde PhysioProffe says:

    CSR is under a lot of political pressure from schmucks like McKnight to keep riff-raff off study section. And even when they choose study section chairs--who come from within the existing roster--there is pressure to appoint the most "elite" members of the panel.

  • physioprof says:

    Dude, will you fix your piece of shitte blogge so it stops sending every single one of my fucken comments to moderation?

  • drugmonkey says:

    Only non "elite" riffraff go to moderation.

  • Ola says:

    Well, if you want to take "PEER" review to its ultimate conclusion, then yes allow unfunded peeps on the study section, but only let them review (as 1/2/3) the proposals of NIs or others without funding. Then the funded old git proposals get reviewed by other funded old gits. That way, everyone gets evaluated by their true peers.

    < tongue now firmly out of cheek >

    The problem I see with this idea (which is otherwise a good one IMO), is that the pool of un-funded PIs is astonishingly small. The days of the start-up with a grace period of 3-5 years to obtain that elusive first R01 are long gone. Even those with K99/R00 at the start are being pushed to get moar moolah! There just isn't a large pool of people running new labs on startup with no outside funding.

    A further problem - those that are in this pool were likely selected for those jobs on the basis of a glam pub record or coming out of a glam lab/school. Do we really need a bunch of millennials with CNS papers who think they're shit hot, stinking up study section?

    Unless of course you're proposing post-docs as the un-funded peeps. That's probably workable, although would be more difficult to get buy-in from the olds.

  • Philapodia says:

    " There just isn't a large pool of people running new labs on startup with no outside funding."

    I disagree. Most newbie faculty I know don't have substantial funding when they start and rely on startup funds to get going for the first 2-3 years until they hit the NIH lottery (if they ever do). Getting an R01 before you have an independent lab is extremely difficult, and since having an R01 is one of the de facto litmus tests for getting on study section, most newbies don't have a chance. Look at the study section rosters and you'll see the occasional Assistant Professor (probably a token ECR), but almost all are Associate rank or above.

  • L Kiswa says:

    "The problem I see with this idea (which is otherwise a good one IMO), is that the pool of un-funded PIs is astonishingly small. The days of the start-up with a grace period of 3-5 years to obtain that elusive first R01 are long gone. Even those with K99/R00 at the start are being pushed to get moar moolah! There just isn't a large pool of people running new labs on startup with no outside funding."

    Yeah -- no. I have been explicitly told by a couple of colleagues on SS that I don't have much of a shot at R01 level funding until I have ~5-10 non-review papers from my own lab. I'm sure this is not a sufficient condition (10 papers= $$$ would be nice), but a simple scroll through REPORTER supports their comments. In my field, a well managed startup, supplemented with some internal and small foundation grants, may reasonably yield ~10 senior author papers over 4-5 years.

    I applied for ECR as soon as I was eligible (2+ senior author papers), but haven't been able to get on. Should I be contacting the SRO to get on a panel?

  • Noncoding Arenay says:

    "Unless of course you're proposing post-docs as the un-funded peeps. That's probably workable"

    Sorry, but postdocs have a lot to learn still before they get onto anything close to SS. I would suggest that any unfunded or lightly funded (R03. R21, etc) PIs who might be considered for SS should have had a certain # of years (say 3 yrs) as independent PIs AND have gone through the ECR program. That way they know what it really takes to run a lab, budget/personnel management and have had a taste of SS as ECRs so that they can merge onto the SS freeway better.

  • Newbie says:

    As a new, lightly-funded PI (R21, R03, foundation, small DoD, and a early faculty development K) I am considering the ECR program for next year. I start gearing up for my full PI R01s in 1.5-2 yrs.

    If anyone here has done or interacted with the program, what amt of time commitment and level of insight do you actually put into and get from the program?

  • girlparts says:

    I just returned from my ECR experience. I highly recommend it. Just seeing a wide variety of grants from someone other than your mentors and close colleagues is incredibly valuable. I got 8 grants (tilted towards R21s) and was first reviewer on a few. It is only for one round, so the time commitment is nothing like serving 3x a year for 3 years. I got to see some of the arbitrariness of it all, and saw grants from folks I really respect get triaged, which made me feel a little better. I *think* I got a better sense of the subtleties of what scope of grants do well in that study section, and some quirks, like what cell lines reviewer x just hates.

  • Philapodia says:

    @Newbie
    "I start gearing up for my full PI R01s in 1.5-2 yrs."

    Why are you waiting so long? If the stars align just right and you get a hit on your first R01 try, you would be in your 3rd year when you get the grant. Chances are that it will take several rounds of submission before you learn how the study section works and get a hit, so by waiting so long you potentially put yourself in a position where your other funding runs out before you get an R01. If you have so much other funding, you likely have lots of data for writing R01s at this point. Apply early and often!

  • Newbie says:

    @ Philapodia

    Half-truth: MIRA, DP2 apps in, (and another new investigator R01 going in Feb) but my partner and I are having first kid in under 9 months and her job requires that I spend some quality fathering time... so I'm dropping grant writing so that my time around lab can be used for mentoring two wonderful postdocs and home time for paper writing. I budgeted my startup with that in mind.

    @girlparts

    Ty, that's what I wanted to hear. How did you pick review group or was it picked for you?

  • Philapodia says:

    @Newbie

    Sounds like you have a good plan and have a lot of apps in, so good on you mate! Being a good parent is the most important thing (IMHO), and I applaud your approach.

    I was an ECR a few years back, but I never applied for the program and never specifically asked to be an ECR. I had received a small R03 a couple of years before which I think put me in the pipeline, and the SRO for the SS I reviewed for contacted me and asked if I wanted to do it. I think it came down to the SRO thinking I had the right skill set and wanting to help out an early career type. Not quite sure, though...

  • drugmonkey says:

    Should I be contacting the SRO to get on a panel?

    Yes. and if you have any friendly colleagues on appropriate panels try to get them to recommend you to the SRO as well.

    I would suggest that any unfunded or lightly funded (R03. R21, etc) PIs who might be considered for SS should have had a certain # of years (say 3 yrs) as independent PIs AND have gone through the ECR program.

    Meh. Learn by doing, is my view. I got thrown in as an ad hoc with a lightish load, sure, but it wasn't this 2 grant baby stuff. and at the time asst prof ad hocs weren't viewed as baby table trainees. as noobs, sure, but it was all very encouraging, from SRO to the other panel members IME. (not least of which because they'd been reviewing my grants for the past few rounds.) Still, I'm not sure the difference of 1-2 years (i was about 4.5 iirc) and with a fundable score (they'd just handed me one, finally) made any difference in my reviewing. you still need to learn the review job on the job.

    That way they know what it really takes to run a lab, budget/personnel management

    I disagree. My view on 'what it really takes', if I expect certain qualities in a reviewer, is not associated with career tenure that directly. By way of example, I suspect some guy who was hired in the 70s on hard money salary and sailed deep into the naughties before ever so much as having to revise a grant application is less likely to understand my reality than a 2nd yr asst prof who landed a job after 5-8 years of postdocing in high pressure soft-money environments.

    what amt of time commitment and level of insight do you actually put into and get from the program?

    It takes a lot of time to review your first grants. I'd assume something on the order of 40 hr of pre-meeting work, but this is going to depend on your starting point vis a vis grants, grant review, etc. the entire process. Also on the actual grants you are assigned. If they are 2 right in your wheel house this is going to be easier than 3 in topic domains slightly outside your home base. It is also going to depend on how grant-geeky you are- you are allowed to read all the apps you are not in conflict with for the whole panel and allowed to read the critiques during the read phase week (just prior to the meeting). I would recommend devouring all of this (see comment from Girlparts, above), personally, but this is optional.

    then there's the meeting. typically 2 days, add time for travel. post-meeting you are supposed to edit but that probably takes less than 2 h.

    "level of insight"? everything. like the Matrix dude.

    I got 8 grants (tilted towards R21s) and was first reviewer on a few.

    glad to hear this. I had heard some ECR gigs being really limited, like 2-3 apps at most and no primaries. Much more effective to get something like a real load.

    Apply early and often!

    Agree

    I think it came down to the SRO thinking I had the right skill set and wanting to help out an early career type.

    Many SROs work hard to find people to serve as reviewers. They note who submits apps and in what domains. They ask their study section members. They may ask other POs. They may ask ex-study section members. They may PubMed search. But they are busy and you might as well make it easy for them.

  • Dr Becca says:

    I did ECR in February, maybe 7-8 months after applying. I had never had a grant reviewed by the study section I sat on (although I wish I had) and had never talked to the SRO, so I'm not sure why it worked out relatively smoothly. However, there were 100 proposals that round, so maybe they were desperate. I was 3rd reviewer for 4 grants, two of which ended up being triaged, so I didn't get to talk all that much during the meeting. It was still quite eye-opening, and worth the work, which even for 4 grants felt pretty substantial. Although maybe the bad ones take more mental energy?

  • Grumble says:

    They all take mental energy. Reviewing grants is like wrestling with dementors.

  • Ewan says:

    Seems like another lottery, really :). I've tried for ECR a couple of times, and have had R01s funded, but barely received even a response much less a SS spot.

  • MorganPhD says:

    @Noncoding Arenay
    "Sorry, but postdocs have a lot to learn still before they get onto anything close to SS"

    I assume that means that there is a formal portion of our postdoc "training" where we learn how to be on Study Section and properly review grants? I guess that happens in my 5th year.

  • Noncoding Arenay says:

    @DM "By way of example, I suspect some guy who was hired in the 70s on hard money salary and sailed deep into the naughties "

    I get the gist of what you are saying, but I was specifically talking about folks who are new on the track/independent position and therefore have pretty much "grown up" in the current funding climate. That said, there's no doubt that there are many postdocs that are exceptions to my previous comment.

    @MorganPhD refer to the rest of my post for that. In this funding climate, I really don't think SS or SROs have the patience to be "training" anyone (least of all postdocs) to review. If that were the case, the ECR program would have taken off big time already. And which is probably also the reason why we have people here saying they haven't been invited or responded to their requests to serve on SS despite being funded on R mechs including R01.

  • girlparts says:

    I was selected for ECR after several years of trying. There were some personal contacts that probably helped, but according to the SRO, I'd been submitting too many grants to that study section. He wasn't confident 6 months ahead of time that I would not have a conflict of interest. I had to assure him I would not send one their way that cycle.

  • drugmonkey says:

    NCA- do you realize Scarpa rammed through his purge of Assistant Professors *over the vociferous objections of many line SROs*? The ECR program hasn't "taken off" for many reasons having to do with the rules they have to live under (initially restricted in number per meeting and assignment load...maybe changing?) and a significant debate over the purpose of it. It started as affirmative action for the underrepresented (small Unis, region as well as reviewer characteristics)and quickly got shouted into "for everyone". Buyin is highly variable.

    Of course SROs and SS members have *always* trained new reviewers. Always. Ime patience is vast. Because everyone wants this process to go well *as they see it*.

  • Noncoding Arenay says:

    "Of course SROs and SS members have *always* trained new reviewers. Always."

    Sure, they have. But what about now? Why is it so difficult for newbies to get on SS (there are panel/IC variations of course)? I think CPP's comment nails it.

  • Newbie PI says:

    Another big surprise for me after study section was that I got "paid" $800. Of course this was a reimbursement for expenses, but NIH already took care of the major expenses, and I'd rather see them fund an extra R01 or two per year than pay for reviewer meals. I mean, we all have decent jobs in science or we wouldn't be there. Sure, 800 bucks appearing in my bank account was nice, but I would have served on the study section regardless because of the obvious career benefits.

  • Philapodia says:

    @Newbie PI

    I believe that the NIH has to pay reviewers as contractors for legal reasons. If we perform "work" for the NIH (i.e. reviewing grants) and travel for that work, the US government has an obligation to pay for that work. If they don't it could open up the NIH to allegations of quid pro quo. I'm sure there is a statute on this somewhere in yawing maw of US law, but I'm too lazy to look it up.

  • Philapodia says:

    BTW, I've chatted with a friend of mine who live in the DC area who does a lot of governmental contract work about what we get paid to do reviews, and he says that we pretty much get screwed like a $5 hooker according to normal contractor standards.

  • drugmonkey says:

    NewbiePI- you can always donate that money to your laboratory.

Leave a Reply