Peer review and the death sentence

Jun 25 2015 Published by under NIH, NIH Careerism

It is relatively easy to kill grant applications because the reviewer knows the applicant can always apply again.

Same thing for hiring decisions because surely some lesser University will hire the three other candidates on the short list.

In many tenure cases, the Department knows that this person will get a professorial rank job elsewhere*.

Germain's scheme is going to require peers in the field to pass a death sentence on the career. And to make the numbers add up, there will be a LOT of this.

Those peers know that they themselves will be up for chopping in the next 5-7 years.

I do not foresee much enthusiasm for scoring progress as deficient and this will only grow more intense with each successive 5 year review interval.

__
*and anyway these are so personal at this point that it is a different matter.

15 responses so far

  • Pinko Punko says:

    I will note that this "relative ease" is the domicile for all conscious/subconscious bias in marginal peer review decisions. I still wouldn't really change the system, but in our current condition it spreads the pain whereas Germain wishes to direct the pain. Currently we have death by 6-8 month intervals.

  • drugmonkey says:

    Right but when making the decision on a grant, a reviewer can easily ignore the possible career implications. Nobody know for sure which negative grant outcome is going to really spell THE END in a final way.

    I think that under Germain's scheme, that 5-7 year review of progress is going to be very different. It will come down to "Shall we kick this PI out of the pool forever? Yes/No?". There will be less ability to escape this framing. [except for the sociopathic types of course]

  • Germain's repeated claims that everyone he talks to likes his ideas sounds rather much like Pauline Kael's "I only know one person who voted for Nixon."

  • kalevala says:

    Here's an equally goofy counter-proposal:

    After your 5-7 years are up, there is no renewal. Want to keep doing science? Reinvent yourself and start something new. Your old research project will be followed up on by a younger PI, ready to try a fresh new approach. Better put some effort into mentoring your trainees if you hope it'll be one of them. You'll be fine. Unless you are riffraff, you'll have lots of ideas for new projects.

  • drugmonkey says:

    Logan's Run? 'Mkay....

  • Dr Becca says:

    Right but when making the decision on a grant, a reviewer can easily ignore the possible career implications. Nobody know for sure which negative grant outcome is going to really spell THE END in a final way.

    I have seriously considered putting this information in my recent grant proposals. This is it, folks. You want to quibble about whether I'm using the right vectors or whether those error bars in my prelim data overlap? Great, you've just closed my lab.

  • qaz says:

    Doc Becca - NIH very explicitly separates information about the business of science (too much money, too little money, labs closing, labs metastasizing) from the science of science. Study section is supposed to focus on whether the science is good and exciting or not. Program is supposed to focus on whether it's a good use of the money. Program definitely does take into account whether the money will close labs or downsize (fire) techs or lose knowledge bases. Program does less on the side of too much money, but I suppose that makes sense. Too much money is less of an immediate crisis. You can argue that program doesn't take this into account enough, but it's technically program's job, not study section's.

    You can argue that this is a problem (and I do), but this is the way our system works now. If you want to take into account whether closing a lab is imminent then we also have to be able to take into account whether there is too much money being sent to a given faculty as well.

    I suspect (and have observed) that study sections do take both of these questions (lab is desperate, lab is overfunded) into account, but only on the sly and never explicitly.

    PS. Kalevala's suggestion makes as much logical sense as killing all the graduate students or only funding the oligarchs.

  • Dr Becca says:

    Program definitely does take into account whether the money will close labs or downsize (fire) techs or lose knowledge bases.

    This was not my experience with my PO, who upon hearing that my lab would definitely be downsized and probably closed within the year because my ESI R01 went from 26% A0 to 44% A1 literally said, "it's not my job to help you."

  • physioprof says:

    I have seriously considered putting this information in my recent grant proposals. This is it, folks. You want to quibble about whether I'm using the right vectors or whether those error bars in my prelim data overlap? Great, you've just closed my lab.

    This is a really bad idea, as it will almost certainly have the opposite effect to what you might hope: it will be perceived as an attempted threat, and will make reviewers less well-disposed towards your application, not more. We had a similar thing at study section a few years ago where a lab head wrote a letter of "support" for an R21 submitted by a research scientist in his lab in which he wrote, "If this grant isn't awarded, the research scientist gets fired."

  • MTomasson says:

    Yeah, this is frustrating. The POs job is to help the rich get richer, essentially. They get on board with people who have won peer review. Until you pass that gauntlet, you're on your own it seems.. Reinforces the beneficial effects of being fortunate enough to "know how it's done."

    I don't know what the answer is, except maybe that there is more than one..

    https://michaeltomasson.wordpress.com/2015/06/26/why-failing-to-understand-evolution-makes-your-nih-reform-ideas-suck/

  • duke of neural says:

    I like kalevala's idea. Full disclosure: I have severe research ADD. If that's a thing.

  • DrugMonkey says:

    The problem with POs being human and taking into account that a person's lab is about to close is that it is highly variable. And therefore biased. Which means unfair. If they cannot do it for all, they should not do it for some.

    Much as I have needed such help in the past and may again in the future, it isn't right.

  • NewbiePO says:

    @Dr Becca, nothing to add to the discussion except I'm very sorry you had to hear that. Good luck.

  • Dr Becca says:

    @PP I know, and that's why I didn't do it.

  • qaz says:

    According to the NIH process, taking into account monetary/lab consequences is program's job, and I've been scolded at study section for trying to bring these issues up.

    But I completely agree with both Dr. Becca and DM. The variability means that the selection is being made in part on the individual program officer (some are wonderful, some not so much) and depends the ability of the PI to network with the program officer. A PO in your camp can be a lifesaver. That's why DM always says to go to the NIH and NSF booths at SFN. Make sure they know who you are. Make sure they are invested in your success. Make sure they don't want to lose you. You don't have to be rich. You just have to talk to them.

Leave a Reply