On productivity and the "unfair" grant funding game

Apr 13 2015 Published by under Careerism, NIH, NIH Careerism

There is an article up on ASBMB Today by Andrew D. Hollenbach that laments the shut-down of his research program. In The reality that dare not speak its name we learn:

It was the day after my lab manager left, forced to find a new job by a vicious funding environment that took a trusted employee and friend from me and shut down my research program.

This is terrible, I will acknowledge. I have feared this outcome for my own research program, only briefly interrupted, for my entire independent career. The wolves are always near the door and winter is most certainly coming.

Hollenbach finds this to be unfair. And that assertion triggers slightly more thought than mere sympathy and empathetic butt clenching.

I spent 20 years studying the mechanisms underlying a childhood muscle tumor. I published more than 20 articles with a lab of no more than three people at one time, intentionally kept small so I could focus on mentoring. I established a new paradigm in my field, identified viable therapeutic targets and trained five students (three of whom went to Harvard University for postdocs). I am recognized worldwide for my research.

You would think that all of that would be enough to bring in money and continue my research. But it’s not.

My immediate thought was no, no I don't think that is enough in this day and age. 20 papers in 20 years of an independent career is not a fantastic publishing rate. Of course, yes, there are going to be field and model specifics that greatly affect publishing rate. There will be differences in publishing style and venue as well...if this had been 20 CNS publications, well, this would be pretty good productivity. But a search of PubMed seems to confirm that the pursuit of the very highest Glamour publications was not the issue. I am not an expert in this guy's field of study but glancing over his publication titles and journals I get the distinct impression of a regular-old Jane/Joe type of scientist here. Many people can claim to have established new paradigms, sent trainees off to impressive-sounding postdoctoral stints (or assistant professorships) and to have identified 'viable' therapeutic targets. I say this not to belittle the guy but to point out that this is not in any way special. It is not an immediately obvious compensation for a rather underwhelming rate of publication. For a PI, that is, who asserts he's had a long-term lab manager and up to three people in his group at a time.

Hollenbach's funding hasn't been overwhelmingly generous but he's had NIH grants. RePORTER shows that he started with a component of a P20 Center grant from 2004-2009 and an R01 from 2009-2013.

Wait. What "20 years"?

Hollenbach's bio claims he was made junior faculty in 2001 and won his first Assistant Professor job in 2003. This matches up better with his funding history so I think we'd better just focus on the past 10 years to really take home a message about careerism. One senior author publication in 2003 from that junior-faculty stint and then the next one is 2007 and then three in 2008. So far, so good. Pretty understandable for the startup launch of a new laboratory.

Then we note that there is only one paper in each of 2009 and 2010. Hmmm. Things can happen, sure. Sure. Two papers in 2011 but one is a middle authorship. One more publication in each of 2012, 2014 and 2015 (to date). The R01 grant lists 7 pubs as supported but two of those were published before the grant was awarded and one was published 9 months into the first funding interval. So 5 pubs supported by the R01 in this second phase. And an average as a faculty member that runs just under a publication per year.

Lord knows I haven't hit an overwhelming publication output rate across my entire career. I understand slowdowns. These are going to happen now and again. And for certain sure there are going to be chosen model systems that generate publishable datasets more slowly than others.

But.

But.....

One paper per year, sustained across 10 years, is not the kind of productivity rate that people view as normal and average and unremarkable. Particularly when it comes to grant review at the NIH level.

I would be very surprised if the grant applications this PI has submitted did not receive a few comments questioning his publication output.

Look at my picture, and you will not see a failure. You will see someone who worked hard, excelled at what he did, held true to himself and maintained his integrity. However, you also will see someone whose work was brought to a halt by an unfair system.

Something else occurs to me. The R01 was funded up to March 2013. So this presumably means that this recent dismissal of the long-term lab manager comes after a substantial interval of grant submission deadlines? I do wonder how many grant applications the guy submitted and what the outcomes were. This would seem highly pertinent to the "unfair system" comment. You know my attitude, Dear Reader. If one is supported on a single grant, bets the farm on a competing continuation hitting right on schedule and is disappointed...this is not evidence of the system being unfair. If a PI is unfunded and submits a grant, waits for the reviews, skips a round, submits the revision, waits for the reviews, skips another round, writes a new proposal..... well, THIS IS NOT ENOUGH! This is not trying. And if you are not trying, you have no right to talk about the "unfair system" as it applies to your specific outcome.

I close, as I often do, with career advice. Don't do this people. Don't let yourself publish on the lower bound on what is considered an acceptable rate for your field, approaches, models and, most importantly, funding agency's review panels.

PS: This particular assertion regarding what surely must be necessary to survive as a grant-funded is grotesquely inaccurate.

Some may say that I did not do enough. Maybe I didn’t. I could have been a slave-driving mentor to get more publications in journals with higher impact factors. I could have worked 80-hour weeks, ignoring my family and friends. I could have given in to unfettered ambition, rolling over anyone who got in my way.

98 responses so far

  • Dave says:

    He must have an average IF of around 3, just by looking quickly at the type of journals involved. He peaked early during his post-doc (I'm assuming) with an EMBOJ pub, but has never reached that level as an independent investigator. Not even close. I wouldn't expect to be competitive in his field (cancer/development) with his record.

    He is exactly the type of investigator that "The Cull" is targeting.

  • Dave says:

    ......and just to add that this does not take away from the good that I'm sure he has done for his students. It seems like he genuinely cares. But unfortunately his success wont be measured by how many students he has sent to Harvard. Perhaps his biggest mistake was assuming that anyone would give him credit for that.

  • Jim Woodgett says:

    Wonder if this is what ASBMB is putting forward to define McKnight's #Riffraff? Hard to defend this history overall and there are many examples of labs closing with far stronger records. The protagonist seems to confuse privilege and entitlement. There are lots of sad tales but you can hear the study section evaluations. It is also true that you need funding to get funding but eeking out on a single grant is tantamount to Russian Roulette and once lost, it's much more difficult to keep head above water. Training is a secondary objective. The primary is always to move new science forward.

    Hope he finds a rewarding new career, but seems he was given ample opportunities over the past decade or so.

  • potnia theron says:

    Unfair? Life is fucking unfair. Quit whining. You got an unfair set of genes from your parents, compared to others.

  • Dave says:

    Hope he finds a rewarding new career, but seems he was given ample opportunities over the past decade or so

    He is 'tenured' by the looks of it, but I am intimately familiar with these types of institutions and I would bet that his tenure is far from secure.

  • Dave says:

    Unfair? Life is fucking unfair. Quit whining. You got an unfair set of genes from your parents, compared to others

    Tad harsh I think. All of us are only a breath away from being in the same situation. His reaction is perfectly understandable and there is no need to throw the boot in when he is down.

  • jmz4gtu says:

    I know what you're saying. The tone seems a bit overwrought, but I can't imagine how devastating this must be.

    Whether or not he put in the maximum effort he could to keep his lab open, he wasn't some lackluster idiot monkeying around in a lab. He was productive, and looking through his publications, also collaborative and attempted to be innovative. However, I think, rather than blaming the system, the fact is that he was unable attain funding because he is unexceptional, and right now, when the funding lines are hovering around 10%, you have to be pretty exceptional to keep your lab going. You have to be exceptionally well-connected, exceptionally good at science, in an exceptionally unique niche, etc.

    It sounds elitist, I know, but what else do we think is going to happen when we have too many researchers and not enough money? Does anyone think his predicament is indicative of systemic flaws. Or is it just that there's not enough money overall to fund the top 50% of scientists and he didn't make the cut?

    A related question he raises is the issues of priorities. He does work on a ARMS, a predominately childhood cancer. Should his work take precedence over more basic research because it has more easily foreseeable medical applications?

  • jmz4gtu says:

    "Does anyone think his predicament is indicative of systemic flaws. "
    -I hasten to add that I know there *are* systemic flaws, just that his particular case doesn't seem to be connected to them.

  • Jim Woodgett says:

    @Dave Tenure ain't what it used to be (says the guy who has never been tenured).

  • Philapodia says:

    PIs like Dr. Hollenbach who lose their funding are just the types who are attracted to go into administration (at least at my Uni). It's seen as a way to keep working with more pay but getting out of the funding chum. Of course this increases the ranks of administration (center directors/assistant deans, associate deans, dean-pro-temp, etc.) and requires active researchers to keep submitting grants to bring IDCs to keep the whole enterprise afloat. Not the most sustainable way to do business.

  • What DoucheMonkey left out is that this dude actually compared his grant funding plight to that of Oscar Wilde being criminally prosecuted for being gay. This guy is a fucken asshole, and fucke him.

    And BTW, this is supposed to be the ASBMB's poster child for the "viciously unfair system" who is supposed to garner sympathy?????

  • UCProf says:

    I see that he is not up to snuff in the R01 game, but don't you think it's a waste to have this guy just give up. He's not alone, I know several people in similar situations. There must be hundreds of people like this across the country.

    NIH has invested years in training these people, shouldn't they have some way to still contribute?

    Maybe an R03 program for midcareer people out of money is a better use of resources than an emeritus grant for an end-of-career scientis?

  • neuropop says:

    @Hollenbach: "I could have given in to unfettered ambition, rolling over anyone who got in my way." What does unfettered ambition and rolling over people have to do with writing good grants?

  • DrugMonkey says:

    Neuropop- I assume he thinks that these are necessary traits of anyone who is funded.

  • mH says:

    obviously he played the game wrong, but his mistake was being small, not unproductive. I worked near glam/BSD labs that put out less per trainee (for probably 5x the cost) of this guy. bet they don't hear "unproductive" much.

  • qaz says:

    The real question is whether we can support small town grocers like this. There was a time when we felt they contributed to the scientific enterprise. I find it extremely sad that there is no longer a place for such as he in the hypercompetitive thunderdome of NIH. Understandable, but sad. Does it really have to be this way?

  • Dave says:

    @neuropop: it plays in to the whole nonsense that to be successful in science you must be a lonely, miserable, nomadic workaholic arsehole.

    He definitely has that twisted.

  • drugmonkey says:

    don't you think it's a waste to have this guy just give up. He's not alone, I know several people in similar situations. There must be hundreds of people like this across the country.

    Yeah. I do. I think putting money into endeavors like this is way better than a lot of things we spend our public funds on as a society. Like bombing the hell out of people in other countries. I would like a lot less of that.

    But this guy writes about how unfair the funding system is, not about how it would be great if there were more money to distribute .

  • mH says:

    obviously small-town grocers are a loser from the med center point of view. the question is from the NIH's point of view is there any benefit to funding either three 5-person labs or one 15-person lab? Three arguments for the former are diminishing returns for increased size, intellectual diversity, and a slightly more sane system-wide PI/trainee ratio. Arguments for the latter are increased PI prestige, which accrues supplemental funding and contributes to institutional ego, reducing grant funds spent on PI salaries, and in some fields maybe there are economies of scale.

    I have to say, though, I doubt economies of scale happen. Most overfunded BSD labs I've seen bleed money on ridiculous overpriced reagents, duplicated infrastucture in the name of empire building...lazy shit like that.

  • physioprof says:

    Maybe an R03 program for midcareer people out of money is a better use of resources than an emeritus grant for an end-of-career scientis?

    This exists de facto, at least in some study sections. We have explicitly discussed R03s as ways for people with productivity lapses to try to get back in the game, and given good scores on that basis if the proposed experiments make sense.

  • Anonymous says:

    @Dave: "But unfortunately his success wont be measured by how many students he has sent to Harvard."

    I disagree. Maybe that doesn't amount to a hill of beans in terms of getting more grant money, but the successes of his students, to a certain degree, will always be his successes as well. If you don't understand that, then you don't know what it is to be a mentor.

  • drugmonkey says:

    We have explicitly discussed R03s as ways for people with productivity lapses to try to get back in the game,

    Softie

  • drugmonkey says:

    Anonymous- what's the end game there on creating new PHDs?

  • Dave says:

    what's the end game there on creating new PHDs

    Real estate agents with PhDs FTW!!!

  • mH says:

    Consultancies and Google like PhDs...they're like smart MBAs or something. Federal funds allocated for biomedical research are super well spent making those fuckers happy.

  • GAATTC says:

    "Don't let yourself publish on the lower bound on what is considered an acceptable rate for your field, approaches, models and, most importantly, funding agency's review panels."

    True dat.

  • damit says:

    I feel sorry for this guy, a little.
    He thought his institution cared about the touchy feely hopes we all have.

    But you know what?

    You want to do science, you'd better deliver.

    Personally, I can't believe he stayed funded as long as he did.

  • Morgan Price says:

    I think death panels for research groups might be a good idea (especially if run by DrugMonkey!) but any mention of impact factors should be banned.

  • drugmonkey says:

    I am far too soft hearted for that

  • Anonymous says:

    @DM: "Anonymous- what's the end game there on creating new PHDs?"

    Seriously, that's what you want to talk about re: a guy who had 3 students in his lab?! Yeah, what a bastard, exploiting all of those people for so many years. Clearly what we need to do is rid ourselves of any PIs that actually give a shit about their students and leave the field full of those who prioritize grants / pubs / $$ above all else. That is bound to solve the oversupply of PhDs problem.

  • ProgamOfficerBob says:

    My hunch is that this appears to be an likely case of small-lab death spiral syndrome. With only 3 people in the lab and one source of funding, all the projects were likely derivative of one another, if not, part of a single multi-aim study. As the renewal for funding came up, the logical step is to increase efforts to create preliminary data for the new application. Which is great if you get funding, if not, things can spiral down rather quickly. Subsequent rejections can cause the reaction of sinking more time and funds towards generating preliminary data and revision experiments, which the PI believes will push the application towards the funding line; but in reality, is nothing more than "doubling down" on an ill fated application. As money and time burn away, starting a de novo project does not seem feasible in the eyes of the PI. Perhaps, then in a panic to save money, they let a postdoc or tech go, further decreasing the productivity. Everything rides on this renewal of this project. Ultimately, if the project is not funded, funds are gone, personnel departed, and hope is dashed. Without department bridge funds, starting a new project not feasible and the lab is closed.

  • gingerest says:

    It's sure as shit unfair to the lab manager. But yeah, only 16 of those 21 papers are first or senior author papers - that's boutique production. My productivity's higher than that, in a field where productivity is generally lower, and I'm still considered uncompetitive until you slap on the "relative to opportunity" modifiers they use here.

  • Karl says:

    You are part of the problem. You sort of realise it but not truly. Next up: casual shoulder shrug, jokey response.

  • Cynric says:

    Yeah, got to join the chorus here.

    I have essentially the same model as Prof Hollenbach: small lab of 3-4 people who I work hard to mentor well. Preference for creative/atypical approaches to questions that means my papers sometimes have a rough ride with reviewers. I am staying "true to my nature" and "keeping my integrity", but have had a couple of dips in productivity when moving institutions and am struggling to get momentum up again.

    Of course I would like my approach to be valued and funded and maintained in parallel with the flashy glamour hype, but the fact that it isn't is hardly evidence of a broken system or "a review system so flawed that a fair review is not possible". It's evidence that it is hard to sustain a career with my preferred working model - it's up to me how I respond, and whether I want to carry on with this job given the current climate.

    Right. Back to paper writing!

  • Davis Sharp says:

    It's troubling when someone loses his job. That said, the grant system is supposed to be merit based and your peers determine your merit. A 20 year record of 20 papers and five students - three of whom did ivy-league postdocs - doesn't cut the mustard in today's environment (did it ever?). Dr. Hollenbach's skills may be best suited for an R15 institution where he can run a small lab, publish infrequently, and still be valued (as long as his work doesn't require lots of animals or expensive core facilities).

    But I think the essay is a misguided effort to convince everyone that he fought the good fight. He did not grow his mom and pop lab and, when money got tight, he lost out to Wal Mart. Heartbreaking? Yes. Unfair? No.

  • becca says:

    I love that you never even consider whether it was fair... for the lab manager. Or even to kids who get cancer, when the entire NIH institute for kids is the least funded, and this guy seemed to really fail because his science was caught between obscure childhood cancer and the competitive melanoma field.

  • Ola says:

    For me, the credibility just floats out the window in the first few paragraphs, with statements such as "With only one out of 20 researchers getting funded by a seriously flawed reviewing system"

    For starters, yes a couple of institutes have paylines of 6%, but most are in the 8-15 range. Second, why the assumption that the fraction of researchers getting funded is a direct result of the review system? Reviewers are not allowed to use the F word (fundable) - the percentile paylines are not set by study sections. Maybe the total lack of understanding of how NIH actually works is what led to his downfall?

  • drugmonkey says:

    The percentage of applicants getting funded is much higher than the payline and even the per-application success rate, of course. Rockey finally came up with numbers on per-applicant success and it showed this long standing assumption to be correct.

  • dsks says:

    A lab manager for a 3-person lab seems a bit weird.

    Anyhoo, I empathize with the dude's rant, I've gone off on one from time to time myself and its quite cathartic. The trick is not to publish it, though, largely because it's usually nonsense.

    "Dr. Hollenbach's skills may be best suited for an R15 institution where he can run a small lab, publish infrequently, and still be valued (as long as his work doesn't require lots of animals or expensive core facilities)"

    Yeah, he certainly shouldn't be throwing the dummy out of the pram like this so soon. NIH AREA, NSF... That said, I think the success rate for R15s has been kicking about the 10% mark for NCI a couple years now, so it's no picnic competing for those either*.

    * Had an app triaged just this last round, actually. Haven't seen the summary statement, but I'm pretty sure it's because the system is broken, it was sent to the wrong study section, the reviewers were all incompetent and jealous of my newly found viable therapeutic target etc... Ahhh, it really is cathartic, though.

  • drugmonkey says:

    Don't forget the errors of fact that were made by the reviewers dsks.

  • neuropop says:

    I am baffled why ASBMB Today even carried this polemic. Even if he did not want to "roll over people in pursuit of his unfettered ambition", surely submitting a second, concurrent R01 a couple of years into his first, would be the most prudent thing to do? I bet this fellow doesn't treat his investments the same way as he treated his funding stream. So why is the system unfair? Too many mouths at the dwindling trough, yes. But that doesn't make the review process unfair. Besides, didn't DataHound crunch the numbers to show that most labs are a single R01 lab? So @Hollenbach did not quite get squeezed out by WalMart/GlamLab.

  • qaz says:

    Actually, @Hollenbach probably did get squeezed out by WalMart (since the US taxpayer funds a large proportion of WalMart's employee salaries - some large percentage of WalMart full time employees are on food stamps) but not by GlamLabs.

  • drugmonkey says:

    I am baffled why ASBMB Today even carried this polemic.
    as others have noted, it does have the flavor of being counterpoint to Steve McKnight's riff-raff theme.

    surely submitting a second, concurrent R01 a couple of years into his first, would be the most prudent thing to do?

    absofriggginlutely

    but he also said:

    You would think that all of that would be enough to bring in money and continue my research....Many are tired of playing a game whose rules change before you even know what the rules are....My career was decided by others... academic committees that focus on one aspect of a person’s career to determine their advancement. ...I stand in an empty lab with no money, no workers, no ability to do research — yet needing to do the research to bring in the money.

    I have to consider that some (many?) scientists really are this clueless about the world around them and think that what worked for the folks who trained them as graduate students when those PIs were early-career will work for the current cohorts. It is really difficult to look at the run of the mill Jane/Joe scientists who are 10, 15 years your senior and realize that beating their performance by a slight margin may not be good enough even to survive any more. I don't know if this is true but it sure feels true. And not everybody has their head up about this sort of thing.

  • dsks says:

    "Don't forget the errors of fact that were made by the reviewers dsks."

    There's no doubt about it. But what can one expect from an AREA special emphasis panel?Only the riffiest and the raffiest of the raff riffery could possible be serving on those.

  • drugmonkey says:

    Exactly. Reviewed by Associate Professors I've never heard of from some University, in Nebraska or some place, that I don't recognize, dsks. The System is Clearly Broken.

  • Jonathan says:

    @becca "when the entire NIH institute for kids is the least funded"

    Huh? NICHD's budget last year was $1.2 billion, making it one of the bigger ICs. And anyway, this guy was funded by NCI, which is the $5 billion gorilla.

    http://officeofbudget.od.nih.gov/pdfs/FY16/Approp%20%20History%20by%20IC%20through%20FY%202014.pdf

  • Davis Sharp says:

    "I love that you never even consider whether it was fair... for the lab manager." Becca, April 14, 2015 at 7:39 am

    Dr. Hollenbach's essay was all about him, not the lab manager other than he (Hollenbach) lost " a trusted employee and friend, " (did the LM die?).

    "Or even to kids who get cancer, when the entire NIH institute for kids is the least funded."

    (1) The NIH institute for cancer is the most funded. NCI attributed~$180M of its FY2013 funds to pediatric cancer which corresponds to approximately 15% of the entire NICHD budget for that year.
    (2) I'm not sure what impact Dr. Hollenbach's research had/has/will have on pediatric cancer. If it is promising, then others will likely fill the gap.

    Having cancer is a statistical probability. It only seems "unfair" to kids. As a former cancer patient, I do my share of weeping for these remarkable children. But that's irrelevant to the topic.

  • zb says:

    What I question about your "review" of this scientist, who has lost out in the NIH game is that you are also relying mostly on one metric (number of publications), as do those who rely on # of glamor pubs, or some numerical calculation of IF times number of pubs, or h-index, or whatever.

    The desire to quantify performance standards is understandable. I can't evaluate the substantive work of someone who studies childhood tumors (or even the cell biology of them). But I can count publications (and so can everyone else). But relying on the number of publications creates gaming impulses as well, ones that skew science just as significantly as the push for glamor publications.

    I do think all methods of evaluation fail when the selection of winners becomes tight, and that gaming and randomness and luck and bias of various sorts end up playing a larger role between the winners and the losers (when, say, you're choosing the 5% instead of the 50%).

    I don't have any good solutions to the problem (in college admissions, I think randomly choosing might actually be a good option, but not in grant funding). For grants, stability needs to be a goal as well. Maybe we can continue to train scientists who find themselves jobless at 45 or 55; maybe the job will be so desirable that people will still enter the field and continue to training, or, maybe we can rely on imported labor in the training phase, and sustain the field. Ultimately, though, I don't think Americans are going to enter research careers if the risk seems so high, especially with the relatively low income during the ramp up phase.

  • zb says:

    PS: So, the folks here are basically agreeing that the guy who lost his funding is riff-raff, and, potentially, that the screed was published at acsmb 'cause it proves the case for McNight?

    What I've noted, though, is the general reaction whenever anyone loses funding, gets laid off, doesn't get tenure is that they must have been "riff-raff", didn't play the game right, should have done something differently. It's the curse of believing that the evaluative system is working, something everyone who is still in the game hopes.

  • drugmonkey says:

    You are missing the difference between what is so and what each of us might wish is so.

  • zb says:

    But, the question is, what do each of us wish is so? I suspect that everyone will want to be judged by the metric on which they excel.

    I will note that I am in a field notorious for slow publication rates, and I can point to a not insignificant number of funded, tenured with publication rates lower than 1/year. But, it's also a field where glamor mag pubs are probably required in the mix (and, a more frequent publication rate won't make up for the lack of glamor). I don't know how all these researchers will fare in the increasingly more competitive NIH RO1 battles -- but, I guess we'll see what is.

  • drugmonkey says:

    I make it very clear that I do not like GlamourHumping approaches and favor a place for incremental advancing scientists doing solid, non-me-too work that happens to interest them.

    But we cannot have endlessly increasing numbers of scientists funded under a fixed NIH budget.

    Analyses of the actual data continue to show that mouths-at-the-trough is the only clearly actionable target. Given that we can 1) hasten retirement, 2) throttle down new production or 3) the Cull.

    The third option is the one that is most brutal but yet permits everyone to avoid responsibility for it. So we're running with that.

  • neuropop says:

    @zb "I suspect that everyone will want to be judged by the metric on which they excel."

    But those aren't the rules. @Hollenbach knew the rules and didn't play by them. If what he did is paradigm-shifting, certainly someone else would take up the threads and come up with a significant advance. There are certainly deep structural issues with the scientific funding enterprise, but "unfairness" in the way he presents is not one, I think.

  • Masked Avenger says:

    (1) The biggest hurt for this guy is the damage done to his ego. Such attention payed to his perceived failure. Get over yourself, get out there, get a job like the rest of us, and make more money for less time worked. Welcome to the real world. *crocodile tears* Unless he's a slave to his ego, he will excel outside of academia, like the rest of us.

  • Neuro-conservative says:

    DM - I think your mouths-at-the-trough analysis is spot-on. Given that, would you go back in time and change your opinions on enhancing ESI funding? Seems like we would be in a marginally better position if we maintained the marginal bias against ESI. (It's not like we were so low on funded ESI that we were truly in jeopardy of eating the seed corn - that was just over-dramatized propaganda).

  • David Russler-Germain says:

    Something my medical school classmates and I have ruminated on in the past is the concept of "being average" (in the how-do-you-stack-up-against-your-peers sense). If you surveyed people in a class of 100 students, far more than 50% would think they were "above-average". And certainly fewer than 10% of the class would think they were in the bottom decile (by any given metric of performance such as test scores, for example). The same thing applies to how scientists view themselves in relation to their competitive communities on the whole.

    [Disclaimer: I'm being intentionally biased in the above paragraph by theorizing about medical students, a group of individuals likely already selected for having an inflated sense of accomplishment]

    Beyond the common, literal miscalculation of one's own "standing" in a competitive community, there is also the contrast between "standing" and "standards". The former is a relative score or ranking (thus encompasses above average, average, below average, etc...), while the latter is a binary determination of worthiness or deservingness (i.e. someone did a good enough job or not).

    In the case of Hollenbach, it appears from his piece that he miscalculates both his "standing" (paylines are what they are, and he didn't foresee or agree with his loss of funding) and his "standards" (his vision of what a PI must do to remain competitively funded does not align with the actual expectations in the current system -- this is precisely DrugMonkey's point, as I read it).

    I find this "psychology of competition" very interesting. If blogs and Twitter can provide scientists something important other than links to cool papers, it is the improved view of the true nature of the broader competitive scientific community. As I think DrugMonkey is getting at, it's all too easy to put on the blinders, and erroneously assume one's own experience/situation is both competitively "good" as well as morally/ethically "good".

  • zb says:

    "But those aren't the rules. @Hollenbach knew the rules and didn't play by them"

    I think, though I have only glanced at the article, that he didn't really know the rules and didn't play by them. I hope the folks I know who just got RO1 renewals w/ 4 publications for the grant period know the rules that they actually have to play by (they're pretty sophisticated, so I think they do understand the rules of their field). But, they won't know, until they don't get renewed.

    I agree that there can only be a limited number of scientists reliably supported by NIH. Even if we think more should be spent, at some point, supply will exceed the budget.

    "Given that we can 1) hasten retirement, 2) throttle down new production or 3) the Cull."

    The Cull is the one that works with the current funding mechanism, in which individual grant applications are considered and evaluated and decisions are made without considering the individual (presumably, unless greybeards have too much influence, this will also hasten retirement). Maybe, ultimately, the Cull will also decrease new production by decreasing the number of students interested in pursuing research (though there, the Cull will only work to decrease entry so long as we don't replace the US students who aren't joining labs with immigrants grad students & post-docs, who will be facing a different labor market).

  • zb says:

    Oh, and I'll note that the "low" publishers I described above, with tenure/renewals, are faculty with funding, independent of their NIH funding (i.e. old-fashioned tenure).

    Will the cull selectively trim those who can't rely on non-NIH funding to weather the NIH's moving finger? Each of the methods of decreasing the number of people eligible to apply to the NIH skews the research population in different ways.

  • Lucky thus far says:

    I've tried the R03 re-boot approach, only to have it turned back b/c none of the institutes most relevant to my work *accept* R03s (or R21s). Any advice in that context? [Yes, of course, I'm also submitting R01s, NSF, and even R15s, as well as charity associations - but the particular avenue of a small-project NIH-funded reboot seems to be unavailable unless I have missed something.]

  • becca says:

    @Jonathan and David Sharp- well that is embarrassing. Let that be a lesson: do not, under any circumstances, listen intently to a scientist in their seminar complaining that their research is disproportionately underfunded, and try to discuss it intelligently years later. I knew NCI was well funded, which is more relevant anyway.

    That said, perhaps I am projecting but... I think that no matter how bitter he is about his personal career, ranting about "unfairness" could be driven by other things. Such as knowing his tech didn't deserve to be let go. Or knowing that, in a real sense, failure to get your grant funded means the scientific community doesn't care about the subject of your research (given that there's nothing in his record to suggest his methods are garbage).

    Also, as David Russler-Germain probably has realized- academia selects for the person who looks at the lowest-performing tenured profs and says "I could do better than that"... not the people who look at the superstars and say "this game isn't for me". (the ways in which that should intersect with who is already represented in the field and privilege I leave as an obvious exercise to the reader...)

  • drugmonkey says:

    Given that, would you go back in time and change your opinions on enhancing ESI funding?
    Absolutely not. It is ridiculously offensive to cull those had suffered this extended training interval so as to avoid asking those at the end of a long interval of flush funding to retire on a normal schedule or to prevent fresh bachelor's folks from entering the labor exploitation scam in the first place. [nb. Note that my support for transitioning investigator fairness objects whole heartedly to the timeline cutoffs because they have this same ridiculous generational effect]

    It's not like we were so low on funded ESI that we were truly in jeopardy of eating the seed corn

    Exactly. And persuant to my recommendation to throttle back production, I note that if we ever got in some crisis of insufficient biomedical investigators (say if we even put an entire moratorium on any PHD entrants for a few years) we really only need 4 years to train the next crop. This is nothing.

    D R-G: perhaps more important is a blindness to the sheer magnitude of the field in the race.

    The Cull is the one that works with the current funding mechanism, in which individual grant applications are considered and evaluated and decisions are made without considering the individual

    I actually use this term to refer not the vagaries of individual grant decision but more to the vagaries of which of essential equally meritorious individuals will be spit out in the course of a years-long semi-random process. This usually takes far more than one grant review and depends on a lot of chance elements of timing, local tolerance, etc.

    I see it as an uncontrolled process of decreasing the current pool of funded/once-funded investigators. I object to the hand-throwing-up attitudes of people like Rockey who deploys terms like "Darwinian" in both a fake-helpless and somewhat inaccurate defense of her own complicity. There is every reason to want this to be a controlled process with respect to all sorts of factors involving research domains, subfields, translational/basic/clinical, PI demographics, Uni geography, Noob/Old, etc, etc, etc.

  • Jonathan Badger says:

    Did anyone note that in the bio at the bottom of the article, Hollenbach is described as an author of "A Practical Guide to Writing a Ruth L. Kirschstein NRSA Grant"?

  • drugmonkey says:

    Yes. Why?

  • UCProf says:

    @lucky thus far

    I've seen people do something similar with small r01's. Propose something with an r03 like budget and time span.

    One came up like that in our study section and was scored very favorably. People recognized the pi didn't have any funding and was starting small.

  • Jonathan Badger says:

    It's just a bit ironic having somebody who had trouble getting grants literally present himself as an expert in getting grants. Medice, cura te ipsum and all that.

  • drugmonkey says:

    Well....he did say he focused on mentoring and placed his students in good postdocs. Maybe his students' NRSA hit rates were excellent?

  • Staying Anonymous says:

    So... I may be the only person commenting here who knows Andrew Hollenbach (and read his book). I want to point out something. In fact, I want to shout it: Nowhere, nowhere in his essay does he use the word unfair or even refer to fairness. I did not read that he thought his situation was unique, but rather that he was an example of someone, like many others that we're not talking about in "the cull", who was good at the 'small town grocer' approach, but was squeezed out by the current circumstances.

    So, yeah, I know him a bit, and I think I can see what he was trying to say, and it looks like he missed the mark on the tone. And yeah, I have some familiarity with his work and his intellectual approach, and I think he's solid and rigorous. And I have some familiarity with the kind of service activities he also undertook, and contributions he made (published) that don't show up in the peer-reviewed literature. He hasn't exactly stood still.

    The book on NRSAs is good, written from the perspective of his having served on the review panels for the F31/F32, and includes a lot of the things that come up in review discussions that aren't in the formal review criteria. It will help your students and post-docs write better applications, and if you've never been the mentor on an F31/F32, his book will tell you things you might not otherwise know about review of the Sponsor and Co-Sponsor's section.

    Finally: David Russler-Germain pointed out that Twitter provides some serious value in understanding the broader culture and landscape. I don't think he's on social media. The unfettered snark of Tiwtter is as essential to understanding this job as the latest output from the NIH Nexus, afaic, because you get what your peers and colleagues *really* think. I have no idea if Andrew has seen the discussions on this blog or on Twitter, but I would guess he would say his intent lay in a VERY different avenue than the "poor me", "unfair, whaaah!" interpretation most people have given to the essay.

  • drugmonkey says:

    However, you also will see someone whose work was brought to a halt by an unfair system.

    Yeah, he did say it was unfair.

  • qaz says:

    @DM: "I actually use this term to refer not the vagaries of individual grant decision but more to the vagaries of which of essential equally meritorious individuals will be spit out in the course of a years-long semi-random process."

    This has come up before, but we need to remember that the "Cull" is highly non-random, and does not affect "equally meritorious individuals". It particularly targets a certain size of lab and science that some of us think is actually important (small town grocers, non-Glamour, people who have lives outside of their science obsession, K-type reproduction (*)) and favors a certain type of lab and science that has dangers and problems (Glamour-chasing, overhyped, high-risk (r-type-reproduction *), and with increased potential for fraud and error).

    The scientific market is a replication-and-selection process, and as genetic algorithm simulations show, even very small percentage changes in probability of survival or replication can swamp entire populations extremely quickly.

    * r and K is of course a reference to the population dynamics variables, where K-type spend a lot of effort on a few descendents, and r-type is a lab with 50 postdocs, one of whom will hit C/N/S and get the faculty job, while the other 49... well, let's say they're casualties of r-type reproduction.

  • This has come up before, but we need to remember that the "Cull" is highly non-random, and does not affect "equally meritorious individuals". It particularly targets a certain size of lab and science that some of us think is actually important (small town grocers, non-Glamour, people who have lives outside of their science obsession, K-type reproduction (*)) and favors a certain type of lab and science that has dangers and problems (Glamour-chasing, overhyped, high-risk (r-type-reproduction *), and with increased potential for fraud and error).

    This is not what I'm seeing on my study section, which reviews plenty of applications from both ends of the spectrum of labs.

  • Staying Anonymous says:

    Well, heck. I had searched the page for 'fair', surprised that he would have said that, and managed to miss it, which probably just demonstrates that because I do know him slightly, I was looking for something exonerating. No, I'm not arguing that Andrew's lack of funding was a result of unfairness, but more of tight money, as you pointed out, and yeah, that undercuts the point I think he was trying to make, that the 'small town grocer' model was doomed. If he were on social media, he might have come at it differently.

  • drugmonkey says:

    qaz- I bet on the end it will be hard to show the Cull hit any particular type of PI harder than all others.

  • bacillus says:

    It is my perspective that most of us want to be discovery scientists rather than development scientists. The latter are involved in far less glamourous work (relatively) than the former, and may well be most affected by the cull. However, just about all fundamental scientific discoveries nowadays require a great deal of value added work before they become even remotely interesting for commercial development. Since this rarely leads to C/N/S papers, the BSDs simply move on to make the next big discovery rather than faffing around trying to utilize his last one. Therefore, I'm worried that the cull will leave us with too few scientists to perform the translational science the NIH keeps banging on about. This may well be nothing more that politically expedient lip service on the part of NIH, but the fact is that it will need to keep showing and growing tangible outcomes of the work it funds if it stands any chance of seeing meaningful budget increases in the years and decades ahead. Thus, although it is critical for NIH to continue to fund the best fundamental research it possibly can, it needs to be careful not to throw too many Andrew Hollenbachs' under the bus in the process. To quote Louis Pasteur "There's no such thing as basic and applied science, just applications of science"

  • neuropop says:

    @Staying Anonymous: "...that the 'small town grocer' model was doomed". I don't know how one can conclude that fact. McKnight and their ilk complain that the "riffraff" at study sections are pecking their grants to death while not respecting their awesome !!eleventy!! vertically ascending science. Most of the kvetching on this blog involves the "small town grocer" model being phased out. I think there is just equal opportunity cull.

  • MoBio says:

    @bacillus
    "It is my perspective that most of us want to be discovery scientists rather than development scientists. "

    This is an interesting notion at odds with my perspective (Note: I do much discovery-based science). Indeed it is my experience that discovery-based research fares poorly in NIH review groups, thesis committees and so forth.

    Indeed it is standard practice in my lab to wait until we've actually published a discovery before attempting to obtain NIH-type funding for the follow-on experiments.

    I'm wondering DM if you've ever polled the readers of this esteemed blog to see if that statement is true--do most readers yearn to do discovery-based research?

  • drugmonkey says:

    I suspect trying to sneer about "development" vs "discovery" science will shed more heat than light, MoBio. I did ask people some time ago whether they describe what they do as "discovery" or "demonstration" in a semi related line of thought.

    I find the "discovery" arguments to be frequently the province of false novelty like that Zuker opto paper that demonstrated the subfornical control of drinking for the millionth time

  • jmz4gtu says:

    This is purely anecdotal, but I've noticed a generational difference in attitudes about this, between my bosses' demographic (50-65, big labs) and junior faculty (less than 10 years). All of my bosses have had an almost fetishistic reverence for technical novelty that eclipses the actual significance of the finding. The significance is often trivial, since novel techniques and approaches generally start off on what are essentially positive controls before being utilized for more interesting questions. Because if you don't do this, you'll get scooped on the technique.
    Younger faculty tend to have less adulation for technical novelty. I wonder if this is a side effect of being closer to the bench, and/or a growing disenchantment with flashy Nature Methods papers that don't end up working any better than the conventional alternatives.

    More generally, I agree with Bacillus. Novelty is hugely overrated by journals and NIH panels alike. And again, I note a generational divide.

  • physioprof says:

    I know you hate glamdouches like Zuker, but that paper was an important and exciting advance in demonstrating the precise cellular identity of the neurons controlling drinking.

  • David Russler-Germain says:

    @Staying Anonymous (and others), just clarifying that I am on social media:

    https://twitter.com/dgermain21

  • MoBio says:

    @DM: no 'sneer' intended.

  • mH says:

    I thought the breakthrough Zuker's lab made is that the SFO has recently moved to the hypothalamus. That's the kind of novel novelty new thing these editors are looking for.

  • drugmonkey says:

    I know you hate glamdouches like Zuker, but that paper was an important and exciting advance in demonstrating the precise cellular identity of the neurons controlling drinking.

    no, it really wasn't "important and exciting". in the fucking least....you've been drinking the flavrade again, haven't you?

  • physioprof says:

    Anyway, you should get Brembs and Eisen on that case right away! Cause "hype" is as bad as fraud!!!!!

  • drugmonkey says:

    Mos def!

  • physioprof says:

    Fucken gyrocopter dude is lucky they didn't blast him out of the motherfucken sky!

  • drugmonkey says:

    I think you are losing it my friend

  • physioprof says:

    NO! I am gaining it!

  • drugmonkey says:

    Loon

  • Cynric says:

    Therefore, I'm worried that the cull will leave us with too few scientists to perform the translational science the NIH keeps banging on about.

    I have a different concern. I think a lot of the big breakthrough discoveries come from the small, unregarded labs, and are seized upon and popularised by the Glamourhounds (e.g. discovery versus utilisation of GFP).

    I remember going to a talk by Philip Cohen about the discovery of kinases, and how hard it was to persuade anyone to take him seriously about trivial changes in the surface chemistry of proteins. 20 years later he was being schmoozed by Big Pharma. He was quite frank that he didn't think his early work would be funded nowadays. I seem to hear that story a lot from now-eminent scientists, and worry that the loss of "low prestige" labs will also mean the loss of this intellectual seed corn.

  • Grumble says:

    "Maybe, ultimately, the Cull will also decrease new production by decreasing the number of students interested in pursuing research "

    As long as most biomedical PhD students get *paid* to get a degree, there isn't going to be a substantial reduction in this number. For some segment of the group defined as "smart undergraduates," going on to get an all-expenses-paid PhD is going to make sense, even if the prospects of continuing on to become a faculty researcher are slim.

  • David Russler-Germain says:

    @Grumble: I've thought long and hard about PhD student compensation, and I understand the logic behind many possible approaches. What is the ideal system in your view? Do you think PhD students should pay a tuition, and received no stipend? Perhaps pay no tuition, but receive no stipend either? I agree with you in that as long as enough people in their 20's view the compensation during PhD training to be acceptable (regardless of their academic job prospects), then there will be thorough demand for PhD training slots. In turn, it will be difficult to reduce the number of people receiving PhDs simply "for the good of the scientific community" (if you agree with that). The bottomline is that how we compensate people, in any field, directly impacts the supply and demand in that field.

    That said, we need to consider the consequences of, for example, eliminating PhD stipends (as it appears you might be suggesting). Does anyone really believe that diversity (in many forms and fashions) would be hurt by doing this? I know people for whom their $28K annual graduate stipend provides them enough income to send home extra money to their parents who earn minimum wage. This was expected of them after graduating college. Cross these and many other people off the list of PhD students once they stop receiving stipends.

    I also think it's important to consider the style of "education" one receives during PhD training (in its current form). In contrast to undergraduate education, where you are really paying (a large sum, albeit) for coursework/teaching as well as "collegiate enrichment" opportunities (so-to-speak), graduate school is much, much closer to working a job than being "in school" compared to all other forms of being "in school". Yes, you take classes (often not very many, and not throughout your degree), and yes you receive training and mentorship from your PI, but I think it's disingenuous for anyone to argue that the average PhD student (particularly in the biological sciences, which I am most familiar with) doesn't experience at least an 80/20 "pseudo-employment"/"educational" breakdown of their time in graduate school. Thus, in my view, the consideration to eliminate graduate stipends (which equates to <$10/hour given my workload, for example) is off the table.

    Rather, a (potentially) better way to reduce the number of PhD students as well as to hold the ones that remain to higher standards might be to pay them more (with the money saved by reducing slots). That isn't to suggest a 50% cut in enrollment and a doubling of pay, but maybe a 20% cut in enrollment etc... This may open the door to PhD training to a broader portion of society, as well as provided a "boost to productivity" (if you view as I do that higher compensation can help improve morale and focus+incentivize workers.)

    I'm not sure if opening the pandoras box of PhD training and compensation was intended, but I thought I'd comment on the topic since it had been brought up.

  • mH says:

    I don't think raising PhD and postdoc salaries is the best way to dial back production, but I do think it's the only way that is possible given structural incentives for PIs and universities. The NIH is in a position to do it and it sounds like they might, at least for PDs.

    I also think that as research career prospects continue to dim, we are offering trainees less and less of the kind of opportunity that we advertise and they are looking for. Increased pay is a fair-ish partial tradeoff for that.

    I have negative sympathy for the "PI autonomy" and "but less science will happen" responses to this. I have some sympathy for "more advantage to BSDs" criticism.

  • Grumble says:

    @DR-G: I'm not at all suggesting that we do away with grad student stipends. From my point of view as a faculty, I love the cheap labor.

    In reality, half the solution you suggested is already being implemented: reducing the number of PhD training slots. My college, for instance, has done this pretty aggressively because they realized that they had to match the number of admitted students to the number of faculty with funds to pay student stipends. So the general lack of increase in NIH budget is inevitably going to shrink the number of new PhDs generated by the system.

    The other half of your proposal, raising student stipends, is also happening, albeit more slowly.

  • Anonymous says:

    I'm guessing 0/5 of this person's trainees went on to become independent faculty/PIs, or else he would have said so (given what else he said). He said 3/5 went on to become Harvard postdocs. Please, those are a dime a dozen. A Harvard postdoc is the definition of the cheap labor / exploitation problem.

    Should academia really be funding mentors who are training no future PIs? They are only training the future cheap labor for other PIs?

    I think there should be a place for this guy and his lab manager/staff scientist. No trainees. But still, an independent position as a scientist in society. TONS AND TONS of smart young scientists would kill for such a job, and they could do amazing things with it. Why do these jobs not exist? I honestly don't understand.

  • […] this week, this sentiment ran thru parts of science Twitter and Ted’s blog comments following his kicking the academic ‘nads of one Andrew Hollenbach after he had the misfortune of posting his story about having to close his […]

  • […] his lab: The reality that dare not speak its name – Andrew D. Hollenbach A harsh response: On productivity and the “unfair” grant funding game – […]

  • […] Last week Dr.Hollenbach wrote a blog describing how he’s closing his lab. If you look at him, he is the stereotypical image that pops up when you use google image with the word scientist. Though they’ve added some females and a splattering of color since Dr.Isis last did this. The stereotypical guy in science didn’t make it. If you read Namaste, Ish at Drugmonkey’s or even DrugMonkeys own take on it, Dr.Hollenbach failed because he wasn’t productive enough. […]

Leave a Reply