Credit where due: McKnight manages to get one right

Jul 08 2015 Published by under Conduct of Science

Steven McKnight's recent President's Message at ASBMB Today focuses on the tyranny of the hypothesis-test when it comes to grant evaluation.

I lament that, as presently constructed, the NIH system of funding science is locked into the straight-jacket of hypothesis-driven research. It is understandable that things have evolved in this manner. In times of tight funding, grant reviewers find it easier to evaluate hypothesis-driven research plans than blue-sky proposals. The manner in which the system has evolved has forced scientists to perform contractlike research that grant reviewers judge to be highly likely to succeed. In financially difficult times, more risky scientific endeavors with no safely charted pathway to success often get squeezed out.

.... But how should we describe the riskier blue-sky research that our granting agencies tend not to favor?

I agree. All science starts with observation. And most science, even a lot of that alleged to be hypothesis testing or lending "mechanistic insight" really boils down to observation.

If we do this, then that occurs.

Science never strays very far from poking something with a stick to see what happens.

The weird part is that McKnight doesn't bring this back to his "fund people not projects mantra". Amazing!

No, he actually has a constructive fix to accomplish his goals on this one.

Were it up to me, and it is clearly not, I would demand that NIH grant applications start with the description of a unique phenomenon. When I say unique, I mean unique to the applicant. The phenomenon may have come from the prior research of the applicant. Alternatively, the phenomenon may have come from the applicant’s unique observation of nature, medicine or the expansive literature.

This is great. A fix that applies to the project-focused granting system that we have. Fair for everyone.

Kudos dude.

17 responses so far

  • MorganPhD says:

    It's a great idea and would be a really cool way to start grant applications. But I can't shake the feeling it's really just more justification for the "people, not projects" hustle.

    Especially with the line,
    "Alternatively, the phenomenon may have come from the applicant’s unique observation of nature, medicine or the expansive literature."

    Perhaps I'm horribly jaded, but the natural response from reviewers will probably be "...but are you qualified to perform the work". Then they go back to track record or the most recent GlamourPubs and we live in a world where a cancer biologist gets $3 million to study squirrel hibernation because he was eminently qualified to have a "unique observation". And someone with a cool idea and less stellar record is passed over again.

    It's difficult for me to reconcile this McKnight post with the previous ones and not be subjective.

  • drugmonkey says:

    Oh I agree entirely that it is very likely that McKnight sees this as more evidence and rationale for people-not-projects.

    But I'm happy that he didn't try to close that circle and that as written it is a fix-the-project-based-approach argument.

  • MorganPhD says:

    I'm going to go back to earn my apparently unfair postdoc salary in support of the unique perspective of my BSD boss.

  • Dave says:

    Wait, McKnight is still an enemy, right?

  • Philapodia says:

    Perhaps I'm not getting what McKnight is on about here:

    "Were it up to me, and it is clearly not, I would demand that NIH grant applications start with the description of a unique phenomenon. When I say unique, I mean unique to the applicant. The phenomenon may have come from the prior research of the applicant. Alternatively, the phenomenon may have come from the applicant’s unique observation of nature, medicine or the expansive literature."

    I don't understand what he means by a unique phenomenon and how what he describes is different than what's in the good ol' specific aims page. In the SA you describe a previous finding/novel observation and how you want to approach understanding it further. I think it's disingenuous to paint the NIH as so risk adverse they only fund "safe" research and don't allow PIs to try new things. In my opinion this is still "people, not project", just in a more subtle package. I'll give him credit for learning some diplomacy, though.

    ".... But how should we describe the riskier blue-sky research that our granting agencies tend not to favor?"

    By putting a little thought into a justification of the potential value of the research/hypothesis and how you want to go about testing it. What's wrong with having a plan, especially when everyone knows research plans are flexible?

  • physioprof says:

    In times of tight funding, grant reviewers find it easier to evaluate hypothesis-driven research plans than blue-sky proposals.

    False dichotomy. And this is just more of McKnight's class warfare on behalf of his buddies who are too fucken lazy/senile to write a decent grant with coherent proposed aims and want to get away with "LOOK AT HOW BIG MY DATASET/FANCY MY KNOCK-IN MOUSE/IMPOSSIBLE MY ELEVENTEEN-ELECTRODE RECORDING IS!" Allz he is saying is that accepted standards of grantsmanship that don't really influence what science actually gets performed are for the little people, and shouldn't get in the way of his vertically ascending buddies.

  • Ola says:

    As PP alludes to, I read Steve O's initial statement as more dissing of reviewers (a la panels filled with riff raff). It's demeaning to imply that the reason reviewers don't fund "blue sky" stuff is that it takes more effort to understand. Oooh it makes my ickle weviewer bwain all hurwty. That blue sky shittio doesn't get funded because we know it's a pile of arse!

  • Pinko Punko says:

    He's creating a straw man. It is true that I think some reviewers are reflexive about this stuff, but I also think panels do go for "this is super interesting, we have no idea how this works, here is how we are going to attack the problem. First, we will examine the relationship between x and y [SEKRIT HYPOTHESIS IN THERE], second, by analogy to z system, we will determine..[WOW, ALMOST A SERIES OF HYPOTHESES TESTED IN PARALLEL- POSSIBLE MODELS THAT EXPERIMENTS WILL DISTINGUISH BETWEEN].

    More about #riffraff because he's seemingly implying that the system as it is now wouldn't recognize this- and he's doing it with no evidence, or worse, he's dealing in anecdata.

  • Jonathan Badger says:

    I guess I don't work in a subfield where hypothesis-driven research is a tyrant. If anything, I hear complaints from the greybeards that there isn't *enough* hypothesis-driven research in the modern age and that the current model is basically to collect a lot of data and try to come up with an explanation afterwords.

  • dsks says:

    A) as others have explained, this just seems like a cynical addendum to the fund-people-not-projects dross

    B) We've been in circles about this before, but hypothesis testing isn't some government-sponsored pedantry designed to trip up and constrain the visionaries and geniuses among us. It is an approach that, by demanding a prediction before the experiment, allows for more vigorous assessment of the results obtained thereafter. One can argue Baysian vs frequentist approaches, or the merits and flaws of falsificationism, but a prediction is necessary, as is an appropriately designed statistical analysis for testing the results. Not because this will set the answer in deductive stone - that's impossible in this racket - but simply because this adds considerably greater strength to the likelihood the answer is meaningful.

  • Philapodia says:

    Steve McKnight posted another essay at ASBMB and this phrase struck me:

    "A worthy project should either question our existing assumptions or propose an uncharted pathway directed toward an unexplained biological phenomenon."

    Is this really all that is worthy for funding? Is fleshing out understanding a system not a worthy project to fund? Using this mindset, you could never fulfill the second part of the NIH mission "NIH’s mission is to seek fundamental knowledge about the nature and behavior of living systems and the application of that knowledge to enhance health, lengthen life, and reduce illness and disability."

    The NSF Mission statement is "To promote the progress of science; to advance the national health, prosperity, and welfare; to secure the national defense...."

    It seems that Jedi Council types in positions of power need to be reminded of the end goals of funding agencies they complain about is ACTUALLY TRY to improve life for US citizens, not following whatever glittery rainbow catches your fancy. If you can't frame your research into some sort of compelling health/improvement of the human condition context, are you really that good of a scientist?

    http://www.asbmb.org/asbmbtoday/201601/PresidentsMessage/

  • drugmonkey says:

    Speaking of "questioning assumptions", McKnight.....

  • Jonathan says:

    @philapodia - yes, that's true for federally funded research, but if you're funded by the Special Floppy Ears Foundation For Bunny Hopping Research there needn't be any payoff beyond discovering that bunnies suck compared to cats.

  • drugmonkey says:

    Hmm vertically ascending.....bunnies. Vertical ascent......bunnies. Tennis look.

  • john says:

    When we questioned the published assumptions of Dr. McKnight, he threatened revenge. So we can question existing assumptions, as long as they are not held by Professor McKnight. He should clarify this in his letter.

  • Philapodia says:

    @Johnathan

    Agreed, although he's been exclusively talking about how federal funding agencies suck so we have to assume he's not talking about the BunnyHopping Foundation...

  • jmz4 says:

    "A worthy project should either question our existing assumptions or propose an uncharted pathway directed toward an unexplained biological phenomenon."
    -Ha, spoken like true PI. Plan and execute, obviously everything will go exactly as my brilliant brain has predicted it...

    Ironically, all the research that his RNA granule work is predicated on would be deemed "unworthy" by him. Why spend all that time and money tracking down what those little RNAs do in flys and worms, that's not directed towards unexplained biological phenomena?

    You can only call something "unexplained" relative to a threshold of certainty. The more certainty we build through incremental work, the more it highlights the interesting unexplored aspects of biology. Arguing that one is more important than the other, or more deserving of funding, is idiocy.

Leave a Reply