Repost: Cha, Cha, Cha and Grant Review

Dec 26 2009 Published by under Careerism, Conduct of Science, NIH

This originally went up August 13, 2007.


Writedit notes:

Science has published an elegant posthumous article by Daniel E. Koshland, Jr. entitled The Cha-Cha-Cha Theory of Scientific Discovery ... representing the 3 categories of discovery: Charge, Challenge, and Chance. In brief:
"'Charge' discoveries solve problems that are quite obvious ... 'Challenge' discoveries are a response to an accumulation of facts or concepts that are unexplained by or incongruous with scientific theories of the time ... 'Chance' discoveries are those that are often called serendipitous and which Louis Pasteur felt favored 'the prepared mind.'"

I want to go a little beyond writedit's point so I'll quote more extensively from the article

"Charge" discoveries solve problems that are quite obvious--cure heart disease, understand the movement of stars in the sky--but in which the way to solve the problem is not so clear. In these, the scientist is called on, as Nobel laureate Albert Szent-Györgyi put it, "to see what everyone else has seen and think what no one else has thought before." Thus, the movement of stars in the sky and the fall of an apple from a tree were apparent to everyone, but Isaac Newton came up with the concept of gravity to explain it all in one great theory.

This is the wheelhouse of the NIH grant review game. Most applications identify problems that are understandable and have obvious importance. Then the applicants proceed to attempt to convince reviewers that they have a new brilliant way to solve the problem which is practically infallible. So far, so good, although we might debate the merits of needing a "practically infallible" approach to receive a good score.

"Challenge" discoveries are a response to an accumulation of facts or concepts that are unexplained by or incongruous with scientific theories of the time. The discoverer perceives that a new concept or a new theory is required to pull all the phenomena into one coherent whole. Sometimes the discoverer sees the anomalies and also provides the solution. Sometimes many people perceive the anomalies, but they wait for the discoverer to provide a new concept. Those individuals, whom we might call "uncoverers," contribute greatly to science, but it is the individual who proposes the idea explaining all of the anomalies who deserves to be called a discoverer.

This one is a little tricker to grasp, the author identifies Watson and Crick's "base pairing in the DNA double helix" as one exemplar. These applications don't tend to do so well in grant review. Mostly because the applicants propose critical experiments to "pull the phenomena into one coherent whole" and the skeptics go mad. First the theoretical position is attacked. Second, the assumption that the critical experiments can pull phenomena into a coherent whole is attacked. Finally, the reviewers come up with endless different experiments that he or she believes will be better than the ones listed. So the "Challenge" grant tends to suffer. I think a related point relevant to grant review is that most times we only find out in the doing. Empirical solutions to theoretical problems. Too much of the time grant review focuses on predicting empirical outcome ("not the right experiments to prove the point", "theory not right") rather than deciding first if the phenomena are important, if the application is a good approach to an empirical solution (rather than a guarantee), whether the PI can conduct a reasonable empirical program (i.e., flexible changes based on outcome), etc.

"Chance" discoveries are those that are often called serendipitous and which Louis Pasteur felt favored "the prepared mind." In this category are the instances of a chance event that the ready mind recognizes as important and then explains to other scientists. This category not only would include Pasteur's discovery of optical activity (D and L isomers), but also W. C. Roentgen's x-rays and Roy Plunkett's Teflon. These scientists saw what no one else had seen or reported and were able to realize its importance.

The NIH grant process does not do at all well with this. The main problem is the obsession with "hypothesis testing" and the universal critique called "it's a fishing expedition". True, science needs hypothesis testing and much effort can be wasted if a plan is not focused. But somehow along the way scientific culture has forgotten that all science starts with observation. Did I emphasize that enough? OBSERVATION! As in "Hey, here's something cool about the natural world. Let me see if I can figure out if it is really true, how it works and what that might mean for understanding other cool stuff". Really, isn't this a sufficient description of the "scientific process"? 😛 So sometimes an application proposes some "let's just kinda see what happens" experiments which draws the ire of reviewers. "Where's the hypothesis". "It's a fishing expedition" they cry. I understand the point. I certainly see applications in which the prior published work of the PI and/or the sub-field suggests that such criticism is warranted and necessary. Ones in which "let's just see what happens" seems to be the entire research program. However I would submit that when the track record suggests that the PI knows how to test hypotheses then perhaps a little credit should be extended for the one or two experiments that seem like discovery efforts.
Finally, I note that the author fits the Ten Commandments, the Magna Carta and the Bill of Rights into the Cha, Cha, Cha framework. So it must be a valid heuristic...

33 responses so far

  • sandokan says:

    REVIEWERS Want To Cha Cha
    (ESIs Want To Twist)
    Gather 'round me all you dancers while I sing this song
    It's about a dancing couple who never could get along
    Reviewer loved to Cha Cha all the time
    ESI loved to dance around him her own way
    Darling, won't you dance like the dancers do?
    And she'd say, darling, I can dance but not follow you
    He just wants to Cha Cha
    She just wants to Twist
    He just wants to Cha Cha
    Like this!
    Cha-cha-cha!
    They went to CSR Workshops to learn a new thing
    The Rhumba, the Fox Trot, the Tango, the Triple Swing
    But both were so eager to do their own way
    I'm sure you could guess what steps they both wanted to
    Darling, can't we dance in a civil way?
    And she'd say, darling, all those steps seem a bit passe
    He just wants to Cha Cha
    She just wants to Twist
    He just wants to Cha Cha
    Like this!
    Cha-cha-cha!
    For a personalized demonstration on how to cha cha cha, go to
    LINK : http://bblyrics.com/cha_cha.html

  • carla says:

    Thanks for the post!. The “cha cha” dance appears to be well-grounded, even on skates. Incorporating beats of twist might add singularity to the dance as a whole.
    Who knows ?. Always look at the bright and novel side of life.

  • qaz says:

    The funny thing about the whole "fishing expedition" criticism is the number of grant reviewers who think "fishing in good waters" is a good thing. At the last study section I sat on, one of the reviewers (not me) used the term "hypothesis generation", which brought a smile to the entire group (and a good score to the proposal).
    While it is true that chance discoveries do not do well before they happen, following up on chance discoveries do seem to do very well. As in: "we saw this. It's real. We want to figure out what it means."

  • aladin says:

    @qaz,
    Also "there's good fishing in trouble waters".
    I don't know if the "hypothesis generation" has a chance to occur in "stagnant waters", rich in "nutrients" yes, eutrophic though.

  • DrugMonkey says:

    Do listen to that 'hypothesis generating' stuff. I have adopted that myself as cover for the more exploratory expts and Aims...Can't guarantee it works but is does tend to defang the discussion IME

  • So don't be a fucking moron. Write a "charge" grant to satisfy the small-minded douchebags on the study section. Get the motherfucker funded. And then spend the money doing the "challenge" and "chance" studies that move science forward.

  • niewiap says:

    I'm with CPP on this one. Stop complaining and play their game. They call the shots, so pretend like you are as mediocre and short-sighted as they are and then take the money and run with it to whatever Cha you want. CPP, your cynicism is balm to my soul LOL!

  • anne says:

    @niewiap,
    I was going to say the same: CPP, your practical approach seems to be the most effective way to survive but I restrained myself. It is sad, however, to have to do it that way.
    Hopefully, those successfully applying that practical strategy will come up in 2-3 years with results that prove that their pretended mediocrity was just a survival strategy. So that we can start dancing cha cha cha from the beginning (writing the proposal) to the end (producing real science).

  • It is sad, however, to have to do it that way.

    Why? Who gives a shit? Proposing crap designed to be fundable and then doing what you know is more interesting has been the grantsmanship game since forever.

  • Dr. Feelgood says:

    Two things:
    I learned an obvious trick from a creative scientist with a long and illustrious career. When I asked him how I was supposed to write up grant renewal when the experiments I conducted were successful but very different from my actual proposal from 5 years ago, he said, "Declare victory, and move on!"
    That is what I have been doing ever since. Just say tadah!!...and damn the prior commitment.
    The other....is why has descriptive science (or fishing), become such a pejorative term?

  • anne says:

    CPP,
    I know that you give a damn !. I understand your point but I hope that we move up further on that "stupid grantmanship". After more than 2 years discussing issues to change things, perhaps no for the best, but at minimum "for a little bit better", I can't give up thinking that we'll succeed.

  • I understand your point but I hope that we move up further on that "stupid grantmanship".

    Like I said, who gives a shit? So long as there are grants, there will be grantsmanship. The exact form that the grantsmanship takes is meaningless.
    All that matters is being aware of it and crafting your grants accordingly. Spending time and effort agitating for a different--but equally meaningless--form of grantsmanship is a waste of time.

  • niewiap says:

    Thus spake Comrade PhysioProf! Harken and eternal bliss in NIH-land shall be yours.

  • whimple says:

    All that matters is being aware of it and crafting your grants accordingly. Spending time and effort agitating for a different--but equally meaningless--form of grantsmanship is a waste of time.
    I disagree. The NIH granting system is like the standardized testing system in public school. Students practice getting good at the test which warps the entire learning process. So it is with grants. You might say, "just get the cash, then do the creative work", but it doesn't work that way. The work you wind up doing is always shaped by the necessity of getting the preliminary data for the next grant. People who stop agitating for improvement become part of the problem.
    My pet idea for "improving" grant review: There should be a "uniqueness" criterion. If we don't fund *this specific grant*, will something new be lost to science, or will someone else just come along and do these kind of experiments anyway? I believe there is too much emphasis today on more-of-the-same. The current "innovation" criterion just isn't cutting it in my opinion.

  • If we don't fund *this specific grant*, will something new be lost to science, or will someone else just come along and do these kind of experiments anyway?

    If that were the sine qua non for funding a grant, no grant would ever be funded. Because of the nature of the scientific enterprise, *nothing* ever is discovered that wouldn't be discovered anyway by someone else, and probably pretty fucking soon.

  • DrugMonkey says:

    and probably pretty fucking soon.
    Not always homes. I'm with whimple on this one. Lots of stuff that doesn't get done for a long time because everyone is funding me-too grants. IMNSHO, of course.
    In drug abuse we can point to an overwhelming focus on cocain and heroin that was in place during the 80s and 90s. We can also point to a focus on the primary rewarding properties of drugs that held sway from the 70s on into the 90s.
    Since we were recently discussing HM, we can point in the memory fields to an unfortunate over-focus on brain region = behavioral measure = cognitive construct rather than an appreciation of circuitry. (Of course this one has cycled a few times over the past century+)

  • S. Rivlin says:

    All that matters is being aware of it and crafting your grants accordingly. Spending time and effort agitating for a different--but equally meaningless--form of grantsmanship is a waste of time.
    It is not surprising that those who have managed to take advantage of the system will resist any change offer to improve it, even when it is clear that it is not working for the majority the system was intended for.

  • whimple says:

    Because of the nature of the scientific enterprise, *nothing* ever is discovered that wouldn't be discovered anyway by someone else, and probably pretty fucking soon.
    If you really believe that, then why are you doing science? What's the point of doing work that someone else will just do next week anyway if you don't do it?

  • If you really believe that, then why are you doing science? What's the point of doing work that someone else will just do next week anyway if you don't do it?

    Because if somebody's gonna have the fun of doing it, it may as well be me. Do you really need to think of yourself as some sort of Leonardo Da Vinci unique genius in order to get your ass into the lab every day? If so, I feel sorry for you, because you are pretty much guaranteed to end up disappointed.

  • S. Rivlin says:

    "Because of the nature of the scientific enterprise, *nothing* ever is discovered that wouldn't be discovered anyway by someone else, and probably pretty fucking soon."
    This attitude about scientific research is a direct outcome of the lack of requirement for innovation and novelty in NIH-funded grant applications and in the rush to publish boring, no-news, no-novelty, papers.
    See also: Not Much Novel Under the Sun, R.B. Schonberger and S.H. Rosenbaum, Science 326 1480-81 (2009).

  • This attitude about scientific research is a direct outcome of the lack of requirement for innovation and novelty in NIH-funded grant applications and in the rush to publish boring, no-news, no-novelty, papers.

    BZZZZTT!!!! Completely 180 degrees opposite totally fucking wrong as usual, Shitlin.
    It is a consequence of the fact that scientific discoveries are constrained by the actual nature of objective motherfucking reality.

  • S. Rivlin says:

    "It is a consequence of the fact that scientific discoveries are constrained by the actual nature of objective motherfucking reality."
    The reality is that the system you are so fond of breeds technicians, not scientists. The lack of novelty and innovation in science today is due to the way by which you train your own posdocs and doctoral students, where their ability to master the different techniques in your lab is the most important part of their training such that when they are on their own they only know how to do CPP-type research, dogmatic and boring. You and your ilk are responsible for the decline in scientific innovation and novelty.

  • The lack of novelty and innovation in science today

    Cocklington, where do you get the idea that science today lacks novelty and innovation compared to the past?
    Oh, right. The same place you get all the senile gibberish you pollute this blog with, pulled straight out of your bitter washed-up wrinkly old ass. Go bother your grandchildren and leave the adults alone.

  • whimple says:

    Do you really need to think of yourself as some sort of Leonardo Da Vinci unique genius in order to get your ass into the lab every day? If so, I feel sorry for you, because you are pretty much guaranteed to end up disappointed.
    Well, it doesn't actually take Da Vinci calibre genius to simply get out of the crowd; just a little effort. Thanks for the complement though. 🙂

  • whimple says:

    Can't even spell 'compliment'. Obviously not a lot of genius over here.

  • S. Rivlin says:

    CPProfane, you, eventually, believe that your science in innovative and novel, just because you have the NIH funding it. But, as you yourself have declared, this funding has been acheived based mostly on grantmanship, not innovation and/or novelty. The only novelty that you bring, which by now is not so novel anymore, is your profanity.

  • CPProfane, you, eventually, believe that your science in innovative and novel, just because you have the NIH funding it.

    Fuckles, the way I judge the innovativeness and novelty of my science has nothing whatsofuckingever to do with it being funded by NIH. Go ask your grandchildren to explain to you the non-funding bases on which I might make this judgment, but please be prepared to take evasive action in case they understandably try to kick you in the fucking balls.

  • Dr. Feelgood says:

    I think CPP and Riv should do one of those Bob Hope and Bing Crosby road movies.
    [singing]"....We're off on the road to Bethesda!...."
    I would definitely watch it when it goes direct to DVD.

  • S. Rivlin says:

    "the way I judge the innovativeness and novelty of my science..."
    That's the point, you fecal orifece! Its not up to you to judge the novelty of your own science. You are the last one who can objectively judge such thing, since you are always running with your head stuck in your ass, and the only thing you can say is: "This is the best odor I've ever smelled."

  • Solly, you win! I can't stop laughing long enough to argue with you any more.

  • S. Rivlin says:

    "Solly, you win! I can't stop laughing long enough to argue with you any more."
    Solly? That's an improvement!

  • DSKS says:

    "Because of the nature of the scientific enterprise, *nothing* ever is discovered that wouldn't be discovered anyway by someone else, and probably pretty fucking soon.
    If you really believe that, then why are you doing science? What's the point of doing work that someone else will just do next week anyway if you don't do it?"

    If we all stopped doing science as a result of deciding that someone else would do it, it would reveal two astonishingly negative things about ourselves. a) that we're too stupid to be doing science anyway if we can't bend out brains around the idea that if everybody quit on the grounds that somebody else would do it eventually, nothing would ever get done. b) because it would indicate that our sole interest in doing science was born of a childish search for validation and fame, rather than to take part in what is, generally, an exciting endeavour.
    Who here is in this game simply to discover something wondrous that will give them leverage in the inevitable lobbying-fest required for Nobel-dom these days, and get their obituary in the NYT? Who here is involved with research that, even in their most sublime moments of narcissistic self-reflection, they honestly think nobody else either is, or will in the near future, tackle successfully for themselves?
    That somebody else will do it if we don't might not be a falsifiable hypothesis but it's still a fairly sound induction.
    All that said, I completely agree that attempting to improve the review system is both a desirable and admirable goal. It seems to me there are problems that have potential solutions, and it makes absolute sense for colleagues to propose and debate these potential solutions.

  • DSKS says:

    DM said,
    "Ones in which "let's just see what happens" seems to be the entire research program. However I would submit that when the track record suggests that the PI knows how to test hypotheses then perhaps a little credit should be extended for the one or two experiments that seem like discovery efforts."
    My feelings are mixed here. On the one hand it seems intuitively sound that, if one is going to take a risk on a project, one should at least mitigate it somewhat by appealing to the proven competence of the person proposing it. However, it could end up being regressive in the sense that the policy reverses the controls put in place to remove bias against new investigators. The latter of whom we would hope would be the most likely to come in with fresh ideas; that is, if the system didn't provide such a powerful incentive to simply sit down and break rocks with everyone else.
    Although, if a productive and accomplished career transitioning postdoc from a productive and accomplished lab was also allowed to apply for a more open grant directed at innovative stuff*, it might serve the middle ground; i.e. exploiting the fresh ideas of the science "yoof" in a stable and proven environment.
    * without the added, and sometimes laborious, mentoring caveats of some of the K awards

Leave a Reply