Scientists "cut corners", eh, NPR?

Sep 15 2014 Published by under Fixing the NIH, NIH, NIH funding

The latest in the NIH/science focused series from Richard Harris is:
Patients Vulnerable When Cash-Strapped Scientists Cut Corners

It hits on some of the expected themes. Including:

Most of the experimental ALS drugs, it turns out, undergo very perfunctory testing in animals before moving into human tests — based on flimsy evidence.

In hopes of figuring out why, scientists went back to take a second look at the mouse experiments that were the basis for the human study, and found them to be meager. Additional, more careful tests found no compelling reason to think the experimental drug would have ever worked.

Stefano Bertuzzi, the executive director of the American Society for Cell Biology, says that's partly because there is little incentive for scientists to take the time to go back and verify results from other labs.

"You want to be the first one to show something," he says — not the one to verify or dispute a finding, "because you won't get a big prize for that."

and then the former head of NINDS, Story Landis checks in:

Landis has thought a lot about how those last-chance patients ended up in this untenable situation. There is no single answer, she says, but part of the explanation relates to a growing issue in biomedical science: the mad scramble for scarce research dollars.

"The field has become hypercompetitive," she says.

Many excellent grant proposals get turned down, simply because there's not enough money to go around. So Landis says scientists are tempted to oversell weak results.

"Getting a grant requires that you have an exciting story to tell, that you have preliminary data and you have published," she says. "In the rush, to be perfectly honest, to get a wonderful story out on the street in a journal, and preferably with some publicity to match, scientists can cut corners."

So. The offending comment came from Story Landis. I am shaking my head with dismay.

Remember, SHE is the one who has made the decision on which grants get funded at the National Institute on Neurological Disorders and Stroke since 2003. Specifically and personally.

All that peer review of science and Program Officer priority and National Advisory Council concurrence? That is all process advisory to the Director who makes the ultimate decision on what to fund.

So, if there are any fingers to be pointed about what is driving particular aspects of scientist behavior in their attempts to stay funded merely so that they can work on thorny problems like ALS, well that finger goes right at Story Landis.

It's really simple, Directors of ICs. Simple as pie.

If you want to prioritize meticulously replicated and extended scientific investigations, you fund those proposals that are planning just that with urgent priority. When you are evaluating PIs to support with the usual spectrum of Programmatic priority handouts, select those with a history of meticulous replication instead of those who hit the hot highlights and never flesh out the story.

I'm telling you, this would snap a lot more PIs right into line in this current environment.

We are just exactly like everyone else. We respond to the contingencies under which we operate. When HawtEleventyGlamourScience and InstantlyTranslational is seen as the route to funding, guess what. We are going to "oversell weak results". When meticulous and incremental advance is seen as the province of irrelevant plodders who do not deserve grant funding, nobody in their right mind* is going to propose a project which mentions any such thing.

So, you want my advice? Find projects in your funded portfolio that meet the meticulous replication standard- give them a R37 MERIT extension and say why. Publicly. Next, find some of these type of proposals in your just-missed pile and fund them. Also brag on that.

Look up the PLoS ONE pubs that are associated with your grants.....presumably they are going to be enriched in negative results, confirmational findings and all the good stuff Story Landis seems to be seeking. Put out a press release on THOSE results. Particularly the negative ones.

In short, put your money where your mouth is, NIH. Don't engage in this double speak when you, yourselves, are a major contributing factor. Don't put this on your extramural investigators and pretend that you played anything other than a central role in their behavior.

__
*I may possibly have proposed** a grant which was dedicated to replication and sorting out failures-to-replicate with the explicit expectation of a lot of essentially negative or pedestrian results.

**and received funding for***

***yes, I would have been, assuming that this indeed transpired, as amazed as you are****.

****should such a thing have occurred, I have absolutely no explanation for how such a feat was accomplished*****. Really, none.

*****I mean, the 2%ile priority score, if such had been the result, only begs the question, right?

11 responses so far

  • You pretend this would be easy for ICs to do, but you are well aware of the structural barriers to large-scale overriding of study section peer-review outcomes and the massive unintended consequences that would ensue.

  • drugmonkey says:

    And you are insufficiently optimistic about the power of the bully pulpit and the impact of using what latitude they habitually use each and every Council round would have on study section peer-review outcomes.

  • AsianQB says:

    That story painted, in the eyes of the general public, scientists as bad people who cut corners and give false hope to people with terminal illnesses, while maintaining the guile that the NIH had no part in all of this.

    Bad NIH, Bad, unless NPR totally edited the story to make it sound like it did.

  • hpmcmf says:

    DM,

    I don’t understand your dismay at Story Landis saying that scientist cut corners. Isn’it true?. And how is that statement or reality different from the fact that more and more papers get retracted in areas relevant to disease?

    Do you think that lay people are so uneducated (not say stupid) not to know what’s going on?.

    Yes. The problem is that there is not enough money and some people want to survive doing research and they want to survive “gloriously”. And that’s the problem

    I am not defending Story Landis. I have no reason whatsoever to defend her.

    Having said that I should say that when people are in positions of power, like Story Landis or Collins or any ICs, they obviously do not want to compromise their job/positions by opening lines of fire. That’s the huge problem. Unfortunately, nobody has the courage to do that because we all know the result: You get killed !!.

  • Spiny Norman says:

    From what I hear from NCI-funded pals, an IC Director can have immense influence on what does and does not get funded — for better, or for worse. Varmus is reputed to take a very personal interest in funding decisions at the per-grant level of granularity.

  • drugmonkey says:

    Spiny,

    Volkow at NIDA is widely reputed to exercise her decision making with....considerable latitude, shall we say.

    I don't really take a position on what the right balance should be, I just suggest that the legal reality is what it is and for a Director such as Landis to pretend that they are not HUGELY pivotal in any problems and proposed solutions is....an evasion at best.

  • hpmcmf says:

    Yes. That maybe the case. But how does the public perceive the advances in curing cancer as opposed to ALS treatments,?(noting also that much more money and years of public funding have accumulated for NCI than for NINDS).

  • becca says:

    Humans love drama. Time pressed reviewers prefer "ZOMG exciting results !!1!!!!elebenty" to "we will rigorously demonstrate...". But you can easily sell "Prof. X says A, Prof. Y says B, fund me and find out TEH TRUTHS! *duhn duhn DUHN!*" as drama.

    As an aside, does anyone know if simply taking part in a research trial increases survival in a disease like ALS?

  • gingerest says:

    Oh, c'mon, DM, we all know that if you neuroscientists really cared about patients, you would fund replication work out of pocket. You know, the same way that you would spend all your waking hours in the lab if you had even a rudimentary interest in your work.

  • gingerest says:

    Becca, the only way I can think of to answer that question is for the ALS Registry to start a longitudinal study comparing registrants who do and don't take part in clinical trials (and there would still be a possible source of bias in that people who do not take part in the registry might also be less likely to take part in clinical trials and might represent a population at special risk who aren't represented in the study.)

  • DrugMonkey says:

    That's kid-cancer, gingerest, not neuroscience

Leave a Reply