Today's Twittsplosion was brought to you by @mbeisen:
he then elaborated
There was a great deal of distraction in there from YHN, MBE and the Twitteratti. But these are the ones that get at the issue I was responding to. I think the last one here shows that I was basically correct about what he meant at the outset.
I also agree that it would be GREAT if all authors of papers had deposited all of their raw data, carefully annotated, commented and described (curated, in a word) with all of the things that I might eventually want to know. That would be kickass.
And I have had NUMEROUS frustrations that I cannot tell even from methods sections what was done, how the data were selected and groomed, etc in many critical papers.
It isn't because I assume fraud but rather that I find that when it comes to behaving animals in laboratory studies that details matter. Unfortunately we all wish to overgeneralize from published reports....the authors want to imply they have reported a most universal TRUTH and other investigators wish to believe it so that they don't have to sweat the details.
This is never true in science, as much as we want to pretend.
Science is ever only a description of what has occurred under these specific conditions. Period. Including the ones we've bothered to describe in the Methods and those we have not bothered to describe. Including those conditions of which we have no knowledge or understanding that they might have contributed.
Let us take our usual behavioral pharmacology model, the 10 m Hedgerow BunnyHopper assay. The gold standard, of course. And everyone knows it is trivial to speed up the BunnyHopping with a pretreatment of amphetamine.
However, we've learned over the years that the time of day matters.
Until...finally....in its dotage seniority. The Dash Lab finally fesses up. The PI allows a trainee to publish the warts. And compare the basic findings, done at nighttime in naive bunnies, with what you get during the dawn/dusk period. In Bunnies who have seen the Dash arena before. And maybe they are hungry for clover now. And they've had a whiff of fox without seeing the little blighters before.
And it turns out these minor methodological changes actually matter.
We also know that dose response curves can be individual for amphetamine and if the dose is too high the Bunny just stims (and gets eaten by the fox). Perhaps this dose threshold is not identical so we're just going to chop off the highest dose because half of them were eaten after that dose. Wait...individuals? Why can't we show the individuals? Because maybe a quarter are speeded up by 4X and a quarter by 10X and now that there are these new genetic data on Bunny myocytes under stressors as diverse as....
So why do the new papers just report the effects of single doses of amphetamine in the context of this fancy transcranial activation of vector-delivered Channelrhodopsin in motor cortex? Where are the training data? What time of day were they run? How many Bunnies were aced out of the study because the ReaChr expression was too low? I want to do a correlation, dammit! and a multivariate analysis that includes my favorite myocyte epigenetic markers! Say, how come these damn authors aren't required to bank genomic DNA from every damn animal they run just so I can ask for it and do a whole new analysis?
After all, the taxpayers paid for it!
I can go on, and on and on with arguments for what "raw" data need to be included in all BunnyHopping papers from now into eternity. Just so that I can perform my pet analyses of interest.
The time and cost and sheer effort involved is of no consequence because of course it is magically unicorn fairy free time that makes it happen. Also, there would never be any such thing as a protracted argument with people who simply prefer the BadgerDigger assay and have wanted to hate on BunnyHopping since the 70s. Naaah. One would never get bogged down in irrelevant stuff better suited for review articles by such a thing. Never would one have to re-describe why this was actually the normal distribution of individual Hopping speeds and deltas with amphetamine.
What is most important here is that all scientists focus on the part of their assays and data that I am interested in.
Just in case I read their paper and want to write another one from their data.
Without crediting them, of course. Any such requirement is, frankly my dear, gauche.