I often lament* the fact that Sb doesn't have a blog that focuses on the methods and practices of the social sciences. The reasons are many but I think my most general formulation is that the social sciences are the most scientific of the sciences and nobody seems to understand this.
Simmer down, simmer down.
The reason I say this is because I think that the relative successes in many other sciences, in terms of reductionist information gathering and (unbelievably artificially**) controlled experimental preparations, leads to a false impression of what the scientific process is. Repeatability of our experimental preparations, combined with close correspondence of prediction to outcome, leavened with really low variability leads to a confidence that we know the Answer.
We do not. We get to better and better (perhaps) approximations of the Answer. That is all.
The social sciences, in contrast, are almost inevitably adorned with...imprecision. Our ability to predict individual human behavior and indeed group behavior with what we think of as specificity or accuracy is often pretty bad. Our ability to conduct experiments that are highly repeatable across samples can be maddeningly poor. We can describe group tendencies but results are met with a drumbeat of anecdota which are supposed to falsify the general outcome. We can specify our experimental conditions, but then someone in the audience wants to kvetch about those conditions not being representative of his or her favored conditions. Interestingly, you see this less often in the more reductionist sciences- when does a lay person start hammering you about your buffer or crappy antibody or selected inbred rat strain?
How silly this is. I mean, we take highly inbred mouse strains, throw in a genetic modification with about major five breeding steps to get the right tissue-specific, experimentally switchable expression, toss the heterozygotes, end up with a sample of maybe 5 or 10 per experiment...and then we want to generalize to humans? This is somehow better science than, e.g. asking a bunch of treatment seeking cannabis smokers about their withdrawal symptoms? Better than studying choice behavior of political voters and applying that to subsequent political behavior of other voters? Better than applying IQ test results from one group of US schoolchildren to other US schoolchildren who don't share geography or socioeconomic status or race or whatnot?
Don't even get me started about technically inferior studies, trust me I've run across plenty of these in the non-social sciences. Plenty. Ditto picking on one specific study with no appreciation of the background of other studies in the field.
I was motivated in my musings on this topic by a recent comment which expressed the usual false-confidence attack on a social science hypothesis. In addressing a contention that dissection labs in secondary school biology classes are highly valuable, commenter becca observes:
where is the evidence that dissecting animals is "needed to give them a fully adequate biology course"? Citation please?
(note: I'm sure dissections are more useful/interesting for some students, and I'm sure computer programs are more useful/interesting for others; but I'm interested in evidence that virtual dissections per se lead to poorer educational outcomes in groups)
All science starts with observation and description. In this case, I would point anyone questioning along these lines to the comments on-blog and in the DonorsChoose comments which testify to the individual appreciation of people for their educational experiences. We can quickly get past mere anecdote and see that we are talking about enough people to constitute data. You can start whinging on about generality and self-selection if you like. That is a methods critique and I have no patience for those that use such triumphalist language to deny the data that are in front of their eyes. Similarly, you might look at the requests from teachers who want to launch dissection projects through DonorsChoose.
Then there are some rather consistent generalities about education to consider. General principles along the lines of learn by doing, hands-on, joy of discovery and the like. There are studies on some of these topics.
In short, it is probably the best-supported stance to start from the assumption that dissection is valuable and to ask what controlled studies prove pro or con. In the absence of new evidence, we are best to adopt a null hypothesis based on the extant evidence. Skepticism in science does not mean, "everything that doesn't accord with my personal anecdotal viewpoint is bullshit until and unless there is overwhelming evidence with hundreds of flawless studies to show otherwise". This is not skepticism, this is anti-science. It may be a narrowly targeted area of anti-science being expressed by an otherwise entirely-scientific mind, but it is still anti-science.
Skeptical science tests alternate hypotheses, yes. Skeptical science is prepared to shift and nuance viewpoint based on new evidence or even good rationale. These are essentials. But skeptical science is not the knee-jerk rejection of current evidence just because you managed to think up a credible alternative hypothesis.
This example brings us to another point which is that the discussion of interesting topics has a way of getting derailed because of a lack of agreement in advance about outcome. In this case, what is the desired end-result of high school biology dissection labs? Is it to turn someone on to science in general? Prepare a future biologist who will be involved with anatomy for reality? Start screening surgeons early? Or merely to generate sticky memories in those who will never go on to additional biology education or vocation? Your conception of desired outcome is critical to balancing ethical implications of supporting dissection projects for school children.
Not only that, but it impacts the chances of ever getting the overwhelming high-N proof about functional impact of a 1-2 day high school lab. There are not that many surgeons in the population. They are highly selected and highly trained individuals- getting absolute proof about influences on their motivations for becoming surgeons is going to be hard to come by. Anecdotes may be all we get. The degree to which various biology lessons stick with a non-biology person lifelong? Probably easier to come by. Either way, however, a lack of evidence is not proof of an alternative..it is just a lack of evidence.
Science means going with the available evidence for now, while trying to figure out how to get more and better evidence in the future.
I googled up a couple of blogs that look interesting to me, might have a post now and again that is exactly what I am looking for- approachable descriptions of how the social science process works. If you have any favorite blogs that communicate these ideas drop me a line, would you?
- American Academy of Political and Social Science Blog
- The adolescent risk behavior blog
- Social Science Statistics Blog
*because I don't have the chops to address this stuff properly. I just have enough grasp to understand my own ignorance and to appreciate what the social sciences can do. but this is bloggin' so what the heck...
**in some cases.