Professional Differences

Jun 10 2015 Published by under Alleged Profession, Science Communication

In real science, i.e., that that includes variability around a central tendency, we deal with uncertainty.

We believe, however, that there IS a central tendency, an approximate truth, a phenomenon or effect. But we understand that any single viewpoint, datum or even whole study may only reflect some part of a larger distribution. That part may or may not always give an accurate viewpoint on the central tendency.

So we have professional standards in place that attempt to honestly reflect this variable reality.

Most simply, we present the central tendency of effects (e.g., mean, median or mode) and some indication of variability around that central tendency (standard error, interquartile range, etc).

Even when we present a single observation (such as a pretty picture of a kidney or brain slice all hilighted up with immunohistochemical tags) we assert that the image is representative. This statement means that this individual image has been judged to be close to the central tendency of the images that were used to generate the distributional estimates that contribute to the numerical central tendency and variability graphs / tables presented.

Now look, I understand that it is a bit of a joke. There are abundant cracks and redefinitions that point out that the "most representative image" really means "the image that best makes our desired point".

There is a critically important point here. Our profession does not validate least representative image as an acceptable standard. Our professional standards say that it really should be representative if we ever present N=1 observations as data.

The alleged profession of journalism does not concern itself with truth and representativeness at all.

Their professional ethical standards, to the extent they exist, focus on whether the N=1 actually occurred AT ALL. In addition it focuses on whether that datum was collected fairly by their rules- i.e., was the quote on the record. Accuracy, again for the alleged profession, focuses only on episodic truth. Did this interviewee literally string these words together in this order at some point in time during the interview? If so, then the quote is accurate. And can be used in a published work to support the notion that this is what that interviewee saw, experienced or believes.

It is entirely irrelevant to the profession of journalism if that accident of strung-together words communicates the best possible representation of the truth of what that person saw, experienced or believes. Truth, in this sense, is not the primary professional ethical concern of journalism.

If the journalist pulls a quote out of an hour of conversation that best fits their pre-existing agenda with respect to the story they are planning to tell, it literally does not matter if every other sentence spoken by that person tells a different tale. It's totally okay because that interviewee literally said those words in that order on the record (and it is on tape!).

If a scientist processes twenty brains in the experiement, grabs the one outlier that tells the story they want to tell, trashes the 19 that say the opposite and calls it a representative image (even if by inference if not directly)....this is fraud and data fakery. Not okay. Clearly outside the professional bounds.

That, my friends, is the difference.

And this is why you should only agree to talk to journalists* that will send you a nearly final draft of their piece to ensure that you have been represented accurately.

If every single one of us scientists insisted on this, it would go a long way to snapping the alleged profession into line. And greatly improve the accurate communication of scientific findings and understandings to nonspecialist** audiences.

__
Representative image from here.

*They exist! I have interacted with more than one of these myself.

**Reminder, we ourselves are nonspecialist consumers of much of the science-media. We have two interests here.

21 responses so far

  • PaleoGould says:

    You're ignoring the fact that, as journalists are routinely lied to, the truth is not contained in the words they record. Which is WHY they have the different standard.

  • Ola says:

    Part of the problem with the alleged profession is that it kinda ceased to exist once social media and twitter and what-have-you came along, and made everyone into a reporter overnight. Journalists now have to differentiate themselves from the unwashed masses.

    What they don't get, is the larger trend that started with robots replacing car builders in the 1980s. Just like Uber made everyone a taxi driver overnight and that "profession" evaporated in a few months. Same goes for "expertise" of the local book seller, replaced by crowd-sourcing from GoodReads and Amazon. Same for travel agents replaced by Expedia and Kayak. Same for rural MDs phased out by nurse practitioners with Skype.

    Every one of these folks thought they had a skill that couldn't be replaced by something both cheaper and better, but they utterly failed to account for the "good enough and a lot cheaper" approach. None of the replacements are as good as the originals, but they're nearly as good and that's all that matters. Yes, strictly-speaking twitter will never replace journalism, but for a lot of things it comes pretty damn close!

    The question we should be asking as scientists, is whether our profession is in similar danger? What technological advances are going to be almost good enough to replace us? At what point does NIH start out-sourcing experiments to cheap labor markets because it's good enough and a lot cheaper than paying US academics to do the same research?

    So yes, laugh all you want at the journalists failing to come to terms with the fact that their job is easy and anyone can do it now. It's a matter of time before the same applies to science.

  • drugmonkey says:

    Dunno Ola, It was clear to me that journalism was dead the moment George H. W. Bush came out of the family compound in Kennebunkport, straightfacedly told the press that Clarence Thomas was the most qualified for the SCOTUS nomination and the stupid press didn't immediately and full-throatedly call him on his obvious bullshit.

    A light in our democracy went out that day.

  • jmz4gtu says:

    This seems topical:

    https://docs.google.com/forms/d/1aYTJMcJRcB3Aa0eA4crGzxQOYJ_g6YN-ZpgSYMU1Wog/viewform?c=0&w=1

    It is a survey by a group called Sense About Science ( http://www.senseaboutscienceusa.org/about/ ), and it asks what information scientists want to know when giving interviews to journalists.

  • Adam says:

    This doesn't sound like a very informed opinion of Science Journalism to me. In particular, it is not accurate to state that journalism as a whole "doesn't concern itself with truth". In fact, the first ethical principle in the Society for Professional Journalists Code of Ethics is "Seek the truth and report it" (http://www.spj.org/ethicscode.asp). It's ironic because generalizing from non-representative observations is precisely the sort of thing that the blog author is criticizing. I suggest reading (or re-reading) "Am I Making Myself Clear?" by former science editor at the New York Times, Cornelia Dean.

  • drugmonkey says:

    I've been a consumer of the offerings of journalism for a long time Adam. The profession needs to get its act together.

  • Adam says:

    Yes, Ms. Dean discusses the exact issue that Mr. Oliver addresses in chapter five of the book that I recommended. The matter of how to be objective when reporting is complex, and not something that can be appropriately discussed in a few glib comments. Ms. Dean gives a detailed analysis in her book.

    Can you not see the irony of cherry-picking an example like this and using it to generalize to all journalism as a whole?

  • DrugMonkey says:

    Can you not see that the profession of journalism *IN PRACTICE* is on display in public and some of us actually pay attention to what it does and does not do? And that any books you refer to are of zero use if the central tendency of the profession is to behave otherwise?

    Mr. Oliver's skit is so truthful because it sends up the reality of how journalism actually behaves on climate change. If journalism was doing a good job, his skit would be laughably stupid, rather than hilariously painful.

    as but one other example:
    Where in your books does it say that any story of people allegedly on recreational drugs should actually come with verification that this is indeed the drug the person ingested? Or at the very least the iron clad principle that it must be followed up later when such information is available from tox testing sources?

  • jmz4gtu says:

    "If a scientist processes twenty brains in the experiement, grabs the one outlier that tells the story they want to tell, trashes the 19 that say the opposite and calls it a representative image (even if by inference if not directly)....this is fraud and data fakery. Not okay. Clearly outside the professional bounds. "
    -I think this would would pretty clearly fall outside of acceptable bounds of journalistic ethics as well. If you talked to 20 people, and 1 reported a fire, and 19 said they didn't see anything, you'd have to be a shady reporter to write up a story about the mystery blaze that everyone seems intent on covering up.
    That being said, the process we currently have for reporting scientific discovery is a bit like telephone: Postdoc->PI->Uni press agent->Reporter. Reporter then comes back to PI, who, after all the hype feels obligated to put the best face on his work. Reporter then punches it up a bit for a general audience. A lot of truth is going to get lost in that process.

    Both journalists and scientists are ostensibly producing the same thing: useful ways of understanding the world. However, unlike us, journalists have the added burden of making it interesting to non-journalists. Naturally, when they report on science, they're going to do so in a way most of us would not approve of amongst ourselves.

    But if we're being honest, I think most scientists have omitted, massaged, or otherwise manipulated data, citations, or the order of events to create a more clear and consistent narrative. This is especially true of glam pubs.

  • physioprof says:

    But if we're being honest, I think most scientists have omitted, massaged, or otherwise manipulated data, citations, or the order of events to create a more clear and consistent narrative. This is especially true of glam pubs.

    I am probably going to regret this, but can you tell us what you mean by "omitted, massaged, or otherwise manipulated...the order of events to create a more clear and consistent narrative"?

  • One partial solution to record your conversation (and let them know you're doing this*). In my experience, when I was regularly dealing with reporters, this pretty much kept them from really egregious behavior.

    *Not doing so can send you to jail in some states if you do it over the phone.

  • jmz4gtu says:

    Just for the order of events?
    For example, I'll notice a paper that gets cited in the intro as justification for a line of experiments, even when I know from personal interactions that most of the experiments in this article were started before the other paper was a gleam in a PI's eye.

    And I think everyone retcons their experiments, in terms of which ones were done first and for what reasons. Did you do that RNA-seq first, with your fully formed hypothesis or was someone harvesting bone marrow from your labs mutants and you decided to take a crack at the pancreas (personal example)?

    Crafting a good narrative face for your science is important for communicating your results, it's just how humans process information. And it can be used maliciously, regardless of one's profession.

    Also, even though the examples I give are innocuous, I think it is important to remind ourselves that they are still, technically, deceit.

  • physioprof says:

    Also, even though the examples I give are innocuous, I think it is important to remind ourselves that they are still, technically, deceit.

    You are a pernicious fucken delusional goddamn lunatic if you really think that "In order to test the hypothesis that blah, blah, blah, we did blah, blah, blah" could ever be "still, technically, deceit", even if it is completely false.

    I swear I need to just stop responding to your fucked up gibbering idiocy. You are actually fucken DoucheMonkey pretending to be a fucken wackaloon just to fucke with me, right???

  • Pinko Punko says:

    DM- what is going on? Look at your post on response to criticism from grant review, and then look at how you are blowing up the world on Twitter. I know there is a history with NG, but be a robot assassin about it- are you playing to the crowd or trying to prove a point?

  • Ola says:

    Something struck me from the Adam/DM exchange above - journalism is a bit like case reports in clinical journals. All you need is one person to do/say/suffer one thing, and bingo you've got a story. No matter if it turns out to be something else later, because by then everyone's moved along. I always thought there was something shifty about case reports. Now I know why, it's just the alleged profession by another name

  • Pinko Punko says:

    Although, Ola, at least with case reports the idea is that anecdata might eventually add up to something, whereas randos saying things is sort infinite. Defects in the human body may also be infinite but there is always chance that some constellation of symptoms might have an underlying cause. I think a case in favor of case reports would be the barfing and the weed that DM talks about.

  • drugmonkey says:

    Ahh Pinko Punko I do enjoy it so when someone connects the dots....well done.

  • Pinko Punko says:

    Could there be a more perfect storm of Nature News steno-graphing an insane Econ paper PUBLISHED IN PLOS BIOLOGY no less, that then leads us to this conversation.

    I just would say there's no need to go back to deep time to attack that DRAMA SCIENCE BROKEN article in Nature News, especially once you had their attention. By the way- this has got to be my fave new industry. Maybe the op-eds on all that pay a lot of scratch. Maybe this paper is an Eisen false flag about peerz reviewz brokenz.

    I wonder that the Economists came up with such a low number for science waste. Only 28 billion? What do they define as waste? From one point of view, digging holes in ground to fill them back in is OK for the economy. Did this waste calculation taken into account the quintillion dollars of innovation and discovery generated by science?

    Why is science now being treated as an unaffordable luxury or worse, a hive of scum and villainy? I wonder what extremely public arguing and various agenda have to do with it. There seems to be a cottage industry for those that have opinions. At what point does desire to share opinions become a desire to control dialogue and descend into narcissistic behavior.

  • jmz4gtu says:

    @pinko:
    That PLOS paper is a pretty dull, in every sense of the word. You can read their criteria for irreproducibility in the S1 section, but basically it takes a bunch of guesstimates (drawn from other studies about irreproducibility) and applies them slapdash over all of "biomedical research", including things like micoplasma infection and cell misidentification (25% of total irreproducibility), which probably only affect a subset of researchers who a) work with cell culture and b) don't follow proper cell line quality control measures. Additionally, they cite the Hughes report of cell line contamination, but then, when calculating the effect this has on irreproducibility, assume the study had no effect on the number of labs decontaminating and verifying their lines, which seems stupid.

    Additionally, their estimate of "irreproducibility" factors in lack of experimental detail in methods sections and negative result reporting (7.6 and 18% of their total irreproducibility) and inadequate statistical power (19.5%). Really only the last one is concerning to me. Everyone knows if you really want to reproduce someone's results, you have to sit down with them and go over their protocol. Or better yet, watch them do it.

  • Pinko Punko says:

    The paper seemed not publishable and then Nature News said "hey this happened". Really dumb. Just waiting for the PPPR lobby to use it to say peer review is broken.

Leave a Reply