Placeholder figures

Honest scientists do not use "placeholder" images when creating manuscript figures. Period. 

See this nonsense from Cell

22 responses so far

  • Grumble says:

    Emilie is fucking deluded. She and her staff "investigate" the rash of "outings", but the bottom line is that she is biased. Does it do Cell's reputation any good if paper after paper gets an erratum saying "Figure 3 has been changed by the original due to an inadvertant error by the authors"? I suspect that she has an enormous willingness to extend the benefit of the doubt to authors in the course of her "investigations".

    This negative opinion is not just a reaction to her mealy-mouthed blog post, but to second-hand knowledge of how journals like Cell handle the "outings" she describes. I've even heard of cases where they don't bother to publish an erratum - they just go ahead and change the online version without telling anyone.

  • drugmonkey says:

    They have a brand to protect. Or something.

  • Stochastic Sam says:

    It's probably a complete coincidence that her blog post follows on the heels of:
    http://link.springer.com/article/10.1007%2Fs11948-015-9668-7

    Summarized on In the Pipeline:
    http://pipeline.corante.com/archives/2015/06/23/not_25_but_still_not_good.php

    The image conjured by, "But this growing onslaught of alerts that captured those gathered around with wine glasses in their hand is problematic on both sides of the coin" is striking, particularly when both sides of the coin that is subsequently presented suggests that pointing out discrepancies in data is a problem rather than a solution. One might have to put down that glass of wine, after all.

  • qaz says:

    Honest scientists do not submit manuscripts with placeholder figures. What the scientist does internally while writing the paper is irrelevant. (As long as the final product is honest.)

  • AcademicLurker says:

    "placeholder figures" is a BS excuse. A placeholder figure is a blank box with "mass spec. data goes here" written in it, not old data pasted in. Who does that?

  • drugmonkey says:

    Fakers do that.

  • physioprof says:

    Honest scientists do not submit manuscripts with placeholder figures. What the scientist does internally while writing the paper is irrelevant. (As long as the final product is honest.)

    I disagree strenuously with this. If any of my trainees ever showed me a figure mockup with fake data in it--even if they were totally up front about it as being a placeholder--I'd blow a gasket. Even if you are a totally honest individual and have zero intent to fake anything for realz, this is a bad idea for all kinds of reason.

  • qaz says:

    CPP - To be fair, I do agree with you on this. (And I suspect that this is what DM is incorrectly calling "storyboarding", but it wouldn't be a good internet argument without misagreement on terms.) The one time a student put in a data-looking figure that looked publication quality as a placeholder figure, I did blow a gasket at him. (It was a him.)

    But that's not a placeholder figure in my book. Nor is it a storyboard. A placeholder figure is a sketch - without numbers, without data points. It's a trend that we expect Y to decrease as X increases; it's that the controls should show no effect; it's a sketch that the last 100 times we ran these controls, we saw an inverted-U curve, so I think we'll see one with this cohort as well; it's a sentence that says "put tuning curves from cell type X here". A placeholder figure is an expectation that would never be mis-interpreted as data.

    I still think that having expectations is not the main source of problems in science. The main source of problems in science is that too many people are scared to be wrong. I think that's due to a lot of factors (GlamourMags and Nobel Prizes and the project-based funding system at NIH and the limited funding available at NIH). It starts at the publication level --- people aren't willing to present an experiment or interpretation that fits the current data but turns out to be wrong in the long run when a better interpretation comes along --- and bleeds over into lab culture.

    Running your lab to ensure that its OK to be wrong is very hard. But if this was easy it wouldn't be science.

  • drugmonkey says:

    http://retractionwatch.com/2011/09/12/group-under-investigation-retracts-second-paper-claims-errant-figure-was-just-a-placeholder/

    Panels C–H of Fig. 5 were incorrectly inserted by the first author, Dr. Naoki Harada. Dr. Harada made this figure for practice for a presentation at an International Congress. He drew data from another experiment in which the same procedure was done. At the time, he did not have correct data because he had not finished experiments to produce genuine data for the figure. Although he went on to finish experiments to get genuine data, he forgot to replace the figure with the correct one before the paper was submitted to The Journal of Immunology. Although the submission of the incorrect figure was unintentional, we hereby withdraw our article.

    is a typical example.

  • drugmonkey says:

    From Cell crosstalk:

    One of the most common reasons an Erratum needs to be published for a paper is that an image was duplicated or an incorrect a blot or graph was included in the figure.

    This often occurs because the authors want to “hold a place” in a figure for forthcoming data and then forget to substitute the place-holding data with the actual relevant data. Here's our recommendation: if advance figure preparation helps you organize and prioritize your remaining experiments, save panel slots with a blank square filled only with text describing what you are hoping to show there. That would not easily be missed in the final proofing of a paper before submission.

    i.e., they apparently believe this bullshit 'placeholder' excuse making. Note that they do not describe how many errata are motivated by the authors catching it and how many by somebody else noticing and complaining.

  • drugmonkey says:

    from Nature

    When presented with the irregularity, most authors quickly find an explanation—it's almost always a clerical mistake and very rarely an experimental error. For instance, a micrograph or a band in a gel may have been duplicated because it was used as a placeholder and never replaced with the correct image.

  • drugmonkey says:

    Anyone who thinks this "oopsie, placeholder" nonsense is not common is fooling themselves.

  • martini says:

    "Honest scientists do not submit manuscripts with placeholder figures. What the scientist does internally while writing the paper is irrelevant. (As long as the final product is honest.)

    I disagree strenuously with this. If any of my trainees ever showed me a figure mockup with fake data in it--even if they were totally up front about it as being a placeholder--I'd blow a gasket. Even if you are a totally honest individual and have zero intent to fake anything for realz, this is a bad idea for all kinds of reason."

    The only time "story boarding" should be done, is in the context of putting existing data into a figure as you are writing the manuscript and/or looking for weaknesses. My lab uses their powerpoint slides from lab meetings to mock up what data they are going to include in each figure and then refines them into final format, but they only use data from completed experiments.

    I don't even let me trainees write "mock text" based on expected results. I know a lot of labs that do this and figure story boarding both of which are asking for trouble.

  • drugmonkey says:

    Mock text on expected results? Are. You. Kidding. Me.?

  • qaz says:

    Taking these statements at face value I agree with you. That's fycked up. Aside from the fact that it is likely to lead to mistakes (like all those "sloppy" examples), it's a waste of time! It takes time to make a good figure. We never waste time making the figures "snazzy" (meaning good graphic design, clear, readable, pretty) until we have all the data in place in the entire paper. I would never let a student put anything that looks like real data in a figure or write text based on expected results. That's crazy.

    But I don't think most of these examples should be taken at face value. How often are these "sloppy" "placeholder figures" really examples of fraud that got caught?

  • drugmonkey says:

    I don't know.

  • qaz says:

    Here's the test. How many of those placeholder figures made the paper look worse? If those placeholder figures are really placeholders, then they should only be right at chance. If they're doing better than chance, then either they're psychic or they're lying.

  • chall says:

    I continue to get surprised on how naive one can be when writing an excuse for a figure in a paper that you submitted AND got published. I mean, really? I can't be the only one freaking out on looking at the raw data, and reanalyzing so I am sure that the figure that I made is correct and not the wrong one when writing a paper? Or double checking the data I send to my PI, and checking the figures that we make to go into the paper? And making sure that I know all the steps A-H when making "the perfect picture for the paper".

    Placeholders to me are "rough text describing on what you want as a picture". Not a "fake picture", not a "modified picture". just a descriptor on what you are expecting... if that.

    As for the "flipping the gel" and "copying the same control band".... I got nothing. If you were to do that to me, I'd flip out. it's mainly an excuse to tell me "I'm not a scientist and I know nothing about real, honest research".

  • JustAGrad says:

    I completely agree about not using placeholder figures or writing text to describe expected results. But perhaps people (mostly students) get the idea to do this because it's basically required when writing abstracts for conferences. How many people actually have a study completed by the deadline 6-12 months prior to the conference?

  • Grumble says:

    "How often are these "sloppy" "placeholder figures" really examples of fraud that got caught?"

    I'll go out on a limb here and say, "most of them." This placeholder excuse is just bullshit. If you are so fucking sloppy that you can't eliminate goddamn PLACEHOLDERS from your final paper in a big-whup journal like Cell, then what does that say about how carefully you conducted the experiments? IF, that is, you conducted them AT ALL.

    Sorry, I have no sympathy for this nonsense.

    "How many people actually have a study completed by the deadline 6-12 months prior to the conference?"

    There are ways of writing abstracts that summarize what you have now and where you are headed, without making shit up. So it is not "required" to do anything at all similar to making placeholder figures. If your PI hasn't taught you this, make her. Or find a less dishonest PI.

  • A Salty Scientist says:

    For me, the only acceptable placeholders are blank text boxes on a grant so that I stay within the damn page limits.

  • drugmonkey says:

    what you have now and where you are headed, without making shit up. So it is not "required" to do anything at all similar to making placeholder figures. If your PI hasn't taught you this, make her. Or find a less dishonest PI.

    Every bit of this.

Leave a Reply