No, Cell, the replication does not have bearing on the original fraud

Sep 12 2016 Published by under Scientific Misconduct, Scientific Publication

Via the usual relentless trolling of YHN from Comrade PhysioProffe, a note on a fraud investigation from the editors of Cell.

We, the editors of Cell, published an Editorial Expression of Concern (http://dx.doi.org/10.1016/j.cell.2016.03.038) earlier this year regarding issues raised about Figures 2F, 2H, and 3G of the above article.
...
two labs have now completed their experiments, and their data largely confirm the central conclusions drawn from the original figures. Although this does not resolve the conflicting claims, based on the information available to us at this time, we will take no further action. We would like to thank the independent labs who invested significant time and effort in ensuring the accuracy of the scientific record.

Bad Cell. BAD!

We see this all the time, although usually it is the original authors aided and abetted by the journal Editors, rather than the journal itself, making this claim. No matter if it is a claim to replace an "erroneous placeholder figure", or a full on retraction by the "good" authors for fraud perpetrated by some [nonWestern] postdoc who cannot be located anymore, we see an attempt to maintain the priority claim. "Several labs have replicated and extended our work", is how it goes if the paper is an old one. "We've replicated the bad [nonWestern, can't be located] postdoc's work" if the paper is newer.

I say "aided and abetted" because the Editors have to approve the language of the authors' erratum, corrigendum or retraction notice. They permit this. Why? Well obviously because just as the authors need to protect their reputation, so does the journal.

So everyone plays this game that somehow proving the original claims were correct, reliable or true means that the original offense is lesser. And that the remaining "good" authors and the journal should get credited for publishing it.

I say this is wrong. If the data were faked, the finding was not supported. Or not supported to the degree that it would have been accepted for publication in that particular journal. And therefore there should be no credit for the work.

We all know that there is a priority and Impact Factor chase in certain types of science. Anything published in Cell quite obviously qualifies for the most cutthroat aspects of this particular game. Authors and editors alike are complicit.

If something is perceived to be hott stuff, both parties are motivated to get the finding published. First. Before those other guys. So...corners are occasionally cut. Authors and Editors both do this.

Rewarding the high risk behavior that leads to such retractions and frauds is not a good thing. While I think punishing proven fraudsters is important, it does not by any means go far enough.

We need to remove the positive reward environment. Look at it this way. If you intentionally fake data, or more likely subsets of the data, to get past that final review hurdle into a Cell acceptance, you are probably not very likely to get caught. If you are detected, it will often take years for this to come to light, particularly when it comes to a proven-beyond-doubt standard. In the mean time, you have enjoyed all the career benefits of that Glamour paper. Job offers for the postdocs. Grant awards for the PIs. Promotions. High $$ recruitment or retention packages. And generated even more Glam studies. So in the somewhat unlikely case of being busted for the original fake many of the beneficiaries, save the poor sucker nonWestern postdoc (who cannot be located), are able to defend and evade based on stature.

This gentleman's agreement to view faked results that happen to replicate as no-harm, no-foul is part of this process. It encourages faking and fraud. It should be stopped.

One more interesting part of this case. It was actually raised by the self-confessed cheater!

Yao-Yun Liang of the above article informed us, the Cell editors, that he manipulated the experiments to achieve predetermined results in Figures 2F, 2H, and 3G. The corresponding author of the paper, Xin-Hua Feng, has refuted the validity of Liang’s claims, citing concerns about Liang’s motives and credibility. In a continuing process, we have consulted with the authors, the corresponding author’s institution, and the Committee on Publication Ethics (COPE), and we have evaluated the available original data. The Committee on Scientific Integrity at the corresponding author’s institution, Baylor College of Medicine, conducted a preliminary inquiry that was inconclusive and recommended no further action. As the institution’s inquiry was inconclusive and it has been difficult to adjudicate the conflicting claims, we have provided the corresponding author an opportunity to arrange repetition of the experiments in question by independent labs.

Kind of reminiscent of the recent case where the trainee and lab head had counter claims against each other for a bit of fraudulent data, eh? I wonder if Liang was making a similar assertion to that of Dr. Cohn in the Mt. Sinai case, i.e., that the lab head created a culture of fraud or directly requested the fake? In the latter case, it looked like they probably only came down on the PI because of a smoking-gun email and the perceived credibility of the witnesses. Remember that ORI refused to take up the case so there probably was very little hard evidence on which to proceed. I'd bet that an inability to get beyond "he-said/he-said" is probably at the root of Baylor's "inconclusive" preliminary inquiry result for this Liang/Feng dispute.

33 responses so far

  • DJMH says:

    Absolutely agree. Once you have the Cell paper, you have doors open. Faking data to get there should be punished. Just because you were clever/lucky enough to fake the data in the right direction doesn't mean you deserve the paper.

  • Another Assistant Prof says:

    nonWestern postdoc? Bad, DrugMonkey! Bad!

    Agree 100% with everything else you've written. But, seriously? That comment has no bearing on what you're actually talking about here. If you want to address cultural/geographic differences in the data fabrication problem, address is directly. Don't use someone's country of origin in an off-handed way to imply it makes the fraud worse. Your point about being unable to locate them anymore is made well enough by remarking that the postdoc can't be located anymore.

  • Jonathan Badger says:

    I think the argument of the accused postdoc being non-Western isn't to impinge the honor of people from non-Western countries but is according to the old excuse "the postdoc that did the experiments is back in his/her home country and can't be reached". That excuse doesn't work so well if the postdoc is somewhere in the US or Western Europe.

  • drugmonkey says:

    If you want to address cultural/geographic differences in the data fabrication problem, address is directly.

    JB is correct. I am referring to the phenomenon that all too frequently the lab head conveniently scapegoats a former trainee who oh-so-conveniently cannot be reached anymore. And editors put up with this being printed in erratums or retractions or what have you. The subtext being that no, I'm not going to buy this scapegoating as an excuse at face value. I was not trying to say that nonWestern trainees are any more or less likely to fake data with this remark.

  • SidVic says:

    "If you want to address cultural/geographic differences in the data fabrication problem, address is directly."

    A system that requires one to lie to get along (I'm thinking the chicoms here, but it can increasingly be applied to the way we do buisness). This system will produce liars.

  • BProf says:

    Isn't it a little fishy that this dude got to pick a couple of his friends to "repeat" his experiments?

  • Another Assistant Prof says:

    Well *that's* good that it was not your intent - thank you for the explanation, which makes sense. But you do need to be careful about adding that kind of demographic qualifier without the detail, because it's just as easy to assume you were referring to two different qualities (demographic, not-accessible) rather than one (not-accessible, not accessible)

    To the bigger issue: nobody in the research zone is not self interested, unless you got someone who had competing theories (opposite hypothesis) but showed the same result - and who would be willing to spend their effort on that? But agreed with BProf that it does make one wonder if there would be other motivations for the other labs to "repeat" (how about, "show for the first time") that the expectation held.

  • jmz4 says:

    I disagree that the whole paper should be retracted. If you actually read the paper, none of the main claims of the paper rest on the faked data (which are 3 sets of co-IPs). 3 panels of fraudulent data do not invalidate the other 48 (in the main text). Also, at least some of the fraudulent data are validated by similar experiments.

    What I do disagree with is the vagueness of Cell's statements. They should make explicit what the claims were (e.g. how the data were faked, and in which way), and what independent tests were done to validate or invalidate them, and publish them if possible (with author names and a full citation).

    To your broader point, the real problem is the allure of hedging to get these gatekeeper publications. There really does need to be less emphasis on the journal something was published in versus the thoroughness and quality of the paper.

  • SidVic says:

    Jmz-" I disagree that the whole paper should be retracted."
    Good Lord! so now we must dissect out the good parts of manuscripts! The well is poisoned. Retract the whole goddamned thing. Sow salt upon the fields (of these guys careers) and move on.

  • drugmonkey says:

    easy to assume you were referring to two different qualities (demographic, not-accessible) rather than one (not-accessible, not accessible)

    I was. It is invariably a nonWestern ex-trainee that is mysteriously unavailable for contact in such situations.

    I disagree that the whole paper should be retracted.
    I didn't offer an opinion on whether this paper should be retracted. I am focused on the notion that replication of the faked data is some sort of justification for not taking further action.

    3 panels of fraudulent data do not invalidate the other 48 (in the main text). Also, at least some of the fraudulent data are validated by similar experiments.
    Why were the data included? How do you know these data had no material impact on whether the study was acceptable for publication in the first place? How do we know what the reviewers would have thought without it? If there is such totally immaterial data being included in a Cell manuscript....dude, what?

    what independent tests were done to validate or invalidate them,
    You are seriously missing the main point here. These independent tests are totally irrelevant to whether or not the data supplied with the original manuscript were faked.

  • drugmonkey says:

    Isn't it a little fishy that this dude got to pick a couple of his friends to "repeat" his experiments?

    I don't care who it is that is doing the replicating. The issue is the same. It wasn't shown with real evidence the first time and therefore whether it replicates or doesn't replicate is totally irrelevant.

  • qaz says:

    @jmz4 - This is why papers should not be monster compilations of 100 studies! If a paper gets into a GlamourMag as a thing then it should live or die as that thing. Maybe the authors had a set of basic good results but couldn't Glamour them so they made up the last little bit of snazzy data that Glamoured it.

  • qaz says:

    Also, this is why we have to get over being first. It's not just fakery. There are a lot of cases where the first out the gate is sloppy.

  • jmz4 says:

    "Why were the data included? How do you know these data had no material impact on whether the study was acceptable for publication in the first place? How do we know what the reviewers would have thought without it? If there is such totally immaterial data being included in a Cell manuscript....dude, what? "
    -If I had to guess, my suspicion would be the fact that there were 4 co-first authors played a role.
    This paper seems like it started as separate projects that were converged into a frankenbeast. In that situation no one wants to get their data cut. The experiments read a little disjointed, like they weren't designed to complement each other originally and had to be crammed together artificially.

    "I didn't offer an opinion on whether this paper should be retracted. I am focused on the notion that replication of the faked data is some sort of justification for not taking further action. "
    -What further action would have them take at Cell if not retraction? To my knowledge they have no squad of ninjas at their disposal to sneak into the offender's lab and dismantle their Western blot apparatus.

    "Good Lord! so now we must dissect out the good parts of manuscripts! The well is poisoned. Retract the whole goddamned thing. Sow salt upon the fields (of these guys careers) and move on."
    -If only, as *alleged* 98% of it is true, I'd rather have it at my disposal than rain fire on a possibly unscrupulous mentor (and hit a lot collateral underlings).

    " Maybe the authors had a set of basic good results but couldn't Glamour them so they made up the last little bit of snazzy data that Glamoured it."
    -I read the paper, the allegedly faked results are largely inconsequential to the central claims being made, even if they were completely false. So it doesn't seem like they were faked for Glam unless they just had a bitchy reviewer demanding them.

    For anyone interested:
    They are blots showing that the dephosphorylation of Smad2 depends on magnesium (2F and 2H), which is supported by a similar experiment (2G). They basically make no additional claims on this, it just points to mechanism of action, which is amply demonstrated by additional means (genetic and molecular). The 3G panel shows that PPM1a interacts specifically with the phosphorylated form of Smad2. This supposedly underlines their point that the interaction occurs in the nucleus, which is backed up in 3E by nuclear fractionation and Im-His co-localization (3D).

    Again, none of this would be considered by a reasonable reviewer (in my opinion) to constitute required experiments for publication. Without knowing the order they were done in, it is impossible to say for sure, I suppose.

    So, either this is a Frankenbeast paper where all the authors were trying to be kept as co-first, and so data was jammed in that might not have been robust. Which would be a failure of mentoring. Or, they had a finicky reviewer who demanded experiments (read: results) that weren't necessary or particularly edifying and data was massaged to fit it. Or, the accuser is bitter and the allegations are unfounded.

    All of these could be caused by the toxic idolatry we've constructed around the CNS journals. I agree.

    But I fail to see what *Cell* did wrong here.

    DM, what would you have done if you were the editor?

  • drugmonkey says:

    What further action would have them take at Cell if not retraction?

    I am focused on their deployment of this particular excuse for not taking further action. Look, the journal could very well have just stiff armed the guy who said he faked data. They did not. They could have accepted the Baylor finding of an inconclusive investigation. They did not. They chose to recruit actual real-live experimentation and, from context, to use the successful replication as some sort of decision criterion. They should not do this.

    it just points to mechanism of action, which is amply demonstrated by additional means (genetic and molecular).

    Right. Because Glam never ever hinges on demands for demonstrating mechanism on the strength of multiple converging lines of evidence.

    none of this would be considered by a reasonable reviewer (in my opinion) to constitute required experiments for publication.

    Right. Because nobody ever complains about how Glam reviewers or editors demand more and more experiments to prove what the authors think has been sufficiently demonstrated, just because they can.

    either this is a Frankenbeast paper where all the authors were trying to be kept as co-first, and so data was jammed in that might not have been robust.

    Right. Which doesn't describe at least half of Glam papers..... #basketofFrankenpapers

    But I fail to see what *Cell* did wrong here.

    They acted as though successful later replication of a result is somehow relevant to the decision of whether a paper published with faked data should remain published.

    DM, what would you have done if you were the editor?

    I would have made my decision without any consideration of whether the faked data later turned out to be replicable or not.

  • jmz4 says:

    "Right. Because nobody ever complains about how Glam reviewers or editors demand more and more experiments to prove what the authors think has been sufficiently demonstrated, just because they can. "
    -So maybe this practice is the more important one to end when it comes to reducing fraud.

    "They chose to recruit actual real-live experimentation and, from context, to use the successful replication as some sort of decision criterion. They should not do this. "
    -Did it say anywhere in the statement that they recruited them? They published the statement when the allegations were sent to them, and two (presumably PI-sympathetic labs) later send them corroborating evidence, which they then duly noted in the header to the paper.
    I'd argue they should have made the evidence public, but otherwise this seems more responsible than any alternative.

    As Qaz pointed out, the real bee in everyone's bonnet is that Glam and novelty are so idolized that everyone is itching to find reasons to indulge in righteous retribution on the blasphemers. But this misses the point of what a scientific paper is supposed to be, a report of useful and accurate information.

    "I would have made my decision without any consideration of whether the faked data later turned out to be replicable or not."
    I agree with the ethical/philosophical point that scientific fraud should not be forgiven because the fraud pointed in the right direction. But:
    **
    It is not the journals responsibility to mete out retractions as a form of punishment(&). Only someone that buys into papers as career accolades would accept that. The journal is there to disseminate accurate scientific information. If the journal has evidence that, despite the complaint, this information is accurate,(%) then it *absolutely* should take that into account when deciding to keep a paper out there.

    (&) Otherwise we would retract papers from leches and embezzlers. We don't.

    (%) In this case, there seems to be substantial evidence (2nd and 3rd party corroboration) that no actual fraud occurred.
    Of course, if the alleged fraud is indistinguishable in outcome from a properly run experiment, this is nearly impossible to prove.

  • drugmonkey says:

    It is not the journals responsibility to mete out retractions as a form of punishment(&).

    Prevention, not punishment. and it most certainly is their responsibility.

    Otherwise we would retract papers from leches and embezzlers. We don't.

    These personal characteristics do not pertain as obviously to the contingencies that encourage or discourage the community of science when it comes to submitting fradulent or nonfraudulent data for consideration for publication.

    In this case, there seems to be substantial evidence (2nd and 3rd party corroboration) that no actual fraud occurred.

    What? The one dude says he faked data. His supervising PI says he didn't. The Uni says a preliminary investigation was inconclusive. This is not "substantial evidence" one way or the other nor any 2nd or 3rd party anything.

    Of course, if the alleged fraud is indistinguishable in outcome from a properly run experiment, this is nearly impossible to prove.

    Why is this so difficult for you to understand? It makes zero difference if the outcome is consistent with the original result or not. A successful replication (actually the last notice claims "their data largely confirm the central conclusions drawn from the original figures" which you assert was already the case from the converging evidence in the original, but whatever) and an unsuccessful replication from properly run experiments are of equal value here. Neither outcome can authoritatively tell us if the original data were faked or not.

  • DJMH says:

    I don't get all the complaints about "but people will be hurt!" if Cell retracts. Yeah, so they can ditch the fake-o figures, and/or redo them honestly, and send it to another journal. Or break it into the 4 papers of each of the individual first authors, and let them enjoy their sole first author paper. Or whatever.

    These people were gaining advantage by fraudulence, knowingly or not; it's ok to have consequences for them. All of them.

  • […] jmz4 made a fascinating comment on a prior […]

  • Pinko Punko says:

    I don't read this PARTICULAR Editorial Note as quite as nefarious. However, if they are to claim that work is repro, they should show that. The claim that they cannot adjudicate allegations and denials is what it is, and given Editorial Expression of Concern, that is sort of reasonable unless investigation performed. However, if they desire to close their books on the possible fraud (presumably notwithstanding future investigations) using third party evidence to support claims of PI that fraud was not committed- yeah, that has no bearing on whether fraud occurred. It does have bearing on how polluted the literature is regarding scientific ideas. If they wish to claim the ideas are still valid, they need to publish those third party data.

  • Draino says:

    If the data were indeed faked, then I agree the paper should be retracted regardless of independent replication. The buck stops with the journal and they need to step up and do the right think.

    However, it seems in this particular case that it is not clear cut that the data was faked. The postdoc says he faked it, but he has reasons to lie. The PI says it wasn't faked, but he too has reasons to lie. So what is the journal to do?

    Well, in this case the journal did a weird thing that distracted from the original issue. They had other labs replicate the experiment. Other labs found that the mechanism was reproducible, which *I guess* is more consistent with the PI's claim that the data wasn't faked. But it certainly doesn't rule it out. If the mechanisms were not reproducible there would be new reasons for retraction, despite the muddled he-said-she-said going on between PI and ex-postdoc. But the other labs got positive results. So the journal has to decide based on the original unclear facts of the case.

    DM thinks the case is clear. When in doubt: retract.

    I disagree. There's a lab at my institute that is hounded by a cranky ex-postdoc who practices exactly these shenanigans. He has contacted journals and tried to have his papers retracted, to damage his old lab and the PI he believes wronged him. I don't think journals should retract every time a disgruntled employee tries to blackmail or exact revenge on his old boss. This means there has to be more evidence than he-said-she-said.

  • Ola says:

    The solution is really quite simple. Just fire Emilie Marcus's ass!
    SRSLY, she needs to get with the times. First she publishes an editorial asking for readers' help in dealing with this issue, then she allows this shit to happen. WTAF? Elsevier should remove her before she does any more damage to their shining reputation 😉

  • I agree with DM here--whether the faked data is reproducible should have no bearing on what Cell does with the paper. A good fake is still a fake. Sometimes, our scientific intuition is correct, but papers are supposed to be based on real data, not stories we make up to support our intuition. Personally, I think that if the paper was published with faked data, it should be retracted.

  • DJMH says:

    There's a lab at my institute that is hounded by a cranky ex-postdoc who practices exactly these shenanigans. He has contacted journals and tried to have his papers retracted, to damage his old lab and the PI he believes wronged him. I don't think journals should retract every time a disgruntled employee tries to blackmail or exact revenge on his old boss.

    Um. If the ex-postdoc doesn't want his data in a manuscript, the PI should not be trying to publish it without his consent. There's a reason for those forms that all the authors have to sign. The ex-postdoc, no matter how batshit crazy or disgruntled, is within his rights to do this.

    So yes, the Cell paper should be retracted in its entirety. You choose to lie down with 20 authors, it shouldn't be a surprise when one of them has fleas.

  • Pinko Punko says:

    DJMH- the anecdote seems to refer to after the fact gambit to damage lab reputation

  • drugmonkey says:

    Other labs found that the mechanism was reproducible, which *I guess* is more consistent with the PI's claim that the data wasn't faked.

    This is false and it is disturbing that scientists believe this.

    DM thinks the case is clear. When in doubt: retract.

    Nope. Not my position on this at all.

    a cranky ex-postdoc who practices exactly these shenanigans. He has contacted journals and tried to have his papers retracted, to damage his old lab and the PI he believes wronged him.

    Absolutely true that there is no reason, a priori, to believe one side over the other.

    You choose to lie down with 20 authors, it shouldn't be a surprise when one of them has fleas.

    Ah. Nice to know the precise ratio to avoid when it comes to multi-author collaborations!

  • DJMH says:

    Nice to know the precise ratio to avoid when it comes to multi-author collaborations!

    Note how I chose 20. Scientists are good people, p < 0.05. 🙂

  • jmz4 says:

    "I agree with DM here--whether the faked data is reproducible should have no bearing on what Cell does with the paper. A good fake is still a fake"
    -At issue here (in this thread) is whether the data was faked or not.
    Cell can't prove an impossibility: that the data was not faked. The most we can show is that the data is consistent with real data, so if it quacks like real data, it is more likely to be real data.

    "Other labs found that the mechanism was reproducible, which *I guess* is more consistent with the PI's claim that the data wasn't faked.

    This is false and it is disturbing that scientists believe this. "
    -It is not false when deciding whose story is more probable; and it is *more* consistent with the PI's story being true than not.
    The PD is saying he faked the data, the PI is saying he is only claiming that in order to hurt her lab. The independent tests show that the data in question is "correct" which eliminates the possibility that the PD took a legitimate negative results and made them positive. This also removes a natural motive for the initial faking of the results, i.e. to satisfy reviewers by giving them an experimental result that could only be obtained fraudulently.
    Sure, the PD could have failed to have gotten the results for technical reasons, or he could have invented them from whole cloth, but one possible explanation has been removed. Hence, his story is less likely, especially if he claimed in his complaint to Cell that he *had* taken negative data and turned it into positive data.

    I think Cell should provide all this information (complaint, rebuttal, and independent data) up in front of the paper, and that this should be standard practice.

  • Poke says:

    "The independent tests show that the data in question is "correct" which eliminates the possibility that the PD took a legitimate negative results and made them positive. "

    Yeah, because the same experiment run by different people never comes out in different ways...

  • drugmonkey says:

    -At issue here (in this thread) is whether the data was faked or not.

    I disagree. The issue at hand is what we do when there is an accusation of faked data.

    The most we can show is that the data is consistent with real data, so if it quacks like real data, it is more likely to be real data.

    This is absolutely 100% wrong.

    It is not false when deciding whose story is more probable; and it is *more* consistent with the PI's story being true than not.

    It is becoming clear you are dangerously wrong in the way you think about scientific data and experimental outcomes.

    which eliminates the possibility that the PD took a legitimate negative results and made them positive.

    How do you come to this conclusion?

    This also removes a natural motive for the initial faking of the results, i.e. to satisfy reviewers by giving them an experimental result that could only be obtained fraudulently.

    There are many experimental outcomes for which a given scientist is unable to replicate it but several others can and have done so. People do not fake data *only* because it *cannot* be generated any other way. Misrepresenting the source of a cut and diced and sliced blot image is commonly at the root of convicted data fakery.

    one possible explanation has been removed. Hence, his story is less likely

    This does not follow. At all.

  • jmz4 says:

    "Yeah, because the same experiment run by different people never comes out in different ways..."
    In this case, as far as we know, all the attempts at the experiment have ended the same way.

    @DM

    I think I understand your point. You're saying it is just as likely (or more?) that a "true" result would be faked, and so the validation is irrelevant to the accusation of fakery. And fakery, not correctness should be the standard for retraction decisions? Are you saying this independent validation is irrelevant no matter the details of the accusation?. I.e there is no way the independent corroborations disprove the accusation of fakery?

  • jmz4 says:

    "It is becoming clear you are dangerously wrong in the way you think about scientific data and experimental outcomes."
    -i think it speaks more to my assumptions about the content of the accusation. In my reading, it seemed falsifiable, but I think I see your point now.

  • drugmonkey says:

    Are you saying this independent validation is irrelevant no matter the details of the accusation?. I.e there is no way the independent corroborations disprove the accusation of fakery?

    Yes.

Leave a Reply