[Blatt] then adds this beauty…
“So, whatever the shortfalls of the peer-review process, I do not accept the argument that it is failing, that it is a threat to progress, or that, as scientists, we need to retake control of our profession. Indeed, if there is a threat to the scientific process, I would argue that, unchecked, the most serious is the brand of vigilante science currently facilitated by PubPeer.”
So let’s get this straight – the problems facing science today are not: (i) a lack of funding, (ii) rampant fakery, (iii) politicians seeking to defund things they don’t like, (iv) inadequate teaching of the scientific method in schools, (v) proliferation of the blood-sucking profiteering publishing industry, (vi) an obsession with impact factor and other outdated metrics, (vii) a broken training to job pipeline in academia, (viii) insert your favorite #scipocalypse cause here.
I had a few issues with Blatt's comments that did not get addressed by Brookes.
I concur with Hilda Bastian,3 who notes, on the one hand, the lack of reliable evidence to support the benefits of reviewer anonymity and, on the other, the importance of assessing whether commenters are “outside their areas of expertise … [or] have conflicts of interest.” Anonymity can conceal much mischief and do great damage. Even PubPeer acknowledges that “anonymity does allow low quality and bad faith comments to be made with impunity”
Ok, so first he mentions a lack of evidence to prove the benefits of anonymous peer-review. Which is in the territory of trying to prove a negative. Right? The main benefit of making peer-review anonymous is to get comments that are honest and un-modified by any threat of retaliation against the one offering honest, but critical, commentary that may irritate the authors. It is almost impossible to know, particularly in an opt-in reviewer selection system, the extent to which an open peer review system has lost honesty of review. It is likely to be hard to detect.
On the other hand, Blatt is making an assertion of a positive ("mischief" and "great damage") without any shred of evidence in support of the existence or scope of any such problem. Nor is he comparing the rate of mischief that is worked because of anonymity versus the rate of mischief that is worked through a sense of impunity that is associated with having a position of power in the field.
Hey, since we're making assertions and all, Editor Blatt, here's mine. "Positions of relative power in science can conceal much more mischief, and facilitate damage to science and careers alike on a grander scale, than anonymous comments ever have or will."
Wheee! Aren't data-free assertions fun, Dr. Blatt?
The second set of comments are rather bizarre and I invite your input, Dear Reader. Blatt writes:
However, the argument for anonymity in postpublication discussion fallaciously equates such discussion with prepublication peer review. The essence of prepublication review is a mutual agreement, a social contract between author and editor, who are known to one another. The author submitting a manuscript for review agrees to accept the judgment of the editor, and the editor agrees to judge the worthiness of the manuscript for publication. To make this assessment, the editor may obtain independent reviews of the author’s work before passing judgment, but (and here is the fundamental flaw in the argument for “post publication peer review” as championed by PubPeer) the reviewers and their anonymity are secondary to the contract between author and editor. In short, anonymity makes sense when reviews are offered in confidence to be assessed and moderated by an editor, someone whose identity is known and who takes responsibility for the decision informed by the reviews. Obviously, this same situation does not apply postpublication, not when the commenters enter into a discussion anonymously and the moderators are also unknown.
I agree that technically, the peer review of manuscripts is merely advisory to the Editor. So, from one viewpoint, he is not wrong here. But I assert that the process of peer review has become something different in practice. At least for many scientists and many editorial teams.
I feel as though I am contracting not merely with the AE and EIC of the journal but with reviewers to whom they send the manuscript for review. We are in a social agreement that they will stand in as proxy for my desired audience of eventual readers and provide honest commentary on the strengths and weaknesses of our work. It is not blind trust. There is variance in the system and I recognize that I will not always agree with reviewers.The reviewers may not agree with each other. And the AE or EIC may not agree with any or all.
It is a semi-chaotic agreement but an agreement nevertheless. I feel as though authors have an expectation of their peers to participate in the process and an expectation of Editors to perform their role.
I see no reason why it makes sense to Editor Blatt that only "someone whose identity is known" can be seen to "take(s) responsibility for the decision informed by the reviews". Or why it is needed that "responsibility" be taken.
As Brookes observed in response to Blatt's comments to the effect that nattering about gels and blots was unimportant:
The way that “real” scientists respond when their data is questioned, is to answer the damn question! Show the data. Produce the originals. In case you hadn’t noticed, the front page of PubPeer cycles once every 3-4 days – if there’s an innocent explanation, you WILL be vindicated and your career will not end if you engage with the commenters.
It doesn't matter that any one individual take responsibility for a question or criticism! Other scientists will have those thoughts as well. And if they are venal mischief, how can they do great damage if they are wrong? If anything this would cause less mischief because a broader public could evaluate the comment. In contrast the power of anonymous peer reviewers to do great mischief by misleading an Editor is much, much greater.
The arguments being made by Blatt's Editorial just don't make consistent sense to me.