Archive for the 'Science Publication' category

When reviewers can't relinquish their bone

Had an interesting category of thing happen on peer review of our work recently.

It was the species of reviewer objection where they know they can't lay a glove on you but they just can't stop themselves from asserting their disagreement. 

It was in several different contexts and the details differed. But the essence was the same. 

I'm just laughing.

I mean- why do we use language that identifies the weaknesses, limits or necessary caveats in our papers if it doesn't mean anything?

Saying "...and then there is this other possible interpretation" apparently enrages some reviewers that this possibility is not seen as a reason to prevent us from publishing the data. 

Pointing out that these papers over here support one view of accepted interpretations/practices/understanding can trigger outrage that you don't ignore those in favor of these other papers over there and their way of doing things. 

Identifying clearly and carefully why you made certain choices generates the most hilariously twisted "objective critiques" that really boil down to "Well I use these other models which are better for some reason I can't articulate."

Do you even scholarship, bro?

I mostly chuckle and move on, but these experiences do tie into Mike Eisen's current fevers about "publishing" manuscripts prior to peer review. So I do have sympathy for his position. It is annoying when such reviewer intransigence over non-universal interpretations is used to prevent publication of data. And it would sometimes be funny to have the "Your caveats aren't caveatty enough" discussion in public.

9 responses so far

Blooding the trainees

In that most English of pastimes, fox hunting, the noobs are smeared about the face with the blood of the poor unfortunate fox after dismembering by hound has been achieved.

I surmise the goal is to get the noob used to the less palatable aspects of their chosen sporting endeavor. 

Anyway, speaking of manuscript review and eventual publication, do you plan a course for new trainees in the lab?

I'm wondering if you have any explicit goals for them- Should a mentor try to get new postdocs or grads a pub, any pub as quickly and easily as possible?

Or should they be thrown into a multi-journal fight so as to fully experience the joys of desk rejection, ultimate denial after four rounds of review somewhere and the final relief of just dumping that Frankensteinian monster of a paper in a lowly journal and being done. 

Do you plan any of this out for your newest trainees?

19 responses so far

#icanhazpdf and related criminal behavior

I was slow to start watching "Better Call Saul" for various reasons. Partially because I still haven't finished "Breaking Bad", partially because I couldn't see *that* as being the spinoff character and partially because I just hadn't gotten around to it. Anyway, the show is about a lawyer who we know from BB becomes deeply involved with criminal law.

There's a point in Season 1 where one character has a heart to heart with another character about the second person's criminal act.

"You are a criminal."

He then goes on to explain that he has known good guy criminals and a bad guy cops and that at the end of a day, committing a crime makes you a criminal.

Anyway, dr24hours has some thoughts for those criminal scientists who think they are good guys for illegally sharing PDFs of published journal articles.

22 responses so far

Manuscript acceptance based on perceived capability of the laboratory

Dave asked:

I think about it primarily in the form of career stage representation, as always. I like to get reviewed by people who understand what it means to me to request multiple additional experiments, for example.

and I responded:

Are you implying that differential (perceived/assumed) capability of the laboratory to complete the additional experiments should affect paper review comments and/or acceptance at a particular journal?

I'm elevating this to a post because I think it deserves robust discussion.

I think that the assessment of whether a paper is 1) of good quality and 2) of sufficient impact/importance/pizzazz/interest/etc for the journal at hand should depend on what is in the manuscript. Acceptance should depend on the work presented, for the most part. Obviously this is were things get tricky because there is critical difference here:

This is the Justice Potter Stewart territory, of course. What is necessary to support and where lies the threshold for "I just wanna know this other stuff"? Some people have a hard time disentangling their desire to see a whole 'nother study* from their evaluation of the work at hand. I do recognize there can be legitimate disagreement around the margin but....c'mon. We know it when we see it**.

There is a further, more tactical problem with trying to determine what is or is not possible/easy/quick/cheap/reasonable/etc for one lab versus another lab. In short, your assumptions are inevitably going to be wrong. A lot. How do you know what financial pressures are on a given lab? How do you know, by extension, what career pressures are on various participants on that paper? Why do you, as an external peer reviewer, get to navigate those issues?

Again, what bearing does your assessment of the capability of the laboratory have on the data?

*As it happens, my lab just enjoyed a review of this nature in which the criticism was basically "I am not interested in your [several] assays, I want to see what [primary manipulation] does in my favorite assays" without any clear rationale for why our chosen approaches did not, in fact, support the main goal of the paper which was to assess the primary manipulation.

**One possible framework to consider. There are data on how many publications result from a typical NIH R01 or equivalent. The mean is somewhere around 6 papers. Interquartile range is something like 3-11. If we submit a manuscript and get a request to add an amount of work commensurate with an entire Specific Aim that I have proposed, this would appear to conflict with expectations for overall grant productivity.

26 responses so far

Representation in manuscript review

Feb 25 2016 Published by under Diversity in Science, Science Publication

One of the things that determines success in science careers is the opinion ~three peer reviewers have about your manuscript as offered up for publication in a given journal.

Hopefully I do not have to rehash the way that journal identify of a scientist's published work affects career success.

Hopefully I do not have to rehash the way that bias creeps into what otherwise is supposed to be objective analysis.

And let us leave your well-intentioned, but hopelessly naive calls for blinded peer review aside until that nirvana is reached.

Do you think about reviewer diversity at all? Many journals publish a year-end list of all reviewers (these don't say how many each reviewer wrote, of course). Have you ever scanned them for, say, gender balance? If you are an AE or EIC....does diversity* concern you?

On the author side, would you work to ensure your suggestions for potential reviewers are not biased? Do you ask for about as many women as men? Does ethnic or other minority characteristic of your suggestions play a role?

I'm guessing the answer is no?

I have taken to trying to suggest equal numbers of male and female reviewers when I submit a manuscript. This is pretty simple in my fields of work, so long as you think about it.

Other forms of representation? Not really possible, is my first thought. I'm thinking about it. Maybe I'll put a few people on my usual lists that I do not typically consider.

And when I get a chance I'm going to go through those published reviewer lists. I'm curious how the journals I think of as being in my field are doing.

*Editorial boards are another place to look, those are published.

39 responses so far

Second Thought on Glamour Pr33p P33ple

Feb 18 2016 Published by under Conduct of Science, Science Publication

If establishing the priority of scientific observations or findings is so important, another thing these people should be doing, tomorrow, is to cite conference abstracts in their papers.

It was not so long ago in the neurosciences that citations of Society for Neuroscience Abstracts would appear in archival reports.

We can return to this and it would go a long way towards documenting the chronology (I am working up an antipathy to "priority", folks) of an area of scientific work.

13 responses so far

Thought on Glamour Pr33p P33ple

Feb 18 2016 Published by under Conduct of Science, Science Publication

I do not know why they don't just submit their stuff to JIF 3 journals.

Everything would be "accept, no revisions and can we get you a coffee Professor?"

All their supposed problems would be solved.

26 responses so far

Thought of the Day

Feb 04 2016 Published by under Careerism, Science Publication

We always seem to go through a run of manuscript rejections and then a run of accepts. I wish we could figure out how to spread these out a little bit more. Would alternation be too much trouble, world?

3 responses so far

Appealing manuscript rejections

Feb 04 2016 Published by under Science Publication, Scientific Publication

This is going to be another one of those posts where we mix up what should be so, what is so and what is best for the individual scientist's career.

Our good blog friend iBAM was musing about reading a self-empowerment book.

The career prescription here...I agree with.

It took me quite awhile to realize that you can appeal when a journal editor rejects your manuscript. I thought a reject was a reject when I first started in this business. It certainly always read like one.

It turns out that you can appeal that decision.

And you should. The process does not end with the initial decision to reject your manuscript.

So what you do is, you email the Editor and you outline why the decision was a mistake, what you can do to revise the manuscript to address the major concerns, why your paper would be a fit for the journal, etc.

You may be ignored. You may be told "Nope, it is still a rejection".

OR. You may be given the opportunity to resubmit a revised version of the manuscript as if it were a new one. Which of course you could do anyway but it would be likely to get hammered down if you have not already solicited the invitation to do so from the Editor.

So you will see here that with a simple email, you have given yourself another chance to get the paper accepted. So do it.

After this we get into the fun part.

Should you appeal each and every decision? I think my natural stance is no, you don't want to get a rep as a chronic whiner. But you know what? There are probably people in science who rebut and complain about every decision. Does it work for them? I dunno. And by extension to sports, you know that phenomenon of working the ref about bad calls, hoping to get a makeup call later? That logic maybe applies here.

I do know there are plenty of testimonials like those of Kaye Tye that complaining about a rejection ended up with a published manuscript in the initially-rejecting journal.

Do appeals work the same for everyone? That is, given the same approximate merits of the case is the Editor going to respond similarly to appeals from anyone? I can't say. This is a human decision making business and I have always believed that when an editor knows you personally, they can't help but treat your submissions a little better. Similarly when an editor knows your work, your pedigree, your department, etc it probably helps your appeal gain traction.

But who knows? Maybe what helps is having a good argument on the merits. Maybe what helps is that the Editor liked the paper and slightly disagrees with the review outcome themselves.

HAHAHA, I crack myself up.

Okay so where do we end up?

First, you need to add the post-rejection appeal into your repertoire of strategies if you don't already include it. I would use it judiciously, personally, but this is something to ask your subfield colleagues about. People who publish in the journals you are targeting. Ask them how often they appeal.

Second, realize that the appeal game is going to add up over time in a person's career. If there are little personal biases on the part of Editors (which there are) then this factor is going to further amplify the insider club advantage. And when you sit and wonder why that series of papers that are no better than your own just happen to end up in that higher rank of journal well.....there may be reasons other than merit. What you choose to do with that information is up to you. I see a few options.

  • Pout.
  • Up your game so that you don't need to use those little bennies to get over the hurdle.
  • Schmooze journal editors as hard as you can.
  • Ignore the whole thing and keep doing your science the way you want and publishing in lower journals than your work deserves.

*not a fan of this myself but some people (including one of my kids' soccer coaches) appear to believe very strongly in this theory.

26 responses so far

Thought of the Day

Feb 03 2016 Published by under Science Publication, Scientific Publication

Dear Editor of Journal,

I find it interesting to review the manuscripts of ours that you have rejected on impact and quality grounds* over the past several years. We quite naturally found publication homes elsewhere for these manuscripts, this is how the system works. No harm, no foul. In none of these cases, I will note, was the manuscript radically changed in a way that would fundamentally alter the review of quality or impact as reflected in your reviewer's comments. Yet I note that these papers have been cited in excess, sometimes far in excess, of your Journal's Impact Factor. Given what we know about the skew in citations distributions which contribute to a JIF, well, this positions our papers quite favorably within the distribution of manuscripts you chose to accept.

This suggests to me there is something very wrong with your review process insofar as it attempts to evaluate quality and predict impact.


*journal fit is another matter entirely. I am not talking about those complaints.

10 responses so far

« Newer posts Older posts »