Scientific peer review is not broken, but your Glamour humping ways are

I have recently had a not-atypical publishing experience for me. Submitted a manuscript, got a set of comments back in about four weeks. Comments were informed, pointed a finger at some weak points in the paper but did not go all nonlinear about what else they'd like to see or whinge about mechanism or talk about how I could really increase the "likely impact". The AE gave a decision of minor revisions. We made them and resubmitted. The AE accepted the paper.

Boom.

The manuscript had been previously rejected from somewhere else. And we'd revised the manuscript according to those prior comments as best we could. I assume that made the subsequent submission go smoother but it is not impossible we simply would have received major revisions for the original version.

Either way, the process went as I think it should.

This brings me around to the folks who think that peer review of manuscripts is irretrievably broken and needs to be replaced with something NEW!!!!11!!!.

Try working in the normal scientific world for awhile. Say, four years. Submit to regular journals edited by actual working peer scientists. ONLY. Submit to journals of pedestrian and/or unimpressive Impact Factor (that would be the 2-4 range from my frame of reference). Submit interesting stories- whether they are "complete" or "demonstrate mechanism" or any of that bullshit. Then submit the next part of the continuing story you are working on. Repeat.

Oh, and make sure to submit to journals that don't require any page charge. Don't worry, they exist.

Give your trainees plenty of opportunity to be first author. Give them lots of experience writing and allow them to put their own thoughts into the paper..after all, there will be many more of them to go around.

See how the process works. Evaluate the quality of review. Decide whether your science has been helped or hindered by doing this.

Then revisit your prior complaints about how peer review is broken.

And figure out just how many of them have more to do with your own Glamour humping ways than they do with anything about the structure of Editor managed peer-review of scientific manuscripts.

__
Also see Post-publication peer review and preprint fans

18 responses so far

  • And figure out just how many of them have more to do with your own Glamour humping ways than they do with anything about the structure of Editor managed peer-review of scientific manuscripts.

    Where does this assumption of non-orthogonality come from? I am in your camp with regard to manuscript peer-review, and do not consider it "broken". And I feel that way even about pretty much all of my experiences with so-called "glamour" journals.

    And BTW, how is going through peer review and getting rejected by a dump journal, revising and submitting to a sub-dump journal, and then being accepted after minor revisions any different from going through peer review at a "glamour" journal and getting rejected, revising and resubmitting to a high-impact field-specific journal, and then being accepted after minor revisions? Your obsession with the evils of "glamour" is really distorting your perception of reality.

  • drugmonkey says:

    Try it for four years and find out, genius.

  • Michael H says:

    Amen. The peer review system is no more broken than any other aspect of science. As I stated before, I submitted to glam when in academics as a lark-not because I assumed it would be accepted. My work was never going to be glam-popular.

    I am sure I could get posses about that, but I chose to move on, grow up, and submit papers to journals appropriate for the content. These journals not only gave great reviewers, but it is possible to have UG 1st authors and no need for high profile PI as a prerequisite for publication.

    I still don't understand how people can honestly feel slighted that EVERY single research idea they had is not simultaneously groundbreaking, controversial, and Nobel-worthy. It isn't.

  • rxnm says:

    Check it out: CPP thinks his experience with glam editors applies to others.

    The whole thing with glam is there preferential treatment of certain people, institutions, and aesthetic/stylistic categories of scientific presentation. (And other irrelevant categorizations.)

  • CPP sure has been being a douche lately, especially here.

    As someone who has some of the privilege but not all of the privilege I definitely see it both ways. When the editor knows me, things go a lot smoother than when not. It's definitely been easier as I get more known and as the reputation of my current institution grows. And I notice myself doing the same thing when I'm reviewing, especially for Glam journals. I try really hard to counter it, but I can't say that I don't give extra benefits of the doubt to Glam authors compared to unknowns, especially since until recently I was always trying to win a beauty contest, that is, trying to make sure that my R&R, reject, etc. matched what the other reviewers said. And the other reviewers and editors similarly respond. The world would be a lot better off with truly blind review.

    I prefer having privilege! But it's hard having that pesky second X chromosome and not being at a top 10 or even top 15 institution. I get the most ridiculous desk rejections sometimes, generally from the same guy who gives similarly ridiculous desk rejections to my female friends. If it weren't for those things, I'd definitely be agreeing with CPP here, oh, you just need to keep aiming high and not worry about getting scooped while you're under review because it'll hit or at least be rejected in a timely fashion, and if you're rejected inappropriately you can complain and they'll reconsider.

    And yes, I do need to start submitting things to Glam pubs, except that when I mention that to folks they say, "oh you'll know when a paper is good enough and you shouldn't submit unless you really think it belongs there." But that is also crap advice that they're giving because I'm not male, top 10 school. Except, it also isn't that crappy when there's all that lack of privilege working against you and the probabilities are rigged.

  • Eli Rabett says:

    OK, so Eli is not the brightest bulb in the valley, but he soldiers on, has a reasonable publication record and years ago figured out there was no way he was ever going to get anything into the Science, Nature, Phys Rev Lett valley when stuff he sent was not even put out for review but simply triaged by the assistant sub editor in charge of nothing. Nobunny has a chance in that bunch without a rabbi.

  • dr24hours says:

    Ding. Exactly.

  • AsianQB says:

    My grad lab used to publish regularly in Science, PNAS et al. When my first paper was being prepared - a methods-heavy paper with 8 figures and 4-6 panels each - I was all shiny-eyed at this being the best paper ever in the (sub-sub-sub) field. I asked my PI, "Is this PNAS-worthy yet?" He politely replied that this whole paper would be in the 'Supplementary Data' section of a PNAS level paper.

    I still wanted to submit, just for kicks, but the Assistant Summer Intern at the glam-mag would have rejected it outright. The paper was then published in AJP with minor corrections.

    Peer review is just fine for the vast variety of journals. But it stops there. Its not a fair playing field in the glams, yet we want to hump the glams, we get rejected, and then we complain about the process.

  • Lorax says:

    Publishing in 'glam' mags meant something more when I was coming through the ranks. With the advent of basically everything being available online and requirements from NIH and other government funded bodies US and elsewhere that publications be publicly accessible, these 'high impact' publications mean much less. If you do quality research, ask interesting questions, obtain solid data, and write well, then your work will be recognized and cited.

  • drugmonkey says:

    Christ, Lorax I was about to ask where you've been.... Glad you are back man. Hope all is going well.

  • Dr Becca says:

    @Lorax - You're right, visibility of one's work depends far less on it being in a glam journal than it used to. However, young scientists are still openly and explicitly judged on whether they publish in glam mags, so to say that high impact pubs mean much less is incorrect.

  • imager says:

    I guess if you are at the top end of the totem it doesn't really matter that much any more if it is the NSC glamour journal(s) - but from my perspective and many others here still climbing up I am afraid we are still judged to a big part by papers in exactly those journals...

  • imager says:

    Not to mention that it can make or break a newly minted PhD's job chances having one or none of those in his portfolio.

  • GAATTC says:

    I've published 15 papers in the last 4 years in solid journals (impact factors from 3-6). That was good enough to garner 2, 3 and 5 for investigator score on my last R01 application, with one reviewing saying that all the important work in my field was done by me and others 10 years ago. So glamour pubs still matter in my area, unfortunately.

  • drugmonkey says:

    Sorry but one grant review means nothing in isolation. IMO

  • Dave says:

    I am faculty at an R1 University. A very good place, but not glam central by any means. Yet, I have observed over the past decade that faculty position applicants without at least one glam pub never make it to the short list. You don't need to prove that you can publish glam papers every time, but you need to prove that you know how. I have also observed, as a grant reviewer, that people with glam pubs are given substantial benefit of the doubt. Will they be able to do what they propose? Maybe, maybe not, but their publication record shows that at the very least they'll do something pretty cool. And that's worth something. Finally, we publish because we want people to know what we learned, right? I think a lot of my papers in society-level journals are as good as, if not sometimes better, than my papers in glam journals. But a hell of a lot more people seem familiar with my papers in the glam journals. Glam journals get read by more people, and a greater variety of people.

    I hate the glamour-humping game too. But it's important to acknowledge reality.

  • Jane Gold says:

    Sorry ...... but a bunch of codgers - females and males alike - with saggy boobs and small penises and cellulite ridden butts, who cannot get their own lives together and are far from perfect and extremely biased, VOTING on and ALLOWING what gets published means NOTHING. It is not the arbiter of ANYTHING. It is BIASED OPINIONS. Science does not find "truth." Theories are removed from actual physical reality.

    LOTS of junk gets published every single day. LOTS of junk gets by the referees. There are ENORMOUS problems with peer-review............. It only meets the MINIMAL requirements. It only serves to keep out things like "coronary heart disease is caused by a singing green toaster" absurd type stuff.

    Studies show that peer-review does NOT improve the quality of journal articles.... Much quality work has been rejected. Much poor work got by....

    Get off your high horse. Science is NOT the only or best way of knowing for CERTAIN subjects.... For SOME things it is....

    Regards,
    Jane

  • Jane Gold says:

    Work that has been replicated- AND NOT ONLY THAT- BUT replicated using many VERY DIFFERENTand independent testing techniques and methods- all pointing in the same direction - ( developing many , many different methods and techniques to test something) THAT is the best indicator you are approximately correct- NOT peer -review............

Leave a Reply