Bias at work

A piece in Vox summarizes a study from Nextions showing that lawyers are more critical of a brief written by an African-American. 

I immediately thought of scientific manuscript review and the not-unusual request to have a revision "thoroughly edited by a native English speaker". My confirmation bias suggests that this is way more common when the first author has an apparently Asian surname.

It would be interesting to see a similar balanced test for scientific writing and review, wouldn't it?

My second thought was.... Ginther. Is this not another one of the thousand cuts contributing to African-American PIs' lower success rates and need to revise the proposal extra times? Seems as though it might be. 

22 responses so far

  • A. Tasso says:

    I am Asian American and I _often_ request for a manuscript to be reviewed by member of the authorship team whose first language is English, or by a professional editing service. Nothing annoys me more greatly than a manuscript submission that is rife with errors of grammar, improper use of articles, errors of tense, etc. Of all the things that could be easily checked off of a pre-submission checklist, "make sure the manuscript is written properly" is one of the easiest things to correct.

  • jmz4 says:

    "It would be interesting to see a similar balanced test for scientific writing and review, wouldn't it?"
    Yes, given how crazy reviewers on study section get about typos (as evidenced by vociferous commenters on this blog), I could see this being a germane to the Ginther report.
    Another interesting report from that same Nextations place notes the insidious effect of (presumed) affirmative action hiring and the taint it leaves on applicants. I could see that being relevant to a minority working at a prestigious institution and not getting the full benefit of the institution due to a presumption of affirmative action hiring.

    Still, no error bars or statistics in the Vox article or the original study release...
    kinda shady.

  • Marina Picciotto says:

    An argument for double-blind review?

  • Recommending professional editing is appropriate, but specifically a native speaker is not.

  • Jonathan Badger says:

    Are we not to *supposed* to point out terrible grammar in manuscripts? Would it be better for things to get published with bad grammar and/or incorrect word usage? I recently reviewed a paper where the authors confused the verb "to shed" with "to shred". As in some cases the sentence actually would make (factually incorrect) but plausible sense, this isn't being merely pedantic.

  • Grumble says:

    "An argument for double-blind review?"

    Maybe. But reviewers' comments about the manuscript needing the attention of an English editor are usually justified (in my experience). Which means that it would be easy for blinded reviewers to ascertain that the authors are foreigners and possibly even localize them to a specific region (Asia, Eastern Europe, ...)

    Now, if there is bias even when the English is perfect, that's another story. But I am not convinced that this sort of bias is pervasive. A study needs to be done. Give two randomly chosen groups of scientists the same paper to review. One group gets a paper with Asian names as authors, the other gets Euro/American names. Then see if the scores differ. This can actually be done objectively because most journals ask for some sort of ranking on the Accept - Minor Changes - Major Changes - Reject scale.

    Sounds like the sort of experiment the editor-in-chief of a major Neuroscience journal might consider doing...

  • jojo says:

    "Are we not to *supposed* to point out terrible grammar in manuscripts? Would it be better for things to get published with bad grammar and/or incorrect word usage?"

    Whoosh.

    Yes you should point out the shit that is wrong or needs clarification/rephrasing but saying "please have someone who is a native English Speaker edit this" is making (often racially biased) implications about the author(s) native language.

    I actually caught myself almost about to do this once and yep, the authors were Chinese. I modified the review to get rid of such statements while leaving in the gist that they needed to take care of the issues.

  • Jonathan Badger says:

    I worked for a time at a biotech firm in Montreal where the working language was French (not my native language). Needless to say, I made quite a few grammar mistakes when I made reports in French, and people could certainly tell it wasn't my native language. It's pretty obvious that certain manuscripts aren't written by native speakers -- there's no real "implication" to be made.

  • jojo says:

    Ask yourself what exactly is supposed to be the added benefit of using a statement like this, as opposed to a more neutral one?

    If in fact they are not native speakers, they will understand that was the root of the grammar issues. As non native speakers, they are highly attuned and sensitive to that fact, just as you say you were about French. Pointing it out adds exactly nothing except the opportunity for bias.

  • Anonymouse says:

    Just make the proposal review process as blind as possible. Hide the personnel's names and institutions. Let the reviewers judge ONLY the idea/hypothesis, its significance and the proposed approach. After the proposals are reviewed and ranked, let the Program Officer validate (pass/fail only) the environment, facilities and PI's credentials.

    But, of course, nobody is going to entertain such an outrageous idea. Panel members love to play their little funding politics, based on various biases.

  • drugmonkey says:

    An argument for double-blind review?

    Arguments for double-blind review are the same as arguments for magic unicorn leprechaun gold funding for the NIH.

    Are we not to *supposed* to point out terrible grammar in manuscripts?

    Sure, but in this context the question is whether such errors interfere with the ability to judge the science itself in an unbiased manner. I bet that it does. Didn't we just do a thing on people who say that too many typos make them assume the science is sloppy too?

    reviewers' comments about the manuscript needing the attention of an English editor are usually justified (in my experience).

    If I had a $25,000 NIH Grant module for every time a reviewer found the two typos in my grant app or manuscript and said "this thing is riddled with errors and needs a comprehensive proofreading", I would be a happy PI. My recollection is that non-native English in manuscript review is not terribly dissimilar, particularly when you figure that the ticky-tack small stuff is really in the hands of the copy editor. My threshold is whether or not the phrasing interferes with understanding.

    But I am not convinced that this sort of bias is pervasive. A study needs to be done.

    Yeah, well that Nextions outfit did one. I see no reason that our null for scientific manuscript review isn't that there is bias, needs to be proved otherwise.

  • Anonymouse says:

    DM's past post is relevant to double-blind review:
    http://drugmonkey.scientopia.org/2012/04/18/double-blind-paper-and-grant-review-cant-work/

    I see no problem though. Proposals should consist only of the following sections: (1) Idea/hypothesis, plus a review of the present state of the knowledge. (2) Significance. (3) Proposed approach.

    That's all. These three components are enough to judge if the proposal makes sense and should be funded, and they do not disclose anything that would allow for reliably identifying the grantee. Well, if underdogs get mistaken for some big guys, all the power to them.

  • A Salty Scientist says:

    Unless the journal does not use a copy editor, I do not point out minor typos. I will ask the authors to revise places that are difficult to understand. This is sometimes due to poor grammar, but more often due to a lack of precision and clarity in the writing. If the entire manuscript was poorly written, I can understand recommending a manuscript be reviewed by a professional editor. However, requesting that the manuscript be reviewed by a "native speaker" is presumptuous and wholly inappropriate.

  • jmz4 says:

    "I see no problem though. Proposals should consist only of the following sections: (1) Idea/hypothesis, plus a review of the present state of the knowledge. (2) Significance. (3) Proposed approach."
    -Feasibility(e.g. research history) and institutional environment are big parts of the score though, right? Would it be it be a good idea to offload those judgements to a PO?

  • bacillus says:

    I've had the "native English speaker" review twice from ~ 100 papers. The last time, I wrote back saying that although I'm a Liverpudlian, I still consider myself a native English speaker. It's not like I write in scouse (Liverpool dialect)! I've always assumed that such comments come from Ivy League Grammar Nazi reviewers just to make the recipients feel inferior. In the UK at least snobbishness was still the rule in science when I started my career. Best choice I ever made was coming to North America where people judge me on what I say, not by the accent with which I say it.

  • Juan Lopez says:

    Anonymouse, so anyone would be able to propose approaches using any technique and equipment? It would not be possible or appropriate to find out if they actually have the equipment, or if they have already spent the multiple years needed to find-tune the methods.

    I agree with jmz4, feasibility is a big deal.

  • Grumble says:

    'ey, bacillus, ay think yous should tertally submit yer next paper in scouse!

  • drugmonkey says:

    JL- maybe there could be two stage review. Ideas first, feasibility afterwards.

  • drugmonkey says:

    And, warming to the theme, maybe it could be JIT-style where you cough up the feasibility prelim data only after the Big Idea looks like it is in contention.

  • bacillus says:

    @ grumble. Ta la, irrle get a lorra laffs if i doz it like tha

  • Anonymouse says:

    "Feasibility(e.g. research history) and institutional environment are big parts of the score though, right? Would it be it be a good idea to offload those judgements to a PO?"

    Why not? It's a pass/fail thing. Either somebody has the necessary facilities, or not. As for the soft factors, well, that's the point - I want to hide these things from the reviewers to eliminate the biases.

  • Eli Rabett says:

    It's a fair thing to say if the paper comes from a country where the language it is written in is rare. Also, with modern economics, printing technology and fast turn around there are no copy editors.

    Less than ten errors Eli will point out. More, and at a minimum a resubmission is needed

Leave a Reply