Jun 11 2016

## NIH Director Collins and CSR Director Nakamura continue to kick the funding disparity can down the road

A News piece in Science by Jeffrey Mervis details the latest attempt of the NIH to kick the Ginther can down the road.

Armed with new data showing black applicants suffer a 35% lower chance of having a grant proposal funded than their white counterparts, NIH officials are gearing up to test whether reviewers in its study sections give lower scores to proposals from African-American applicants. They say it’s one of several possible explanations for a disparity in success rates first documented in a 2011 report by a team led by economist Donna Ginther of the University of Kansas, Lawrence.

Huh. 35%? I thought Ginther estimated more like a 13% difference? Oh wait. That's the award probability difference. About 16% versus 29% for white applicants which would be about a 45% lower chance. And this shows "78-90% the rate of white...applicants". And there was Nakamura quoted in another piece in Science:

At NIH, African-American researchers “receive awards at “55% to 60% the rate of white applicants,” Nakamura said. “That's a huge disparity that we have not yet been able to seriously budge,” despite special mentoring and networking programs, as well as an effort to boost the number of scientists from underrepresented minorities who evaluate proposals.

Difference vs rate vs lower chance.... Ugh. My head hurts. Anyway you spin it, African-American applicants are screwed. Substantially so.

Back to the Mervis piece for some factoids.

Ginther..noted...black researchers are more likely to have their applications for an R01 grant—the bread-and-butter NIH award that sustains academic labs—thrown out without any discussion...black scientists are less likely to resubmit a revised proposal ...whites submit at a higher rate than blacks...

So, what is CSR doing about it now? OK HOLD UP. LET ME REMIND YOU IT IS FIVE YEARS LATER. FIFTEEN FUNDING ROUNDS POST-GINTHER. Ahem.

The bias study would draw from a pool of recently rejected grant applications that have been anonymized to remove any hint of the applicant’s race, home institution, and training. Reviewers would be asked to score them on a one-to-nine scale using NIH’s normal rating system.

It's a start. Of course, this is unlikely to find anything. Why? Because the bias at grant review is a bias of identity. It isn't that reviewers are biased against black applicants, necessarily. It is that they are biased for white applicants. Or at the very least they are biased in favor of a category of PI ("established, very important") that just so happens to be disproportionately white. Also, there was this interesting simulation by Eugene Day that showed a bias that is smaller than the non-biased variability in a measurement can have large effects on something like a grant funding system [JournalLink].

Ok, so what else are they doing?

NIH continues to wrestle with the implications of the Ginther report. In 2014, in the first round of what NIH Director Francis Collins touted as a 10-year, $500 million initiative to increase the diversity of the scientific workforce, NIH gave out 5-year,$25 million awards to 10 institutions that enroll large numbers of minority students and created a national research mentoring network.

As you know, I am not a fan of these pipeline-enhancing responses. They say, in essence, that the current population of black applicant PIs is the problem. That they are inferior and deserve to get worse scores at peer review. Because what else does it mean to say the big money response of the NIH is to drum up more black PIs in the future by loading up the trainee cannon now?

This is Exhibit A of the case that the NIH officialdom simply cannot admit that there might be unfair biases at play that caused the disparity identified in Ginther and reinforced by the other mentioned analyses. The are bound and determined to prove that their system is working fine, nothing to see here.

So....what else ?

A second intervention starting later this year will tap that fledgling mentoring network to tutor two dozen minority scientists whose R01 applications were recently rejected. The goal of the intervention, which will last several months, is to prepare the scientists to have greater success on their next application. A third intervention will educate minority scientists on the importance of resubmitting a rejected proposal, because resubmitted proposals are three times more likely to be funded than a de novo application from a researcher who has never been funded by NIH.

Oh ff..... More of the same. Fix the victims.

Ah, here we go. Mervis finally gets around to explaining that 35% number

NIH officials recently updated the Ginther study, which examined a 2000–2006 cohort of applicants, and found that the racial disparity persists. The 35% lower chance of being funded comes from tracking the success rates of 1054 matched pairs of white and black applicants from 2008 to 2014. Black applicants continue to do less well at each stage of the process.

I wonder if they will be publishing that anywhere we can see it?

But here's the kicker. Even faced with the clear evidence from their own studies, the highest honchos still can't see it.

One issue that hung in the air was whether any of the disparity was self-inflicted. Specifically, council members and NIH officials pondered the tendency of African-American researchers to favor certain research areas, such as health disparities, women’s health, or hypertension and diabetes among minority populations, and wondered whether study sections might view the research questions in those areas as less compelling. Valantine called it a propensity “to work on issues that resonate with their core values.” At the same time, she said the data show minorities also do less well in competition with their white peers in those fields.

Collins offered another possibility. “I’ve heard stories that they might have been mentored to go into those areas as a better way to win funding,” he said. “The question is, to what extent is it their intrinsic interest in a topic, and to what extent have they been encouraged to go in that direction?”

Look, Ginther included a huge host of covariate analyses that they conducted to try to make the disparity go away. Now they've done a study with matched pairs of investigators. Valantine's quote may refer to this or to some other analysis I don't know but obviously the data are there. And Collins is STILL throwing up blame-the-victim chaff.

Dude, I have to say, this kind of denialist / crank behavior has a certain stench to it. The data are very clear and very consistent. There is a funding disparity.

This is a great time to remind everyone that the last time a major funding disparity came to the attention of the NIH it was the fate of the early career investigators. The NIH invented up the ESI designation, to distinguish it from the well established New Investigator population, and immediately started picking up grants out of the order of review. Establishing special quotas and paylines to redress the disparity. There was no talk of "real causes". There was not talk of strengthening the pipeline with better trainees so that one day, far off, they magically could better compete with the established. Oh no. They just picked up grants. And a LOT of them.

I wonder what it would take to fix the African-American PI disparity...

Ironically, because the pool of black applicants is so small, it wouldn’t take much to eliminate the disparity: Only 23 more R01 applications from black researchers would need to be funded each year to bring them to parity.

Are you KIDDING me? That's it?????

Oh right. I already figured this one out for them. And I didn't even have the real numbers.

In that 175 bin we'd need 3 more African-American PI apps funded to get to 100%. In the next higher (worse) scoring bin (200 score), about 56% of White PI apps were funded. Taking three from this bin and awarding three more AA PI awards in the next better scoring bin would plunge the White PI award probability from 56% to 55.7%. Whoa, belt up cowboy.

Moving down the curve with the same logic, we find in the 200 score bin that there are about 9 AA PI applications needed to put the 200 score bin to 100%. Looking down to the next worse scoring bin (225) and pulling these 9 apps from white PIs we end up changing the award probability for these apps from 22% to ..wait for it..... 20.8%.

Mere handfuls. I had probably overestimated how many black PIs were seeking funding. If this Mervis piece is to be trusted and it would only take 23 pickups across the entire NIH to fix the problem....

I DON'T UNDERSTAND WHAT FRANCIS COLLINS' PROBLEM IS.

Twenty three grants is practically rounding error. This is going to shake out to one or maybe three grants per year for the ICs, depending on size and what not.

Heck, I bet they fund this many grants every year by mistake. It's a big system. You think they don't have a few whoopsies sneak by every now and again? Of course they do.

But god forbid they should pick up 23 measly R01s to fix the funding disparity.

Apr 12 2016

## Bias at work

A piece in Vox summarizes a study from Nextions showing that lawyers are more critical of a brief written by an African-American.

I immediately thought of scientific manuscript review and the not-unusual request to have a revision "thoroughly edited by a native English speaker". My confirmation bias suggests that this is way more common when the first author has an apparently Asian surname.

It would be interesting to see a similar balanced test for scientific writing and review, wouldn't it?

My second thought was.... Ginther. Is this not another one of the thousand cuts contributing to African-American PIs' lower success rates and need to revise the proposal extra times? Seems as though it might be.

Apr 09 2016

## Thought of the Day

"Uppity" is a fascinating concept when it comes to NIH Grant award.

We know the sentiment applies to newer and younger investigators. I've heard countless review and Program Officer comments which amount to "let's not get too big for your britches, young un!" in my day.

I wonder how much of the Ginther effect is related to sentiments similar to "damn uppity [insert subvocalization]"?

Mar 10 2016

Feb 02 2016

## CSR Head Nakamura Makes Bizarre Pronouncement

An email from the CSR of the NIH hit late yesterday a few days ago, pointing to a number of their Peer Review Notes including one on the budget bump that we are about to enjoy.
Actually that should be "some selected few of us will enjoy" because

“While $2 billion is a big increase, it is less than a 10 percent increase, and a large portion of it is set aside for specific areas and initiatives,” said Dr. Nakamura. “Competition for funding is still going to be intense, and paylines will not return to historic averages . . . Yeah, as suspected, that money is already accounted for. The part that has me fired up is the continuation after that ellipsis and a continuing header item. So make sure you put your best effort into your application before you apply.” Counterproductive Efforts “We know some research deans have quotas and force their PIs to submit applications regularly,” said Dr. Nakamura. “It’s important for them to know that university submission rates are not correlated with grant funding. Therefore, PIs should be encouraged to develop and submit applications as their research and ideas justify the effort to write them and have other scientists review them.” As usual I do not know if this is coming from ignorance or calculated strategy to make their numbers look better. I fear both possibilities. I'm going from memory here because I can't seem to rapidly find the related blog post or data analysis but I think I recall an illustration that University-total grant submission rates did not predict University-total success rates. At a very basic level Nakamura is using the lie of the truncated distribution. If you don't submit any grant applications, your success rate is going to be zero. I'm sure he's excluding those because seemingly that would make a nice correlation. But more importantly, he is trying to use university-wide measures to convince the individual PI what is best for her to do. Wrong. Wrong. Wrong. Not everyone's chances at that institution are the same. The more established investigators will probably, on average, enjoy a higher success rate. They can therefore submit fewer applications. Lesser folk enjoy lower success rates so therefore they have to keep pounding out the apps to get their grants. By extension, it takes very little imagination to understand that depending on your ratio of big important established scientists to noobs, and based somewhat on subfields, the apparent University-wide numbers are going to swamp out the information that is needed for each individual PI. In short, this is just another version of the advice to young faculty to "write better grants, just like the greybeards do". The trick is, the greybeards DO NOT WRITE BETTER GRANTS! I mean sure, yes, there is a small experience factor there. But the major driver is not the objective quality but rather the established track record of the big-deal scientist. This gives them little benefits of the doubt all over the place as we have discussed on this blog endlessly. I believe I have yet to hear from a new-comer to NIH grant review that has not had the experience within 1-2 rounds of a reviewer ending his/her review of a clearly lower-quality grant proposal with "....but it's Dr. BigShot and we know she does great work and can pull this off". Or similar. I have been on a study section round or two in my day and I am here to tell you. My experience is not at all consistent with the idea that the "best" grants win out. Merit scores are not a perfect function of objective grant quality at all. Imperfectly prepared or boring grants get funded all the time. Really exciting and nearly-perfect grants get unfundable scores or triaged. Frequently. This is because grant review hinges on the excitement of the assigned reviewers for the essence of the project. All else is detail. You cannot beat this system by writing a "perfect" grant. Because it may not be perfect for all three reviewers no matter how well it has been prepared and how well vetted by whatever colleagues you have rounded up to advise you. Nakamura should know this. He probably does. Which makes his "advice" a cynical ploy to decrease submissions so that his success rate will look better. One caveat: I could simply be out of touch with all of these alleged Dean-motivated crap apps. It is true that I have occasionally seen people throw up grant applications that really aren't very credible from my perspective. They are very rare. And it has occasionally been the case that at least one other reviewer liked something about an application I thought was embarrassingly crappy. So go figure. I also understand that there are indeed Deans or Chairs that encourage high submission rates and maybe this leads to PIs writing garbage now and again. But this does not account for the dismal success rates we are enjoying. I bet that magically disappearing all apps that a PI submitted to meet institutional vigor requirements (but didn't really mean to make a serious play for an award) would have no perceptible effect on success rates for the rest of us. I just haven't ever seen enough non-credible apps for this to make a difference. Perhaps you have another experience on study section, DearReaders? Finally, I really hate this blame-the-victim attitude on the part of the CSR and indeed many POs. There are readily apparent and demonstrable problems with how some categories of PIs' grants are reviewed. Newer and less experienced applicants. African-American PIs. Women. Perhaps, although this is less well-explicated lately, those from the wrong Universities. For the NIH to avoid fixing their own problems with review (for example the vicious cycle of study sections punishing ESI apps with ever-worsening scores when the NIH used special paylines to boost success rates) and then blame victims of these problems by suggesting they must be writing bad grants takes chutzpah. But it is wrong. And demoralizing to so many who are taking it on the chin in the grant review game. And it makes the problems worse. How so? Well, as you know, Dear Reader I am firmly convinced that the only way to succeed in the long term is to keep rolling the reviewer dice, hoping to get three individuals who really get what you are proposing. And to take advantage of the various little features of the system that respond to frequent submissions (reviewer sympathy, PO interest, extra end of year money, ARRA, sudden IC initiatives/directions, etc). Always, always you have to send in credible proposals. But perfect vs really good guarantees you nothing. And when perfect keeps you from submitting another really good grant? You are not helping your chances. So for Nakamura to tell people to sacrifice the really good for the perfect he is worsening their chances. Particularly when the people are in those groups who are already at a disadvantage and need to work even harder* to make up for it. __ *Remember, Ginther showed that African-American PIs had to submit more revisions to get funded. Nov 23 2015 ## Scenes In the past few weeks I have been present for the following conversation topics. 1) A tech professional working for the military complaining about some failure on the part of TSA to appropriately respect his SuperNotATerrorist pass that was supposed to let him board aircraft unmolested...unlike the the rest of us riff raff. I believe having his luggage searched in secondary was mentioned, and some other delays of minor note. This guy is maybe early thirties, very white, very distinct regional American accent, good looking, clean cut... your basic All-American dude. 2) A young guy, fresh out of the military looking to get on with one of the uniformed regional service squad types of jobs. This conversation involved his assertions that you had to be either a woman or an ethnic minority to have a shot at the limited number of jobs available in any given cycle. Much of the usual complaining about how this was unfair and it should be about "merit" and the like. Naturally this guy is white, clean cut, relatively well spoken.... perhaps not all that bright, I guess. 3) A pair of essentially the most privileged people I know- mid-adult, very smart, blonde, well educated, upper middle class, attractive, assertive, parents, rock of community type of women. Literally *everything* goes in these women's direction and has for most of their lives. They had the nerve to engage in a long running conversation about their respective minor traffic stops and tickets and how unfair it was. How the cops should have been stopping the "real" dangers to society at some other location instead of nailing them for running a stop sign a little too much or right on red-ing or whatever their minor ticket was for. One of the great things about modern social media is that, done right, it is a relatively non-confrontational way to start to see how other people view things. For me the days of reading science blogs and the women-in-academics blogs were a more personal version of some of the coursework I enjoyed in my liberal arts undergraduate education. It put me in touch with much of the thinking and experiences of women in my approximate career. It occasionally allowed me to view life events with a different lens than I had previously. It is my belief that social media has also been important for driving the falling dominoes of public opinion on gay marriage over the past decade or so. Facebook connections to friends, family and friends of the same provides a weekly? daily? reminder that each of us know a lot of gay folks that are important to us or at the very least are important to people that are important to us. The relentless circulation of memes and Bingo cards, of snark and hilarity alike, remind each of us that there is a viewpoint other than our own. And the decent people listen. Occasionally they start to see things the way other people do. At least now and again. The so-called Black Twitter is similar in the way it penetrated the Facebook and especially Twitter timelines and daily RTs of so many non-AfricanAmerican folks . I have watched this develop during Ferguson and through BlackLivesMatter and after shooting after shooting after shooting of young black people that has occurred in the past two years. During the three incidents that I mention, all I could think was "Wow, do you have any idea that this is the daily reality for many of your fellow citizens? And that it would hardly ever occur to non-white people to be so blindly outraged that the world should dare to treat them this way?" And "Wait, so are you saying it sucks to have a less-assured chance of gaining the career benefits you want due to the color of your skin or the nature of your dangly bits....it'll come to you in a minute". This brings me to today's topic in academic science. Nature News has an editorial on racial disparity in NIH grant awards. As a reminder the Ginther report was published in 2011. There are slightly new data out, generated from a FOIA request: Pulmonologist Esteban Burchard and epidemiologist Sam Oh of the University of California, San Francisco, shared the data with Nature after obtaining them from the NIH through a request under the Freedom of Information Act. The figures show that under-represented minorities have been awarded NIH grants at 78–90% the rate of white and mixed-race applicants every year from 1985 to 2013 I will note that Burchard and Oh seem to be very interested in how the failure to include a diverse population in scientific studies may limit health care equality. So this isn't just about career disparity for these scientists, it is about their discipline and the health outcomes that result. Nevertheless, the point of these data are that under-represented minority PIs have less funding success than do white PIs. The gap has been a consistent feature of the NIH landscape through thick and thin budgets. Most importantly, it has not budged one bit in the wake of the Ginther report in 2011. With that said, I'm not entirely sure what we have learned here. The power of Ginther was that it went into tremendous analytic detail trying to rebut or explain the gross disparity with all of the usual suspect rationales. Trying....and failing. The end result of Ginther was that it was very difficult to make the basic disparate finding go away by considering other mediating variables. After controlling for the applicant's educational background, country of origin, training, previous research awards, publication record, and employer characteristics, we find that black applicants remain 10 percentage points less likely than whites to be awarded NIH research funding. The Ginther report used NIH grant data between FY 2000 and FY 2006. This new data set appears to run from 1985 to 2013, but of course only gives the aggregate funding success rate (i.e. the per-investigator rate), without looking at sub-groups within the under-represented minority pool. This leaves a big old door open for comments like this one: Is it that the NIH requires people to state their race on their applications or could it be that the black applications were just not as good? Maybe if they just keep the applicant race off the paperwork they would be able to figure this out. and this one: I have served on many NIH study sections (peer review panels) and, with the exception of applicants with asian names, have never been aware of the race of the applicants whose grants I've reviewed. So, it is possible that I could have been biased for or against asian applicants, but not black applicants. Do other people have a different experience? This one received an immediate smackdown with which I concur entirely: That is strange. Usually a reviewer is at least somewhat familiar with applicants whose proposals he is reviewing, working in the same field and having attended the same conferences. Are you saying that you did not personally know any of the applicants? Black PIs are such a rarity that I find it hard to believe that a black scientist could remain anonymous among his or her peers for too long. Back to social media. One of the tweeps who is, I think, pretty out as an underrepresented minority of science had this to say: Not entirely sure it was in response to this Nature editorial but the sentiment fits. If AfricanAmerican PIs who are submitting grants to the NIH after the Ginther report was published in the late summer of 2011 (approximately 13 funding rounds ago, by my calendar) were expecting the kind of relief provided immediately to ESI PIs.....well, they are still looking in the mailbox. The editorial The big task now is to determine why racial funding disparities arise, and how to erase them. ...The NIH is working on some aspects of the issue — for instance, its National Research Mentoring Network aims to foster diversity through mentoring. and the News piece: in response to Kington’s 2011 paper, the NIH has allocated more than$500 million to programmes to evaluate how to attract, mentor and retain minority researchers. The agency is also studying biases that might affect peer review, and is interested in gathering data on whether a diverse workforce improves science.

remind us of the entirely toothless NIH response to Ginther.

It is part and parcel of the vignettes I related at the top. People of privilege simply cannot see the privileges they enjoy for what they are. Unless they are listening. Listening to the people who do not share the set of privileges under discussion.

I think social media helps with that. It helps me to see things through the eyes of people who are not like me and do not have my particular constellations of privileges. I hope even certain Twitter-refuseniks will come to see this one day.

Jun 09 2015

## Really NIDA National Advisory Council? REALLY????

I was reading over the minutes from the 119th meeting of the National Advisory Council for NIDA. As you may imagine, Dear Reader, I am more than usually interested in the doings of this particular IC.

I ran across the part where Hannah A. Valantine, M.D., MRCP, Chief Officer for Scientific Workforce Diversity, National Institutes of Health came to report on diversity issues, explicitly linked to the wake of the Ginther report.

She began her focus as the COSWD by reviewing evidence from the NIH Intramural Research Program as a Laboratory for Testing Interventions to Diversify the Biomedical Workforce evaluation. The results, as of October 2014 showed that among intramural tenure track investigators 38 % are female and 1.4% are African American, 10% are Hispanic, and 0.5% are Native American compared to approximately 61% being both males and white.

PhotoCredit: NIDA IRP

Wow, NIDA IRP, wow. Only 1.4%...okay that's just Jean Lud Cadet, correct? I mean, you only list 28 "PIs" so how many tenure track investigators could there be?

Not something that is on our radar that much out here in extramural researchistan but.... yeah. Get on that NIH ICs. I would be fascinated to see the representation numbers for the various NIH Intramural Research Programs at the various levels of lab heads and non-head so-called tenure-track investigators. I imagine this is not going to look good.

There's a lot more blah de blah in the NIDA Council notes from Valantine about the NIH efforts, particularly the "pipeline" solutions that so irritate me.

But that isn't the hilarious part. The hilarious part comes after Valentine described

...in detail the strategy and essential components, such as a strategic partnership with research intensive institutions, and tracking and evaluation. Program deliverables, would include: a national network to support career transitions; evidence-based literature to eliminate/reduce barriers to key career transition points; Individual access to the network in support of career development success; organizational infrastructure to support career development and transitions; and tools and resources to catalyze and sustain career transition success.

Here was the Council response.

NIDA Council members thanked Dr. Valantine for her dedication to this issue and for implementing so many diverse and potentially transformative programs. Council members encouraged, as applicable by the research, efforts to measure success with endpoints other than “tenure”.

Jesus.

Oh hey, let's not create too high of a barrier for ourselves. Let's not set it at sustained tenured research careers THAT WERE THE VERY POINT OF THE GINTHER REPORT! Differential success of PIs at getting grant funding. How is this not intimately tied to the tenure success of African-American academic scientists?

Feb 23 2015

## Patricia Arquette, Privilege and the Oppression Olympics

I was watching the Oscars last night when Patricia Arquette busted out some equal-pay feminism in her acceptance speech.

"To every woman who gave birth, to every taxpayer and citizen of this nation, we have fought for everybody else's equal rights," Arquette said, her voice intensifying. "It is our time to have wage equality once and for all and equal rights for women in the United States of America!"

HECK YEAH!!!!!

I was hooting and hollering too much, I assume, because I got shushed. Apparently some other people in the room wanted to hear what else was being said or whatever.

So it was with some confusion that I saw backlash later on the Twitts about her. It seemed to be of the intersectionality sort of criticism. Also known as the Oppression Olympics. Not to make light of it but look, we all come with various attributes that confer privileges upon us in this society we inhabit. Most of us have one or two attributes that confer the opposite. Some unlucky folks have a pretty tough menu of biases slanting against them. So yeah, there seemed to be a drumbeat of Twitterage against Patricia Arquette's immense privilege of wealth, whiteness and heteronomativity. I thought at first that this was undeserved, based on what she said from the stage...it's the Oscars for goodness sake, of course they are all white and perfect and immensely rich.

Then today I finally happened upon her expanded backstage comments. From this account:

"The truth is: even though we sort of feel like we have equal rights in America, right under the surface, there are huge issues that are applied that really do affect women," she mused. "And it's time for all the women in America and all the men that love women, and all the gay people, and all the people of color that we've all fought for to fight for us now."

Breathtakingly tone-deaf.

Look, I've spent a lot of time in my life feeling sorry for myself. I get it. It is really, really easy to focus narrowly on that one aspect, attribute, experience, factor or misfortune that leaves the self at apparent disadvantage. And it is correspondingly easy to forget all about all the other factors and attributes that have conveyed immense privileges upon our lives.

This is not solved by the data, of course. Firstly, because we can all pick and choose which truthy stat we want to brandish. Is it equal pay? Very easy to brandish the generally accepted, broad brush stats for men versus women. And very easy to ignore that women of color are even more screwed than woman not of color. Easy to have no idea whatsoever how well minority men are paid relative to women not of color. Or what being gay confers in terms of salary.

And it is incredibly seductive to argue the anecdote. Well, Oprah! And J.Lo. And Eddie Murphy! And FFS Neil Patrick Harris is the Master of Ceremonies for goodness sake! They are sitting right there, so therefore why would anyone think of how their respective skin tones and desired life-partner would have anything to do with equal pay for women, eh?

Academic science is no different my friends. If this highly public case makes the intersectionality issue clearer to you than it has ever been, do try to turn that inwards.

We run across these examples on the blog all the time, of course. Whether we're discussing the struggles of women in science, the Ginther report, outing yourself to search committees or thesis advisors, the Baby Boomer hegemony of NIH Grant funding, postdoctoral pay rates or the evils of PIs with too many grants, the issues are the same.

"Sure, sure, there are these other biases in careers. But what is REALLY important is that I, the speaker, haven't experienced* any of those advantages that adhere to my classes and characteristics. And let me tell you about my specific set of life events that prove that really, I personally have been at huge disadvantage. So it is totally misplaced to talk about the general advantages of my characteristic X because the anecdote of me proves that X is much less important than this totally other thing that I happen to suffer from."

At this point one or the other of you, DearReader, may suspect I am talking about you in particular. Naturally, I am not. This is a common theme. Very common.

It is something that I have suffered from in my life and continue to do so. I have felt immensely sorry for myself a lot over the years.

Like many of you, I can claim one or two disadvantages within a context of immense privileges when it comes to pursuing the career of academic science. Like many of you, I CANNOT HELP BUT IGNORE MY PRIVILEGES AND PITY MYSELF ABOUT MY HARDSHIPS. Like many of you, I feel compelled to speak out about perceived injustices in the world. Like many of you, some of those injustices I speak about happen to be ones that I think affect me. Like many of you, some of those injustices I speak about do not happen to affect me in any direct way.

And, like many of you and Patricia Arquette, I often speak about injustices in a way that appears to ignore the fact that other people have it a lot worse.

Social media has a way of helping us to remember that other people have it even worse. And that trying to recruit others to help you in your fights, without ever appearing** to be that concerned about their fights comes across as selfish and tone deaf.

__
*of course you have, you just think that this is totally normal and average and deserved, and thus not worthy of inclusion in any discussion.

**For all I know Patricia Arquette is a huge fighter for underrepresented groups, including ethnic minorities and LGBT folks. But her comments certainly didn't convey that.

The tldr; version of this post:

Feb 11 2015

## A tiny bias goes a long way when it comes to grant review

From ScienceInsider:

Now, a new computer simulation explores just how sensitive the process might be to bias and randomness. Its answer: very. Small biases can have big consequences, concludes Eugene Day, a health care systems engineer at the Children's Hospital of Philadelphia, in Research Policy. He found that bias that skews scores by just 3% can result in noticeable disparities in funding rates.

T. E. Day, The big consequences of small biases: A simulation of peer review, 2015, Research Policy [epub ahead of print 28 Jan] [Publisher Site]

from the paper Abstract:

When total review bias exceeds 1.9% of grant score, statistically significant variation in scores between PC and NPC investigators is discernable in a pool of 2000 grant applications. When total review bias exceeds 2.8% of total grant score, statistically significant discrepancies in funding rates between PC and NPC investigators are detectable in a simulation of grant review.

Day generated a Preferred Class of applications and a NonPreferred Class of applications and ran a bunch of 3-reviewer scenarios with and without reviewer bias against the NPC applications. As far as I can tell the takeaway conclusion about funding here refers to a situation in which the effective payline is 10%. You will immediately grasp that NIH grant review was a strong contributor to the model parameters.

I will admit I am only able to grasp the main points here and I am in no way able to evaluate the nitty gritty.

But it appears to have a very strong message. Namely, that our introspections that "well, if there is bias it is very tiny so we don't have to be worried about it" needs to change.

There is something even scarier in this paper. From the Discussion:

The threshold level of bias in this environment seems to be 2.8% of the total possible score of the grant; this is the level at which the 95% CI of the odds ratio “kisses” 1.00. This represents a single reviewer with a bias of 0.75 points (or three reviewers each with biases of 0.25 points), which is less than half (44.4%) of the standard deviation in a single reviewer’s score. What this suggests is that levels of bias which are sub-noise – that is, that are dramatically less detectable than normal variation in reviewer scores – are sufficient to substantially bias the number of funded applications in favor of preferred investigators.

RIGHT???? The bias can be of smaller effect size than many "normal" sources of variability in scoring that we accept as the resolution of the system. And it still leads to a statistically significant bias in funding outcome.

We are talking in recent days about bias in favor of highly established, older scientists. It has been longer but the Ginther report indicating disparity of grant review outcome for African-American PIs is clearly relevant here.

What this simulation cannot do, of course, is to model the cumulative, iterative effects of review bias. Namely, the way that selection of PC applications for funding has a tendency to increase the bias in the reviewer pool, since those beneficiaries become the next reviewers. Also, the way that over the long haul, disparity of the first award can lead to actual quality differences in the subsequent applications because PI #1 had the money to pursue her science and PI #2 did not have as easy of a time generating data, publishing papers and recruiting postdoc talent.

Feb 09 2015

## Perspectives from senior scientists on the Emeritus award discussion

Tthe comments just keep coming over at RockTalking.

In 2012-13 my NIH renewal proposal with 4 specific aims was turned down 2X by the GM, NCSD Panel, with 35%+ priority scores. ...I appealed the grant reviews to the GM Council and they awarded the grant to me for 3 years at somewhat reduced funding. This funding will finance my lab until Sept 2016, after which I will close down. I am NOT closing down because I have lost my energy for, or interest in, research: [blah, blah we published and showed them!] I am closing down my lab because I can no longer put up with the aggravation of having my grants turned down

Cry me a river.

another senior investigator is on fire:

So far, all the comments fully support age discrimination. How sad! -

Age limits are silly and discriminatory. Merit worked well UNTIL there was no more money in the NIH bank.

There should be COMPETITIVE opportunities for scientists at all stages of their careers (notice I said competitive) . So it’s not a handout at all and it would be merit-based….just like those for newly trained scientists.

There are wonderful examples of some amazing senior scientists. ONE special initiative is not unreasonable for this group!

Finally, I don’t believe I said that I “deserve” anything… except not to be discriminated against for age, gender or whatever characteristic you wish to select.

updated: omg, the old guy again!

At the last American Society for Cell Biology meeting in Philadelphia (December, 2014) I stood up at the membership meeting ( only ca. 30 people) attended by some of the senior wheels in cell biology. I asked that the society appoint a new standing committee that did
nothing but try to re-design the NIH extramural grant system in its entirety, eg. the grant applications, who can apply, how review panels are chosen, requirements for NIH supported investigators to serve on panels, and how key personnel of the NIH bureaucracy are chosen, and what type of grants should be awarded. My suggestion was not greeted with great enthusiasm and, in fact, elicited some negative comments.