The Aims shall be Three, and Three shall be the number of Aims.
Four shalt there not be, nor Two except as they precede the Third Aim.
Five is right out.
The Aims shall be Three, and Three shall be the number of Aims.
Four shalt there not be, nor Two except as they precede the Third Aim.
Five is right out.
The first study section rounds that are obliged to grapple with the new SABV policy are upon us.
SROs are instructing panels and issuing grant assignments to reviewers.
If you are reviewing, what are your thoughts?
Me, I see more of the entirely predictable ahead- people ignoring it (or accepting thin excuses for not studying both sexes, in reality) or brandishing it as a cudgel in highly variable fashion. I'm cynical, perhaps unduly so, eh?
The opportunity to beat a panel into better agreement will come far too late for most applications. There is no way that guilt over consistency will drag the triaged apps up for discussion.
I still seek consistency. In my own reviewing and in any panels I serve. I think this a virtue to strive for.
And I think that consideration and discussion of the approach to tricky review issues is the way to advance toward that goal.
I also think that when you accept a reviewer position, you are agreeing to give the NIH what it is requesting, to the best of your ability. If you fight against the SABV push, you are doing it very wrong, IMO.
So....what do you think? How are you approaching the SABV mandate? Now that you have a few examples of how applicants have dealt with it, have you learned anything useful for us to consider?
Open Mike blog on SABV mandate
My prediction is that the grants that will do best in the next few rounds are those that successfully excuse themselves from including both male and female subjects.
The grants that try to respond to the spirit of the new NIH SABV initiative will get comparatively hammered in review.
— Andrew Pruszynski (@andpru) May 2, 2016
I am genuinely curious as to how you people see this. Is there any particular difference between people arguing that that acquisition of the first major grant award should be protected versus multiple award and the people arguing that acquisition of the first and third concurrent awards should be on an equal footing?
If we agree that NIH (or NSF or CIHR or whatever) grants are competitively awarded, it follows that nobody is actually entitled to a grant. And as far as I am aware, all major funding agencies operate in a way that states and demonstrates the truth of this statement.
Specifically in the NIH system, it is possible for the NIH officials to choose not to fund a grant proposal that gets the best possible score and glowing reviews during peer review. Heck, this could happen repeatedly for approximately the same project and the NIH could still choose not to fund it.
Nobody is entitled to a grant from the NIH. Nobody.
It is also the case that the NIH works very hard to ensure a certain amount of equal representation in their awarded grants. By geography (State and Congressional district), by PI characteristics of sex and prior NIH PIness, by topic domain (see the 28 ICs) or subdomain (see Division, Branches of the ICs. also RFAs), etc.
Does a lean to prioritize the award of a grant to those with no other major NIH support (and we're not just talking the newcomers- plenty of well-experienced folks are getting special treatment because they have run out of other NIH grant support) have a justification?
Does the following graph, posted by Sally Rockey, the previous head of Extramural Research at the NIH make a difference?
This shows the percentage of all PIs in the NIH system for Fiscal Years 1986, 1998, 2004 (end of doubling) and 2009 who serve as PI on 1-8 Research Project Grants. In the latest data, 72.3% had only one R01 and 93% had 1 or 2 concurrent RPGs. There were 5.4% of the PIs that held 3 grants and 1.2% that held 4 grants. I just don't see where shifting the 7% of 3+ concurrent awards into the 1-2 grant population is going to budge the needle on the perceived grant chances of those without any major NIH award. Yes, obviously there will be some folks funded who would otherwise not have been. Obviously. But if this is put through in a systematic way*, the first thing the current 3+ grant holders are going to do is stop putting in modular grants and max out their allowable 2 at $499,999 direct costs. Maybe some will even get Program permission to breach the $500,000 DC / y threshold. So there won't be a direct shift of 7% of grants back into the 1-2 grant PI population.
There has been a small trend for PIs holding more grants concurrently from 1986 to the late naughties but this is undoubtedly down to the decreasing purchasing power of the modular-budget grant.
I"ve taken their table of yearly adjustments and used those to calculate the increase necessary to keep pace with inflation (black bars) and the decrement in purchasing power (red bars). The starting point was the 2001 fiscal year (and the BRDPI spreadsheet is older so the 2011 BRDPI adjustment is predicted, rather than actual). As you can see, a full modular $250,000 year in 2011 has 69% of the purchasing power of that same award in 2001.
Without that factor, I'd say the relative proportions of PIs holding 1, 2, 3 etc grants would be even more similar across time than it already is.
So I come back to my original question. What is fair? What policies should the NIH or any broad governmental funding body adopt when it comes to distributing the grant wealth across laboratories? On what basis should they do this?
Fairness? Diversity of grant effort? PR/optics?
*and let us face it, it is hugely unlikely that the entire NIH will put through a 2-grant cap without any exceptions. Even with considerable force and authority behind it, any such initiative is likely to be only partially successful in preventing 3+ grant PIs.
DISCLAIMER: As always, I am an interested party in these discussions. My lab's grant fortunes are affected by broad sweeping policies that the NIH might choose to adopt or fail to adopt. You should always read my comments about the NIH grant game with this in mind.
MillerLab noted on the twitters that the NIA has released it's new paylines for FY2016. If your grant proposal scores within the 9%ile zone, congrats! Unless you happen to be an Early Stage Investigator in which case you only have to score within the top 19% of applications, woot!
The problem that new investigators have in obtaining funding is not necessarily a result of bias but rather that it is more challenging for new investigators to write applications that are competitive with those of established investigators because as newcomers, they have less data and fewer accomplishments to cite.
and I disagree, viewing this as assuredly a bias in review. The push to equalize success rates of ESI applicants with those of established investigators (generational screw-job that it is) started back in 2007 with prior NIH Director Elias Zerhouni. The mechanism to accomplish this goal was, and continues to be, naked quota based affirmative action. NIH will fund ESI applications out of the order of review until they reach approximately the same success percentages as is enjoyed by the established investigator applications. Some ICs are able to game this out predictively by using different paylines- the percentile ranks within which almost all grants will be funded.
As mentioned, NIA has to use a 19%ile cutoff for ESI applications to equal a 9%ile cutoff for established investigator applications. This got me thinking about the origin of the ESI policies in 2007 and the ensuing trends. Luckily, the NIA publishes its funding policy on the website here. The formal ESI policy at NIA apparently didn't kick in until 2009, from what I can tell. What I am graphing here are the paylines used by NIA by fiscal year to select Exp(erienced), ESI and New Investigator (NI) applications for funding.
It's pretty obvious that the review bias against ESI applications continues essentially unabated*. All the talk about "eating our seed corn", the hand wringing about a lost generation, the clear signal that NIH wanted to fund the noobs at equivalent rates as the older folks....all fell on deaf ears as far as the reviewers are concerned. The quotas for the ESI affirmative action are still needed to accomplish the goal of equalizing success rates.
I find this interesting.
Told about the quotas, study sections began “punishing the young investigators with bad scores,” says Zerhouni.
Note: It is probably only a coincidence that CSR reduced the number of first time reviewers in FY2014, FY2015 relative to the three prior FYs.
Eric Hand reported in Science that one NSF pilot program found that allowing for any-time submission reduced applications numbers.
Assistant Director for Geosciences Roger Wakimoto revealed the preliminary results from a pilot program that got rid of grant proposal deadlines in favor of an anytime submission. The numbers were staggering. Across four grant programs, proposals dropped by 59% after deadlines were eliminated.
I have been bombarded with links to this article/finding and queries as to what I think.
Pretty much nothing.
I do know that NIH has been increasingly liberal with allowing past-deadline submissions from PIs who have served on study section. So there is probably a data source to draw upon inside CSR if they care to examine it.
I do not know if this would do anything similar if applied to the NIH.
The NSF pilot was for
geobiology and low-temperature geochemistry, geomorphology and land-use dynamics, hydrological sciences, and sedimentary geology and paleobiology.
According to the article these are fields in which
"many scientists do field work, having no deadline makes it easier for collaborators to schedule time when they can work on a proposal".
This field work bit is not generally true of the NIH extramural community. I think it obvious that continual-submission helps to schedule time but I would note that it also eliminates a stick for the more proactive members of a collaboration to beat the slaggards into line. As a guy who hits his deadlines for grant submission, it's probably in my interest to further lower the encouragements the lower-energy folks require.
According to a geologist familiar with reviewing these grants
The switch is “going to filter for the most highly motivated people, and the ideas for which you feel the most passion,” he predicts. When he sits on merit review panels, he finds that he can usually reject half of the proposals right away as being hasty or ill-considered. “My hope is that this has taken off the bottom 50%,” he says. “Those are the ones you read and say, ‘Did they have their heart in this?’”
Personally I see very few NIH grant proposals that appear to me to be "hasty or ill-considered" or cause me to doubt the PI has her heart in it. And you know how I feel about the proposition that the RealProblem with NIH grant success hinges on whether or not PIs refine and hone and polish their applications into some shining gem of a document. Applications are down so therefore success rates go up is the only thing we need to take away from this pilot, if you ask me. Any method by which you could decrease NIH applications would likewise seem to improve success rates.
Would it work for NIH types? I tend to doubt it. That program at NSF started with only two submission rounds per year. NIH has three rounds for funding per year, but this results from a multitude of deadlines including new R01, new R21/R03, two more for the revised apps, special ones for AIDS-related, RFAs and assorted other mechanisms. As I mentioned above, if you review for the NIH (including Advisory Council service) you get an extra extension to submit for a given decision round.
The pressure for most of us to hit any specific NIH deadline during the year is, I would argue, much lower at baseline. So if the theory is that NSF types were pressured to submit junky applications because their next opportunity was so far away....this doesn't apply to NIH folks.
I immediately thought of scientific manuscript review and the not-unusual request to have a revision "thoroughly edited by a native English speaker". My confirmation bias suggests that this is way more common when the first author has an apparently Asian surname.
It would be interesting to see a similar balanced test for scientific writing and review, wouldn't it?
My second thought was.... Ginther. Is this not another one of the thousand cuts contributing to African-American PIs' lower success rates and need to revise the proposal extra times? Seems as though it might be.
Neuroscientist Bita Moghaddam asked a very interesting question on Twitter but it didn't get much discussion yet. I thought I'd raise it up for the blog audience.
— Bita Moghaddam (@bita137) February 22, 2016
My immediate thought was that we should first talk about the R13 Support for Scientific Conferences mechanism. These are often used to provide some funding for Gordon Research Conference meetings, for the smaller society meetings and even some very small local(ish) conferences. Examples from NIDA, NIMH, NIGMS. I say first because this would seem to be the very easy case.
NIH should absolutely keep a tight eye on gender distribution of the meetings supported by such grant awards.The FOA reads, in part:
Additionally, the Conference Plan should describe strategies for:
Involving the appropriate representation of women, minorities, and persons with disabilities in the planning and implementation of, and participation in, the proposed conference.
Identifying and publicizing resources for child care and other types of family care at the conference site to allow individuals with family care responsibilities to attend.
so it is a no-brainer there, although as we know from other aspects of NIH the actual review can depart from the FOA. I don't have any experience with these mechanisms personally so I can't say how well this particular aspect is respected when it comes to awarding good (fundable) scores.
Obviously, I think any failure to address representation should be a huge demerit. Any failure to achieve representation at the same, or similar meeting ("The application should identify related conferences held on the subject during the past 3 years and describe how the proposed conference is similar to, and/or different from these."), should also be a huge demerit.
At least as far as this FOA for this scientific conference support mechanism goes, the NIH would appear to be firmly behind the idea that scientific meetings should be diverse.
By extension, we can move on to the actual question from Professor Moghaddam. Should we use the additional power of travel funds to address diversity?
Of course, right off, I think of the ACNP annual meeting because it is hands down the least diverse meeting I have ever attended. By some significant margin. Perhaps not in gender representation but hey, let us not stand only on our pet issue of representation, eh?
As far as trainees go, I think heck no. If my trainee wants to go to any particular meeting because it will help her or him in their careers, I can't say no just to advance my own agenda with respect to diversity. Like it or not, I can't expect any of them to pay any sort of price for my tender sensibilities.
Myself? Maybe. But probably not. See the aforementioned ACNP. When I attend that meeting it is because I think it will be advantageous for me, my lab or my understanding of science. I may carp and complain to certain ears that may matter about representation at the ACNP, but I'm not going on strike about it.
Other, smaller meetings? Like a GRC? I don't know. I really don't.
I thank Professor Moghaddam for making me think about it though. This is the start of a ponder for me and I hope it is for you as well.
As a reminder, the NIH issued warning on upcoming Simplification of the Vertebrate Animals Section of NIH Grant Applications and Contract Proposals.
Simplification! Cool, right?
There's a landmine here.
For years the statistical power analysis was something that I included in the General Methods at the end of my Research Strategy section. In more recent times, a growing insistence on the part of the OLAW that a proper Vertebrate Animals Section include the power analysis has influenced me to drop the power analysis from the Research Strategy. It became a word for word duplication so it seemed worth the risk to regain the page space.
The notice says:
Summary of Changes
The VAS criteria are simplified by the following changes:
A description of veterinary care is no longer required.
Justification for the number of animals has been eliminated.
A description of the method of euthanasia is required only if the method is not consistent with AVMA guidelines.
This means that if I continue with my current strategy, I'm going to start seeing complaints about "where is the power analysis" and "hey buddy, stop trying to evade page limits by putting it in the VAS".
So back to the old way we must go. Leave space for your power analysis, folks.
If you don't know much about doing a power analysis, this website is helpful: http://homepage.stat.uiowa.edu/~rlenth/Power/
Scientifically, that is.
it's hard when sr. PIs ask you where you hope your lab is in 5 yrs because the honest answer is: I hope I'm still here (1)
— Zoe McElligott (@nanopharmNC) January 12, 2016
I like the answer Zoe gave for her own question.
I, too, just hope to be viable as a grant funded research laboratory. I have my desires but my confidence in realizing my goals is sharply limited by the fact I cannot count on funding.
Edited to add:
When I was a brand new Assistant Professor I once attended a career stage talk of a senior scientist in my field. It wasn't an Emeritus wrap-up but it was certainly later career. The sort of thing where you expect a broad sweeping presentation of decades of work focused around a fairly cohesive theme.
The talk was "here's the latest cool finding from our lab". I was.....appalled. I looked over this scientist's publication record and grant funding history and saw that it was....scattered. I don't want to say it was all over the place, and there were certain thematic elements that persisted. But this was when I was still dreaming of a Grande Arc for my laboratory. The presentation was distinctly not that.
And I thought "I will be so disappointed in myself if I reach that stage of my career and can only give that talk".
I am here to tell you people, I am definitely headed in that direction at the moment. I think I can probably tell a slightly more cohesive story but it isn't far away.
I AM disappointed. In myself.
And of course in the system, to the extent that I think it has failed to support my "continuous Grande Arc Eleventy" plans for my research career.
But this is STUPID. There is no justifiable reason for me to think that the Grande Arc is any better than just doing a good job with each project, 5 years of funding at a time.