Archive for the 'NIH funding' category

NIA paylines and anti-ESI bias of review

Apr 20 2016 Published by under NIH, NIH funding

MillerLab noted on the twitters that the NIA has released it's new paylines for FY2016. If your grant proposal scores within the 9%ile zone, congrats! Unless you happen to be an Early Stage Investigator in which case you only have to score within the top 19% of applications, woot!

I was just discussing the continuing nature of the ESI bias in a comment exchange with Ferric Fang on another thread. He thinks

The problem that new investigators have in obtaining funding is not necessarily a result of bias but rather that it is more challenging for new investigators to write applications that are competitive with those of established investigators because as newcomers, they have less data and fewer accomplishments to cite.

and I disagree, viewing this as assuredly a bias in review. The push to equalize success rates of ESI applicants with those of established investigators (generational screw-job that it is) started back in 2007 with prior NIH Director Elias Zerhouni. The mechanism to accomplish this goal was, and continues to be, naked quota based affirmative action. NIH will fund ESI applications out of the order of review until they reach approximately the same success percentages as is enjoyed by the established investigator applications. Some ICs are able to game this out predictively by using different paylines- the percentile ranks within which almost all grants will be funded.

NIA-fundingpolicyAs mentioned, NIA has to use a 19%ile cutoff for ESI applications to equal a 9%ile cutoff for established investigator applications. This got me thinking about the origin of the ESI policies in 2007 and the ensuing trends. Luckily, the NIA publishes its funding policy on the website here. The formal ESI policy at NIA apparently didn't kick in until 2009, from what I can tell. What I am graphing here are the paylines used by NIA by fiscal year to select Exp(erienced), ESI and New Investigator (NI) applications for funding.

It's pretty obvious that the review bias against ESI applications continues essentially unabated*. All the talk about "eating our seed corn", the hand wringing about a lost generation, the clear signal that NIH wanted to fund the noobs at equivalent rates as the older folks....all fell on deaf ears as far as the reviewers are concerned. The quotas for the ESI affirmative action are still needed to accomplish the goal of equalizing success rates.

I find this interesting.

__
*Zerhouni noted right away [PDF] that study sections were fighting back against the affirmative action policy for ESI applications.

Told about the quotas, study sections began “punishing the young investigators with bad scores,” says Zerhouni.

Note: It is probably only a coincidence that CSR reduced the number of first time reviewers in FY2014, FY2015 relative to the three prior FYs.

16 responses so far

On removing grant deadlines

Apr 20 2016 Published by under NIH, NIH Careerism, NIH funding

Eric Hand reported in Science that one NSF pilot program found that allowing for any-time submission reduced applications numbers.

Assistant Director for Geosciences Roger Wakimoto revealed the preliminary results from a pilot program that got rid of grant proposal deadlines in favor of an anytime submission. The numbers were staggering. Across four grant programs, proposals dropped by 59% after deadlines were eliminated.

I have been bombarded with links to this article/finding and queries as to what I think.

Pretty much nothing.

I do know that NIH has been increasingly liberal with allowing past-deadline submissions from PIs who have served on study section. So there is probably a data source to draw upon inside CSR if they care to examine it.

I do not know if this would do anything similar if applied to the NIH.

The NSF pilot was for

geobiology and low-temperature geochemistry, geomorphology and land-use dynamics, hydrological sciences, and sedimentary geology and paleobiology.

According to the article these are fields in which

"many scientists do field work, having no deadline makes it easier for collaborators to schedule time when they can work on a proposal".

This field work bit is not generally true of the NIH extramural community. I think it obvious that continual-submission helps to schedule time but I would note that it also eliminates a stick for the more proactive members of a collaboration to beat the slaggards into line. As a guy who hits his deadlines for grant submission, it's probably in my interest to further lower the encouragements the lower-energy folks require.

According to a geologist familiar with reviewing these grants

The switch is “going to filter for the most highly motivated people, and the ideas for which you feel the most passion,” he predicts. When he sits on merit review panels, he finds that he can usually reject half of the proposals right away as being hasty or ill-considered. “My hope is that this has taken off the bottom 50%,” he says. “Those are the ones you read and say, ‘Did they have their heart in this?’”

Personally I see very few NIH grant proposals that appear to me to be "hasty or ill-considered" or cause me to doubt the PI has her heart in it. And you know how I feel about the proposition that the RealProblem with NIH grant success hinges on whether or not PIs refine and hone and polish their applications into some shining gem of a document. Applications are down so therefore success rates go up is the only thing we need to take away from this pilot, if you ask me. Any method by which you could decrease NIH applications would likewise seem to improve success rates.

Would it work for NIH types? I tend to doubt it. That program at NSF started with only two submission rounds per year. NIH has three rounds for funding per year, but this results from a multitude of deadlines including new R01, new R21/R03, two more for the revised apps, special ones for AIDS-related, RFAs and assorted other mechanisms. As I mentioned above, if you review for the NIH (including Advisory Council service) you get an extra extension to submit for a given decision round.

The pressure for most of us to hit any specific NIH deadline during the year is, I would argue, much lower at baseline. So if the theory is that NSF types were pressured to submit junky applications because their next opportunity was so far away....this doesn't apply to NIH folks.

6 responses so far

Bias at work

A piece in Vox summarizes a study from Nextions showing that lawyers are more critical of a brief written by an African-American. 

I immediately thought of scientific manuscript review and the not-unusual request to have a revision "thoroughly edited by a native English speaker". My confirmation bias suggests that this is way more common when the first author has an apparently Asian surname.

It would be interesting to see a similar balanced test for scientific writing and review, wouldn't it?

My second thought was.... Ginther. Is this not another one of the thousand cuts contributing to African-American PIs' lower success rates and need to revise the proposal extra times? Seems as though it might be. 

22 responses so far

On sending trainees to conferences that lack gender balance

Neuroscientist Bita Moghaddam asked a very interesting question on Twitter but it didn't get much discussion yet. I thought I'd raise it up for the blog audience.

My immediate thought was that we should first talk about the R13 Support for Scientific Conferences mechanism. These are often used to provide some funding for Gordon Research Conference meetings, for the smaller society meetings and even some very small local(ish) conferences. Examples from NIDA, NIMH, NIGMS. I say first because this would seem to be the very easy case.

NIH should absolutely keep a tight eye on gender distribution of the meetings supported by such grant awards.The FOA reads, in part:

Additionally, the Conference Plan should describe strategies for:

Involving the appropriate representation of women, minorities, and persons with disabilities in the planning and implementation of, and participation in, the proposed conference.
Identifying and publicizing resources for child care and other types of family care at the conference site to allow individuals with family care responsibilities to attend.

so it is a no-brainer there, although as we know from other aspects of NIH the actual review can depart from the FOA. I don't have any experience with these mechanisms personally so I can't say how well this particular aspect is respected when it comes to awarding good (fundable) scores.

Obviously, I think any failure to address representation should be a huge demerit. Any failure to achieve representation at the same, or similar meeting ("The application should identify related conferences held on the subject during the past 3 years and describe how the proposed conference is similar to, and/or different from these."), should also be a huge demerit.

At least as far as this FOA for this scientific conference support mechanism goes, the NIH would appear to be firmly behind the idea that scientific meetings should be diverse.

By extension, we can move on to the actual question from Professor Moghaddam. Should we use the additional power of travel funds to address diversity?

Of course, right off, I think of the ACNP annual meeting because it is hands down the least diverse meeting I have ever attended. By some significant margin. Perhaps not in gender representation but hey, let us not stand only on our pet issue of representation, eh?

As far as trainees go, I think heck no. If my trainee wants to go to any particular meeting because it will help her or him in their careers, I can't say no just to advance my own agenda with respect to diversity. Like it or not, I can't expect any of them to pay any sort of price for my tender sensibilities.

Myself? Maybe. But probably not. See the aforementioned ACNP. When I attend that meeting it is because I think it will be advantageous for me, my lab or my understanding of science. I may carp and complain to certain ears that may matter about representation at the ACNP, but I'm not going on strike about it.

Other, smaller meetings? Like a GRC? I don't know. I really don't.

I thank Professor Moghaddam for making me think about it though. This is the start of a ponder for me and I hope it is for you as well.

16 responses so far

Your Grant in Review: Power analysis and the Vertebrate Animals Section

Feb 11 2016 Published by under Grant Review, Grantsmanship, NIH funding

As a reminder, the NIH issued warning on upcoming Simplification of the Vertebrate Animals Section of NIH Grant Applications and Contract Proposals.

Simplification! Cool, right?

There's a landmine here.

For years the statistical power analysis was something that I included in the General Methods at the end of my Research Strategy section. In more recent times, a growing insistence on the part of the OLAW that a proper Vertebrate Animals Section include the power analysis has influenced me to drop the power analysis from the Research Strategy. It became a word for word duplication so it seemed worth the risk to regain the page space.

The notice says:

Summary of Changes
The VAS criteria are simplified by the following changes:

  • A description of veterinary care is no longer required.

  • Justification for the number of animals has been eliminated.

  • A description of the method of euthanasia is required only if the method is not consistent with AVMA guidelines.

 

This means that if I continue with my current strategy, I'm going to start seeing complaints about "where is the power analysis" and "hey buddy, stop trying to evade page limits by putting it in the VAS".

So back to the old way we must go. Leave space for your power analysis, folks.
__
If you don't know much about doing a power analysis, this website is helpful: http://homepage.stat.uiowa.edu/~rlenth/Power/

17 responses so far

Where will your lab be in 5 years?

Scientifically, that is.

I like the answer Zoe gave for her own question.

I, too, just hope to be viable as a grant funded research laboratory. I have my desires but my confidence in realizing my goals is sharply limited by the fact I cannot count on funding.

Edited to add:

When I was a brand new Assistant Professor I once attended a career stage talk of a senior scientist in my field. It wasn't an Emeritus wrap-up but it was certainly later career. The sort of thing where you expect a broad sweeping presentation of decades of work focused around a fairly cohesive theme.

The talk was "here's the latest cool finding from our lab". I was.....appalled. I looked over this scientist's publication record and grant funding history and saw that it was....scattered. I don't want to say it was all over the place, and there were certain thematic elements that persisted. But this was when I was still dreaming of a Grande Arc for my laboratory. The presentation was distinctly not that.

And I thought "I will be so disappointed in myself if I reach that stage of my career and can only give that talk".

I am here to tell you people, I am definitely headed in that direction at the moment. I think I can probably tell a slightly more cohesive story but it isn't far away.

I AM disappointed. In myself.

And of course in the system, to the extent that I think it has failed to support my "continuous Grande Arc Eleventy" plans for my research career.

But this is STUPID. There is no justifiable reason for me to think that the Grande Arc is any better than just doing a good job with each project, 5 years of funding at a time.

137 responses so far

Ask DrugMonkey: JIT and Progress Reports

Nov 10 2015 Published by under Ask DrugMonkey, NIH, NIH Careerism, NIH funding

Two quick things:

Your NIH grant Progress Report goes to Program. Your PO. It does not go to any SRO or study section members, not even for your competing renewal application. It is for the consumption of the IC that funded your grant. It forms the non-competing application for your next interval of support that has already passed competitive review muster.

Second. The eRA commons automailbot sends out requests for your JIT (Just In Time; Other Support page, IRB/IACUC approvals) information within weeks of your grant receiving a score. The precise cutoff for this autobot request is unclear to me and it may vary by IC or by mechanism for all I know. The point is, that it is incredibly generous. Meaning that when you look at your score and think "that is a no-way-it-will-ever-fund score" and still get the JIT autobot request, this doesn't mean you are wrong. It means the autobot was set to email you at a very generous threshold.

JIT information is also requested by the Grants Management Specialist when he/she is working on preparing your award, post-Council. DEFINITELY respond to this request.

The only advantage I see to the autobot request is that if you need to finalize anything with your IRB or IACUC this gives you time. By the time the GMS requests it, you are probably going to be delaying your award if you do not have IRB/IACUC approval in hand. If you submit your Other Support page with the autobot request, you are just going to have to update it anyway after Council.

15 responses so far

NIH grant application changes are in the offing

Oct 16 2015 Published by under NIH, NIH Careerism, NIH funding

The Weekly NIH Guide (for 16 October 2015, that link will update) has a whole slew of changes summarized in NOT-OD-16-004.

NOT-OD-16-006 seeks to simplify the Vertebrate Animals section by deleting requirement for describing vet care, euthanasia if consistent with AVMA guidelines and justification for the number of animals used.

NOT-OD-16-011 seeks to implement rigor and transparency in grant applications. Focus is on "the scientific premise forming the basis of the proposed research, rigorous experimental design for robust and unbiased results,consideration of relevant biological variables , and authentication of key biological and/or chemical resources."

oh, and for certain people around here NOT-OD-16-009 plans a change in allowable fonts that can be used in NIH grant applications. Key features are black text, 6 lines per vertical inch, 15 characters per linear inch and 11 pt type. The usual fonts are "recommended" although "other fonts (both serif and non-serif) are acceptable if they meet the above requirements"

19 responses so far

Publisher wants to take journal Open Access

Someone forwarded me what appears to be credible evidence that Wiley is considering taking Addiction Biology Open Access.

To the tune of $2,500 per article.

At present this title has no page charges within their standard article size.

This is interesting because Wiley purchased this title quite a while ago at a JIF that was at or below my perception of my field's dump-journal level.

They managed to march the JIF up the ranks and get it into the top position in the ISI Substance Abuse category. This, IMO, then stoked a virtuous cycle in which people submit better and better work there.

At some point in the past few years the journal went from publishing four issues per year to six. And the JIF remains atop the category.

As a business, what would you do? You build up a service until it is in high demand and then you try to cash in, that's what.

Personally I think this will kill the golden goose. It will be a slow process, however, and Wiley will make some money in the mean time.

The question is, do most competitors choose to follow suit? If so, Wiley wins big because authors will eventually have no other option. If the timing is good, Addiction Biology makes money early and then keeps on going as the leader of the pack.

All y'all Open Access wackaloons believe this is inevitable and are solidly behind Wiley's move, no doubt.

I will be fascinated to see how this one plays out.

50 responses so far

End of year pickups

Sep 29 2015 Published by under NIH, NIH funding

It's one of those times of years to go a-RePORTERing, my friends. Select 9/1 or 9/15 in the Project Start Date field and put your favorite IC in the field for that.

As the NIH reaches the end of the federal fiscal year, they have to balance their budget. Meaning that in many cases they will pick up out-of-order grants to satisfy some goal or other. No doubt sometimes it is just making the dollars and cents add up by slotting in a few more R03 or R21 grants.

Maybe it is a chance for them to trigger on priorities that they have been letting simmer on the back burner or maybe it is a class of grants that has to wait until the end of the year for some reason. BigMechs seem to be funded during September in several of my favorite ICs.

I seem to notice SBIR/STTR grants (R41, R42, R43, R44 mechs) rolling out, which makes sense. The overall NIH has a certain percentage it has to meet in terms of SBIR awards and I assume this rolls downhill to the IC level. So this is part of the balancing of books for the final accounting.

The thing I was noticing this year is that the list of grant awards from my favorite ICs seems...interesting. To me anyway. And given when I tend to find interesting (i.e., the unusual) it would be no surprise if this was as feature not a bug. I.e, real.

Look at it this way. The unusual has the potential to be treated somewhat poorly by study sections or it wouldn't be an unusual application. If you subscribe to a view that study sections suffer from a certain conservatism (and I do subscribe) than it makes sense that the end of year pickups might be interesting due to them being unusual. Perhaps there are POs who likewise look at the list of near-misses and are attracted to the grant application that offers a little breath of fresh air. Perhaps it is because there are the odd RFA extras that can be squeezed under the budget line.

...or maybe I am extrapolating too far from very limited data.

13 responses so far

« Newer posts Older posts »