Archive for the 'NIH' category

NIH Mandate to Consider Sex as a Biological Variable in Grant Apps

Jun 09 2015 Published by under NIH, NIH funding, Sex Differences

The NIH has published NOT-OD-15-102 Consideration of Sex as a Biological Variable in NIH-funded Research which informs us:

This notice focuses on NIH's expectation that scientists will account for the possible role of sex as a biological variable in vertebrate animal and human studies. Clarification of these expectations is reflected in plans by NIH's Office of Extramural Research (OER) to update application instructions and review questions; once approved by the Office of Management and Budget (OMB), these updates will take effect for applications submitted for the January 25, 2016, due date and thereafter.

Also:

Accounting for sex as a biological variable begins with the development of research questions and study design. It also includes data collection and analysis of results, as well as reporting of findings. Consideration of sex may be critical to the interpretation, validation, and generalizability of research findings. Adequate consideration of both sexes in experiments and disaggregation of data by sex allows for sex-based comparisons and may inform clinical interventions. Appropriate analysis and transparent reporting of data by sex may therefore enhance the rigor and applicability of preclinical biomedical research.4

NIH expects that sex as a biological variable will be factored into research designs, analyses, and reporting in vertebrate animal and human studies. Strong justification from the scientific literature, preliminary data, or other relevant considerations must be provided for applications proposing to study only one sex. Investigators are strongly encouraged to discuss these issues with NIH program staff prior to submission of applications.

Additional information is provided in a three page PDF overview:

Literature review. Consider and describe how sex and gender may influence the research question(s) at hand. Conduct a review of the human clinical literature and any relevant preclinical literature. If there are differences between males and females in previous preclinical or clinical studies, this would provide a strong rationale for building consideration of sex into the research design and analyses of data. The absence of previous study data in an area of research does not, by itself, constitute strong justification to study only one sex.

Very nice. So helpful. Look NIH, clearly this is going to be a place where applicants who do not wish to incorporate SABV into the design are going to seek a loophole. What would be helpful here would be a more assertive statement about what does and, most importantly does not, constitute a "strong justification to study only one sex". Uncontrolled, this will devolve back to the reviewers who are already failing (going by your highly effortful and high profile new initiative) to appropriately favor* SABV in research grant proposals. They are the ones that will decide that the tiniest fig leaf of excuse making is acceptable "justification" if you give them half a chance to do so. This part needs strengthening.

and later on in the document:

Single-sex studies. Applicants must provide strong justification for applications proposing to study only one sex. Such justification may include the study of sex-specific conditions or phenomena (e.g., ovarian or prostate cancer), acutely scarce resources (e.g., non-human primates), or investigations in which the study of one sex is scientifically appropriate. The absence of evidence regarding sex differences in an area of research does not constitute strong justification to study only one sex.

Sex-specific conditions or phenomena, check. Good. Will hard-to-breed mice constitute "acutely scarce resources"? Human drug abusers of various characteristics that make it hard to recruit female or male participants? The devil will be in the detail. But "scientifically appropriate"? Again, this holds open a big old loophole of escape. And a repeat of the absence-of-evidence statement. What does this mean? What are the limits on this strong justification? How are you going to get reviewers on board with this, instead of leaving them to accept any old excuse?

Research design, data analysis, and reporting.
....Where little or no sex-specific data is available, sex-specific hypotheses may not be possible, whereas previously observed sex differences may prompt sex-specific hypotheses.

Dude what? Are you kidding with this? We all know there must be a supported hypothesis in the research plan. And if there has not been any sex-differences research in the past, well, there are no hypotheses we can advance. And therefore, so sorry, we must avoid proposing anything that investigates SABV because the study section will kill us for lack of a clear hypothesis**. Another whoppingly huge escape clause for the SABV resistant PI.

Acknowledge limitations in the applicability of findings that may arise from the samples, methods, and analyses used, in the research plan as well as in progress reports and publications.

Emphasis added. HAHAHAHHAHHAA!!!! Yeah RIGHT! Every NIH Grant awardee who does not explicitly include SABV in a paper must make sure to add the caveat in the Discussion that their results cannot be extended to the other sex. Sure that's going to happen. Sure.

Finally, one for my peers who already conduct SABV research with regularity.

Researchers working with animal models should consider if and how the female estrous cycle is relevant for experimental design and analysis; it may be relevant for some research questions and not others

This one is pointed straight at the buzz saw of the sex-differences aficionado Stock Criticism of grant applications. One of the ways that sex-differences gets stamped out of research proposals is that the "real" experts start in on "YOU MUST DO THE SEX-COMPARISONS RIGHT AND AS WE HAVE DONE". This may include cycle synchronization, gonadectomy, pharmacologico-hormonal manipulations, endless groups, etc, etc, etc.

There is little tolerance from these people for "First, let's give it a go in female (or male) animals/cells/tissues and see what we turn up" exploratory fishing expeditions.

I would argue that tolerance for fishing expeditions is precisely what the NIH needs if they want to jumpstart real change. You have to make the barrier low and, especially in this day and age, of low cost. Demanding that it has to be SABV design 101eleventy at all times or it is not worth doing is going to motivate resistance. Resistance on the part of PIs doing their grant proposing and on the part of peers doing the grant reviewing.

I propose that a NIH policy of "Any old Third Aim that will engage in sex-differences comparisons is good enough and a total freebie for the first five years***" is what is necessary.

_
*oh yes, believe you me there are puh-lenty of investigators who propose SABV aspects in proposals and get it beat out of them at review.

**StockCritiqueTM

***that may have to be slightly more formal

34 responses so far

Learning and training

Jun 05 2015 Published by under NIH, NIH Careerism, Peer Review

Every aspect of human endeavor that involves teaching newcomers how to do something involves both didactic and practical experiences. 

That is just the way it works.

Grant review is one of those things. Formal instruction only gets the job partially done. More learning takes place in the doing.

6 responses so far

NIGMS MIRA for ESI/NI differs slightly. Ok, fundamentally.

Jun 03 2015 Published by under Fixing the NIH, NIH, NIH Careerism, NIH funding

NIGMS has been attempting to grapple with the problem of stability in research funding for its extramural awardees. Which is a great thing to focus on, given the instability in recent years and the wasted time and effort of PIs and their laboratories which is devoted to maintaining stable funding.

In January NIGMS launched the MIRA program (R35 mechanism) to issue 5 year awards (instead of current NIGMS average of 4) of up to $750,000 in direct costs. The idea is that current NIGMS awardees would consolidate their existing NIGMS awards into this one R35, promise to devote at least 51% of their research effort to this R35 and overall take less in NIGMS funding.

The immediate objections were severalfold but more or less focused on why the already privileged NIGMS stalwarts with three or more concurrent full-modular ($250,000 direct) awards' worth of funding should now get this extra isolation from the review process. If the limited number of such individuals selected for MIRA now had research funds that were isolated from the grasp of peer review (the fifth year, the all-or-none nature of the $750,000 direct in one award) then obviously the unlucky would be further disadvantaged.

One such pool of the unlucky would be the ESI (and NI) investigators.

NIGMS assured us that they were planning to extend MIRA to ESI/NI in the very near future. Peter Preusch commented:

We plan to issue a MIRA funding opportunity for early stage investigators as quickly as possible. We hope the first application due date will be sometime this summer.

Well, RFA-GM-16-003 has arrived. And it is nothing like the real MIRA for the highly established insider club* of NIGMS extramural funding.

1) It is limited to $250,000 in direct costs
2) It will be for the duration of the "current average of R01 awards to new investigators", read 4 years, I assume. Even if the current average is 5, this can change. Why not just write in 5 as for the main MIRA?
3) Competing renewals "may" be allowed to increase substantially. There is of course no guarantee of this and if they were serious they could have simply written in language such as "the second interval will increase the limit to $500K direct and the third to $750K direct". They did not.

This is either ridiculously ill-considered or a cynical figleaf designed to give political cover for the excesses of the real target, the MIRA for the highly-established.

Here is what is so fundamentally foot-shooting about this, if you assume that NIGMS has any interest in shepherding the careers of their future stalwarts. The current stalwarts they are trying to protect are multi-grant awardees. Three full-modular and two-plus if you assume one of those awards is a traditional budget up to the $500K stiff (but not insurmountable) limit. Yet here they are trying to take what might be thought of as this same population at an earlier career stage and making sure they only get one full-modular worth of NIGMS funding from the start. This is insanely ill-considered.

And no ESI PI who thinks of herself as a future multi-grant NIGMS stalwart (and perhaps real-MIRA qualified) should have any interest in this baby MIRA whatsoever. All it comes with are limits for such a PI.

A secondary consideration is the review of such applications. Wisely, NIGMS has made this an RFA which means they get to design their own review panels.

This is wise because these special-flower-protection grants (real MIRA and baby-MIRA alike) stand a good risk of getting shredded in regular study sections. I'm thinking there is a good risk of them getting shredded in whatever SEPs they manage to convene too, unless they do a good job of selecting quid-pro-quo qualified reviewers.

Related Aside: BigMechs like Program Projects and Centers are very often reviewed by panels of other BigMech Program Directors and component PIs. This is consistent with the general requirement that grants should be reviewed by panels with like-experience. However, this lets in a great deal of quid-pro-quo reviewing in the sense that the reviewers know these applicants will be coming back to review their Boondoggles, sorry BigMechs when they are up for competing review. Thus, these mechanisms are very unlikely to face review of the kind that disagrees fundamentally with the concept of the BigMech. Unlikely to get anyone saying "none of this is worth the cost, these shouldn't be funded and the money should be put back** in the R-mech pool".

Regular R-mech study sections are disproportionally staffed by midcareer scientists. Given the likely number of MIRAs on offer, disproportionally staffed by scientists who will not feel like they have the slightest chance at a MIRA award. I predict a good deal of skepticism from the general reviewer about these R35 mechanisms and I predict very bad scores.

Which is why NIGMS will have to be careful to cherry pick a quid-pro-quo qualified reviewer pool. And, as is usually the case with BigMechs, be prepared to fund them with scores that would not be remotely competitive for regular R01 review.

__
*Remember, PIs with more than two concurrent RPGs are less than 9% of the entire NIH funded population (in FY2009, according to Rockey). How many can there be with three or more NIGMS awards?

**there are some technicalities with pools of $$ that make this slightly more complicated than this but you get the flavor

29 responses so far

It's all just political

Jun 01 2015 Published by under #FWDAOTI, NIH, NIH Careerism

First of all, if you don't understand that anything featuring groups of humans is in the broader sense "political" than you are a fool.

The typical charge that NIH grant review is "all political" made by disappointed applicants, however, always sounds a little more...specific. Take this guy:

Nice and truthy. But what does it mean?

As you might expect I set about trying to get home slice here to define terms and be more specific about what "politics" there are that are making the decision among grant applications which survive triage. Naturally he started dodging and weaving and refused to define what he meant by "politics" save for

which is ridiculous. Yes, big scale stuff like this involves a lot of real actual political behavior. But this has very little to do with the round-by-round review of grants in study sections. In fact, the Brain Initiative folks launched their political effort precisely because they were not enjoying the success they thought they deserved in the usual NIH grant review process!

The closest our friend came to honesty was

which is nice and wishy washy as a definition. Obviously it means that he has decided that the people who he thinks should not get funded do win NIH grants. Since he has determined, in his wisdom, that it is unjustified that they are funded then clearly it is because of "undue personal influence".

It cannot possibly be that the many players in the system come with their own unique constellation of beliefs about what constitutes the most-meritorious proposals, see? It has to be politics and undue personal influence.

And this is such an important factor in deciding what gets funded out of the 40-50% of proposals that do not get triaged, that he is suggesting wholesale revision of the process to award below the triage line via lottery.

I find this laughable. Yes, there is a great deal of randomness as far as which grants get selected for funding in a given round. I continue to believe, however, that non-random factors are important and that over the entirety of NIH grant selection, the 5%ile grant is likely to be selected over the 45%ile grant for nonpolitical reasons. We may not agree individually with all of these reasons, but I think dismissal of it all being "undue personal influence" is wrong. YHN is a prickly and unfriendly customer in real life and yet is funded. I know of many really friendly and awesome scientists who struggle to get NIH funding. Time after time on study section I hear the crappy application from the highly successful PI being lauded on the basis of past accomplishments and never once on the personal influence. The vast majority of the time, people are reviewing grants from people they don't really even know personally.

I remain confused as to what this charge of "politics" really means, if it is anything other than personal disgruntlement. But I am eager to learn.

So by all means, Dear Reader, have at it.

What does it mean to you to say grant review is "political"? Be specific in your terms. How could we reduce undue influence? What changes to regular old unsolicited grant review should be made to combat this truthy sounding boogeyman?

59 responses so far

The NIH Cull and the K99/R00 cohorts

May 28 2015 Published by under Fixing the NIH, NIH, NIH Careerism

DataHound posted two key analyses on the state of the NIH-funded extramural work force. In the first one he presents the number of unique PIs from 1985-2014. It looks to me, roughly, that there are about 18% fewer PIs than the peak and approximately 10% fewer PIs if we ignore the ARRA interval.

Most of the real drop (i.e., not postARRA) occurred between FY2011 and FY2012 but there has been a downward trend from 2012 to 2014 so this looks to be the new reality.

The Cull is in full view now.

Has it seemed like grants are getting funded slightly more easily lately? If so, you can thank the Cull. (No doubt the pressure is more about applications than funded awards to unique PIs. But if the applications are seeing similar drops, this explains the feeling of relief, if you have it.)

The second post at DataHound presents several graphs on the K99/R00 awardees by original award year.

Transition to the R00 phase did not vary much up through the 2010 cohort and the cohorts are on the same trajectory, given the time function. Importantly the 2007-2009 cohorts follow the exact same trajectory, 2010 cohorts have a little bit of drop-off at the far end, due to less time since original award. Six years after the K99 award is the hard ceiling on transition to R00 in the first three cohorts and 2010 K99ers aren't quite there yet.

Where the K99 awardee cohorts are not on the same trajectory is the transition to R01. DataHound's plots show a clear plateau for the 2007-2010 cohorts. The 2007 awardees topped out at about 58% transitioning to R01 funding and subsequent cohort success rates are lower, year over year. Success in gaining an R01 for the 2009 cohort is about 70% that of the 2008 grouup and about half that of the 2007 cohort. The 2010 cohort is at least 20% less-successful than the 2009 K99 awardees.

It is pretty clear the Cull described in the first linked post is falling harder on the K99/R00 awardees than on the general pool of NIH-funded PIs. Depending on whether you take the ARRA high water mark for unique PIs or something lower that adheres to the normal trend, the Cull is only about a 10-20% as of FY2014.

This is BACKWARDS!

All this talk about getting more new scientists over the hump to faculty level career status. All this whinging and moaning about eating our seed corn. All the handwringing over ESIs.

And the program that is the crown jewel in doing something about transition is....not working.

If history is any guide, it would have taken official NIHdom about 15 years to "suddenly realize" this is the case and to try something new.

Thank goodness for DataHound. I anticipate he has accelerated this process by posting these two key analyses.

75 responses so far

Thought on the Ginther report on NIH funding disparity

May 24 2015 Published by under Fixing the NIH, NIH, NIH Careerism, NIH funding

I had a thought about Ginther just after hearing a radio piece on the Asian-Americans that are suing Harvard over entrance discrimination. 

The charge is that Asian-American students need to have better grades and scores  than white students to receive an admissions bid. 

The discussion of the Ginther study revolved around the finding that African-American applicant PIs were less likely than PIs of other groups to receive NIH grant funding. This is because Asian-Americans, for example, did as well as white PIs. Our default stance, I assume, is that being a white PI is the best that it gets. So if another group does as well, this is evidence of a lack of bias. 

But what if Asian-American PIs submit higher quality applications as a group? 
How would we ever know if there was discrination against them in NIH grant award?

20 responses so far

Thoughts on NIH grant strategy from Associate Professor H. Solo

We spend a fair amount of time talking about grant strategy on this blog. Presumably, this is a reflection of an internal process many of us go through trying to decide how to distribute our grant writing effort so as to maximize our chances of getting funded. After all we have better things to do than to write grants.

So we scrutinize success rates for various ICs, various mechanisms, FOAs, etc as best we are able. We flog RePORTER for evidence of which study sections will be most sympathetic to our proposals and how to cast our applications so as to be attractive. We worry about how to construct our Biosketch and who to include as consultants or collaborators. We obsess over how much preliminary data is enough (and too much*).

This is all well and good and maybe, maybe....perhaps....it helps.

But at some level, you have to follow your gut, too. Even when the odds seem overwhelmingly bad, there are going to be times when dang it, you just feel like this is the right thing to do.

Submitting an R01 on very thin preliminary data because it just doesn't work as an R21 perhaps.

Proposing an R03 scope project even if the relevant study section has only one** of them funded on the RePORTER books.

Submitting your proposal when the PO who will likely be handling it has already told you she hates your Aims***.

Revising that application that has been triaged twice**** and sending it back in as a A2asA0 proposal.

I would just advise that you take a balanced approach. Make your riskier attempts, sure, but balance those with some less risky applications too.

I view it as....experimenting.

__
*Just got a question about presenting too much preliminary data the other day.

**of course you want to make sure there is not a structural issue at work, such as the section stopped reviewing this mechanism two years ago.

***1-2%ile scores have a way of softening the stony cold heart of a Program Officer. Within-payline skips are very, very rare beasts.

****one of my least strategic behaviors may be in revising grants that have been triaged. Not sure I've ever had one funded after initial triage and yet I persist. Less so now than I used to but.....I have a tendency. Hard headed and stupid, maybe.

13 responses so far

NIH Program Officers do not understand what happens during review

May 22 2015 Published by under Grant Review, Grantsmanship, NIH, NIH Careerism

It is one of the most perplexing things of my career and I still don't completely understand why this is the case. But it is important for PIs, especially those who have not yet experienced study section, to understand a simple fact of life.

The NIH Program Officers do not completely understand what contributes to the review and scoring of your grant application.

My examples are legion and I have mentioned some of them in prior blog posts over the years.

The recent advice from NIAID on how to get your grant to fit within a modular budget limit.

The advice from a PO that PIs (such as myself) just needed to "write better grants" when I was already through a stint on study section and had read many, many crappy and yet funded grants from more established investigators.

The observation that transitioning investigators "shouldn't take that job" because it was soft money and K grants were figuring heavily in the person's transition/launch plans.

Apparently honest wonder that reviewers do not read their precious Program Announcements and automatically award excellent scores to applications just because they align with the goals of the PA.

Ignorance of the revision queuing that was particularly endemic during the early part of my career (and pretend? ignorance that limiting applications to one revision round made no functional difference in this).

The "sudden discovery" that all of the New Investigator grants during the checkbox era were going to well-established investigators who simply happened not to have NIH funding before, instead of boosting the young / recently appointed investigators.

An almost comically naive belief that study section outcome for grants really is an unbiased reflection of grant merit.

I could go on.

The reason this is so perplexing to me is that this is their job. POs [eta: used to] sit in on study section meetings or listen in on the phone. At least three times a year but probably more often given various special emphasis panels and the assignment of grants that might be reviewed in any of several study sections. They even take notes and are supposed to give feedback to the applicant with respect to the tenor of the discussion. They read any and all summary statements that they care to. They read (or can read) a nearly dizzying array of successful and unsuccessful applications.

And yet they mostly seem so ignorant of dynamics that were apparent to me after one, two or at the most three study section meetings.

It is weird.

The takeaway message for less NIH-experienced applicants is that the PO doesn't know everything. I'm not saying they are never helpful....they are. Occasionally very helpful. Difference between funded and not-funded helpful. So I fully endorse the usual advice to talk to your POs early and often.

Do not take the PO word for gospel, however. Take it under advisement and integrate it with all of your other sources of information to try to decide how to advance your funding strategy.

25 responses so far

Crystal clear grant advice from NIAID

May 21 2015 Published by under Grant Review, Grantsmanship, NIH, NIH Careerism

from this Advice Corner on modular budgeting:

As you design your research proposal, tabulate a rough cost estimate. If you are above but near the $250,000 annual direct cost threshold, consider ways to lessen your expenses. Maybe you have a low-priority Specific Aim that can be dropped or a piece of equipment you could rent rather than buy new.

H/t: PhysioProf

Related Reading:

NIAID
Sample Grants

26 responses so far

Fighting with the New Biosketch format

I have been flailing around, of and on for a few months, trying to write my Biosketch into the new format [Word doc Instructions and Sample].

 

I am not someone who likes to prance around bragging about "discoveries" and unique contributions and how my lab's work is I am so awesomely unique because, let's face it, I don't do that kind of work. I am much more of a work-a-day type of scientist who likes to demonstrate stuff that has never been shown before. I like to answer what are seemingly obvious questions for which there should be lots of literature but then it turns out that there is not. I like to work on what interests me about the world and I am mostly uninterested in what some gang of screechy monkey GlamourHumpers think is the latest and greatest.

Ahem.

This is getting in the way of my ability to:

Briefly describe up to five of your most significant contributions to science. For each contribution, indicate the historical background that frames the scientific problem; the central finding(s); the influence of the finding(s) on the progress of science or the application of those finding(s) to health or technology; and your specific role in the described work.

Now interestingly, it was someone who works in a way most unlike the way I do that showed me the light. Actually, he gave me the courage to think about ignoring this supposed charge in the sample / instruction document. This person recommended just writing a brief sentence or two about the area of work without trying to contextualize the importance or significance of the "contribution". I believe I actually saw one of the five permitted subheadings on his version that was more or less "And here's some other stuff we work on that wasn't easily categorized with the rest of it."

I am at least starting from this minimalist standpoint. I don't know if I will have the courage to actually submit it like this, but I'm leaning towards doing so.

I have been hearing from quite a number of you that you are struggling with creating this new version of the NIH Biosketch. So I thought I'd open it up to comment and observation. Anyone have any brilliant solutions / approaches to recommend?

UPDATE:
One of the things that has been bothering me most about this is that it takes the focus off of your work that is specific to the particular application in question. In the most recent version of the Biosketch, you selected 15 pubs that were most directly relevant to the topic at hand. These may not be your "most significant contributions" but they are the ones that are most significant for the newly proposed studies.

If one is now to list "your most significant contributions", well, presumably some of these may not have much to do with the current application. And if you take the five sections seriously, it is hard to parse the subset of your work that is relevant to one focal R01 sized project into multiple headings and still show now those particular aspects are a significant contribution.

I still think it is ridiculous that they didn't simply make this an optional way to do the Biosketch so as to accommodate those people that needed to talk about non-published scholarly works.

64 responses so far

« Newer posts Older posts »