The funding is the science II: "Why do they always drop the females?"

The titular quote came from one of my early, and highly formative, experiences on study section. In the course of discussing a revised application it emerged that the prior version of the application had included a sex comparison. The PI had chosen to delete that part of the design in the revised application, prompting one of the experienced members of the panel to ask, quite rhetorically, "Why do they always drop the females?"
I was reminded of this when reading over Dr. Isis' excellent post on the, shall we say less pernicious, ways that the course of science is slanted toward doing male-based research. Really, go read that post before you continue here, it is a fantastic description.
What really motivated me, however, was a comment from the always insightful Stephanie Z:

Thank you. That's the first time I've seen someone address the reasons behind ongoing gender disparities in health research. I still can't say as it thrills me (or you, obviously), but I understand a bit better now.

Did somebody ring?


As I pointed out explicitly at least once, research funding has a huge role in what science actually gets conducted. Huge. In my book this means that if one feels that an area of science is being systematically overlooked or minimized, one might want to take a close look at the manner by which science is funded and the way by which science careers are sustained as potential avenues for systematic remedy.

Funding

There are a couple of ways in which the generalized problems with NIH grant review lead to the rhetorical comment with which I opened the post. One very common StockCritique of NIH grant review is that of an "over ambitious" research plan. As nicely detailed in Isis' post, the inclusion of a sex comparison doubles the groups right off the bat but even more to the point, it requires the inclusion of various hormonal cycling considerations. This can be as simple as requiring female subjects to be assessed at multiple points of an estrous cycle. It can be considerably more complicated, often requiring gonadectomy (at various developmental timepoints) and hormonal replacement (with dose-response designs, please) including all of the appropriate control groups / observations. Novel hormonal antagonists? Whoops, the model is not "well established" and needs to be "compared to the standard gonadectomy models", LOL >sigh<.
manWomanControlPanel.jpg
Grant reviewers prefer simplicity
Keep in mind, if you will, that there is always a more fundamental comparison or question at the root of the project, such as "does this drug compound ameliorate cocaine addiction?" So all the gender comparisons, designs and groups need to be multiplied against the cocaine addiction/treatment conditions. Suppose it is one of those cocaine models that requires a month or more of training per group? Who is going to run all those animals ? How many operant boxes / hours are available? and at what cost? Trust me, the grant proposal is going to take fire for "scope of the project".
Another StockCritique to blame is "feasibility". Two points here really. First is the question of Preliminary Data- of course if you have to run more experimental conditions to establish that you might have a meritorious hypothesis, you are less likely to do it with a fixed amount of pilot/startup/leftover money. Better to work on preliminary data for two or three distinct applications over just one if you have the funds. Second aspect has to do with a given PIs experience with the models in question. More opportunity to say "The PI has no idea what s/he is doing methodologically" if s/he has no prior background with the experimental conditions, which are almost always the female-related ones. As we all know, it matters little that the hormonal assays or gonadectomy or whatever procedures have been published endlessly if you don't have direct evidence that you can do it. Of course, more latitude is extended to the more-experienced investigator....but then s/he is less likely to jump into gender-comparisons in a sustained way in contrast to a newly minted PI.
Then there are the various things under grantspersonship. You have limited space in a given type of grant application. The more groups and comparisons, the more you have to squeeze in with respect to basic designs, methods and the interpretation/alternative approaches part. So of course you leave big windows for critiques of "hasn't fully considered...." and "it is not entirely clear how the PI will do..." and "how the hypothesis will be evaluated has not been sufficiently detailed...".

Career

Although research funding plays a huge role in career success, it is only part of the puzzle. Another critical factor is what we consider to be "great" or "exciting" science in our respective fields.
The little people can fill in the details. This is basically the approach of GlamourMagz science. (This is a paraphrase of something the most successful GlamourMagz PI I know actually says.) Cool, fast and hot is not compatible with the metastasizing of experimental conditions that is an inevitable feature of gender-comparison science. Trouble is, this approach tends to trickle down in various guises. Lower (than GlamourMag) impact factor journals sometimes try to upgrade by becoming more NS-like (Hi, J Neuro!). Meticulous science and exacting experimental designs are only respected (if at all) after the fact. Late(r) in someone's career they start getting props on their grant reviews for this. Early? Well the person hasn't yet shown the necessity and profit for the exhaustive designs and instead they just look...unproductive. Like they haven't really shown anything yet.
As we all know splashy CNS pubs on the CV trump a sustained area of contribution in lower journals six ways to Sunday. This is not to say that nobody will appreciate the meticulous approach, they will. Just to say that high IF journal pubs will trump. Always.
So the smart young PI is going to stay away from those messy sex-differences studies. Everything tells her she should. If he does dip a toe, he's more likely to pay a nasty career price.
This is why NIH efforts to promote sex-comparison studies are necessary. Promoting special funding opportunities are the only way to tip the equation even slightly more favorable to the sex-differences side. The lure of the RFA is enough to persuade the experienced PI to write in the female groups. To convince the new PI that she might just risk it this one time.
My suspicion is that it is not enough. Beyond the simple need to take a stepwise approach to the science as detailed by Isis, the career and funding pressures are irresistible forces.

33 responses so far

  • D. C. Sessions says:

    This almost reads like an indiriect counter to a recent comment to one of your earlier posts.

  • DSKS says:

    "So the smart young PI is going to stay away from those messy sex-differences studies. Everything tells her she should. If he does dip a toe, he's more likely to pay a nasty career price."
    Precisely, and there goes a key prerequisite for innovation right out the window. I heard second hand that a certain Nobel laureate proclaimed that the patch clamp technique probably could not have been developed in the current red hot right now! research atmosphere. Young investigators spending inordinate amounts of time breaking electrodes on the bottom of a prep bath for the sake of a pub every couple of years in Pflugers Archiv? Unthinkable today. So one wonders what innovative techniques are currently going unsupported because no bugger with any sense can afford to invest the risk and time to see them through to their workable conclusion.

  • Stephanie Z says:

    Thanks to you too, DM. I hadn't considered the indirect funding problems.

  • drdrA says:

    Dude- did you have to slap me with the realities and make me re-read all those stock critiques right now. Depressing.
    But hey- I don't get why the experiments on the boys and the experiments on the girls have to be in the same grant proposal.

  • pinus says:

    I am interested in a specific thing that relates to females only. If there were a RFA that I could apply to that relates, the priority of this particular thing would get bumped up (on my own listing of research priorities). I hope it does happen, because i have a kick ass fucking hypothesis that I would love to test.

  • DrugMonkey says:

    DC Sess, you mean this?
    I cringe at the use of the word "career" with respect to academia. Even though the definition obviously doesn't preclude its use in this manner, the term has come to represent an end in itself. e.g. research is just another kind of work one can do in order to pursue a "successful career".
    this blog is roughly 80% counter to such views....

  • D. C. Sessions says:

    DC Sess, you mean this?

    Yup.

    this blog is roughly 80% counter to such views....

    Hey, it was on my mind. I'd just tried to post a particularly snarky reply that got lost in the link-moderation trap.
    Basic point: academic science isn't monastic, requiring poverty and chastity.

  • As someone who doesn't do sex-dependent research, this may seem like a silly question but since I don't believe that dumb question exist, I will ask it.
    If we want to know "does this drug compound ameliorate cocaine addiction?" why do we have to remove / account for the estro cycles? If the end goal is to get the drug compound into the human population, you wouldn't say to women "you must have a hysterectomy to take this medication?" To me (perhaps incorrectly) that the female reproduction/hormonal systems are an innate part of women, any treatment needs to be viable with them included. Conversely, do men not experience some sort of hormonal cycles? Do researchers eliminate the effect of testosterone?.

  • You're quite correct in that performing such studies in females not only exponentially increases the number of groups but one must also determine which points of the hormonal cycles to use (I've seen anything up to 4) and how to best determine those time points with accuracy. In human studies, this is more accurately done using serum hormone levels rather than relying on self-report. In rodents, you need to do smears to determine which phase the animal is in (one phase per day, one complete cycle every four days).
    So the initial research question of "What is the effect of x on y?" becomes infinitely more complicated:
    1. What is the best method of determining phases a, b, c and d of the estrous cycle?
    2. What is the effect of x on y at times a, b, c and d?
    3. How will we confirm the phase before doing x?
    4. Do we still have enough power to detect the effect of x?
    5. What do we do if our budget won't stretch to cover the gazillion additional groups and analyses?

    As much as I hate to say it, it's much, much easier to start with "What is the effect of x on y?" and then attack the gender differences later (although almost nobody ever does).

  • DrugMonkey says:

    ScientistMother,
    men have hormonal cycles yes, but the effect sizes tend to be smaller. I think Isis covered this, or perhaps it was a comment.
    it is not correct, or it shouldn't be, that a medication needs to pass the test of being equivalently effective in men/women and across the menstrual cycle. our science should be working more toward understanding of parameters that dictate how and why potential meds might work or might not work. so what if we have different compounds that are effective in different subpopulations as long as they are indeed effective?
    gonadectomy models are not there to recommend women have hysterectomies, far from it. They are there to provide experimental control over the hormonal milieu in an animal model which may not be an ideal match to women in native state.

  • D. C. Sessions says:

    ScientistMother:
    It's a simplifying assumption. As with any such, it's bogus -- but probably necessary.
    The shame isn't that works starts with the simplified cases, but that it too often ends there.

  • Becca says:

    These have been some awesome posts and all but... does anyone have any evidence that the effects of female hormonal cycles are really so great as to jeopardize seeing effects for most experimental variables? If so, do we know which areas of investigation are most and least suceptible to this?
    Isn't the (*cough*bullshit*cough*) LarrySummersArgument (tm) that males have greater variability in several interesting biological traits? Isn't there some kind of handwaving that, due to X-inactivation chimerism, sex-chromosome properties will tend to be somewhat averaged/mellowed out in women?
    I don't believe for a second that the idea male animals are simpler is all some kind of whacko post-hoc rationalization for men using male animals because they forget about women and then assuming all those crazy female-hormones must muck everything up with girl rats. But it sure would be nice to be able to prove that isn't the case.
    *whistles*

  • Thanks DM and DC,
    I don't think drugs should be expected to be equivalently effective in men/women or even across menstrual cycles. It was more a question of the validity of the assumption. Considering that their is evidence that some drugs work better in particular racial groups vs other groups, I simply want to understand why these hormonal considerations are only thought to occur in women.

  • bsci says:

    I think this post and Dr. Isis' post to a lesser extend conflate two distinct questions.
    1) Are there sex differences?
    2) What is the nature of those sex differences?
    When we're talking about the bias of male subjects, I'll argue that the first question is much more important. As DrugMonkey notes, it's very time consuming and expensive to answer question 2. It's also a waste of energy if the result turns out that there aren't any differences. What we really want is to identify where the differences arise and that will allow more focused sex difference work in areas that matter.
    This fits very much into the funding and career issue. Merely to identify differences, you'd need a larger population size. For female volunteers or nonvolunteers it would be important to sample at different points in the hormonal cycle to maximize the chance of observing differences. Then, the only two needed tests would be whether there is a male vs. female difference and if the variance in females is greater than the variance in males. This involves extra work, but it is manageable.
    This also provides an opening for new funding methods. If an initial grant commits to try to answer question 1 and it finds sex or cycle differences, the lab could submit a fast-track supplemental grant application with a higher acceptance rate to detail those differences.
    This concept would obviously not work for all studies, such as studies where each subject involves a huge amount of time or money, but it would probably work for some.

  • These have been some awesome posts and all but... does anyone have any evidence that the effects of female hormonal cycles are really so great as to jeopardize seeing effects for most experimental variables? If so, do we know which areas of investigation are most and least suceptible to this?
    Becca, I can only speak from a cardiovascular standpoint and the answer is a resounding YES, there are very big differences between males and females both in vivo and in vitro. One simple case in point: female hearts are more resistant to ischemia/reperfusion than males and this is lost with menopause/ovariectomy.

  • whimple says:

    These topics could be addressed in order:
    Null hypothesis 1: This drug compound does not ameliorate cocaine addiction.
    if false,
    Null hypothesis 2: This drug works equally well in males and females.
    if false,
    Null hypothesis 3: The efficacy of the drug is unrelated to estrous cycle variations.
    etc.

  • DrugMonkey says:

    Becca, I'm on the outside fringes of my areas here so I don't have the examples immediately to hand. I will suggest that browsing the abstracts pulled up by this link and this link will give you a toehold into the idea that effects of cocaine may depend on sex and hormonal status. Just to show that in this narrow area, at least, it is not idiotic to think that we should be concerned about sex differences and the influences of hormonal fluctuation.

  • DrugMonkey says:

    whimple you are entirely correct and this is consistent with Dr. Isis' original post. My point, and those of some comments here, is that once null one is rejected, most people never bother to move on to test null two and three, they choose another test compound and repeat null one.

  • These have been some awesome posts and all but... does anyone have any evidence that the effects of female hormonal cycles are really so great as to jeopardize seeing effects for most experimental variables?
    PiT gave a great example of a condition in which femaleness and cycling is important. Activation of the estrogen alpha receptor is cardioprotective and, since we cycle in terms of estrogen...well, we are more of less cardioprotected depending on our estrogen state. Dr. Isis agreed to be catheterized tomorrow to study the effect of her female cycle on her cardiovascular system because of the timing of her cycle. I'm pleased my colleague is studying this...maybe I'll liveblog it.
    So, why bother at all? 1) We want to produce data that, in the end, leads to the best practice of medicine and 2) we know that cycle and hormonal state is important in certain conditions. There is no reason to believe that it's not important in other conditions and, thus, we should be asking the question. However, as I said on my own blog, maleness is not the control for the female condition. So why do we use male animals? Because day-to-day fluctuations in testosterone are smaller and we cross our fingers and hope they're not important. Is it right? Who knows? But the fact of the matter remains, if you present a study with female creatures someone's going to oask about hormonal control. If you present a study with male creatures, people will pat you on the head and congratulate you for your foresight.
    So, Dr. Isis is going to say something crazy. A while ago the NIH mandated the inclusion of women in clinical research cohorts. I think every study should be designed with sufficient male and female animals to answer at least the global sex question and investigators should receive the appropriate funding to do it. Discuss.
    Until then Dr. Isis will keep studying female animals on the D-L.

  • Becca says:

    How much variation does hormonal cycling add if you're on exogenous hormones?
    We could put our rats on the pill. I was thinking it'd cut the noise, and (as I see it) modeling women-on-hormonal-birth-control > modeling-only-men.
    @DM- totally cool studies, but not actually an answer to the question I wanted. I was in no way disputing hormonal cycling or femaleness are really interesting variables in some traits.
    I was more pondering if we did all our experiments with females by default, if there were lots of things we wouldn't discover.
    I think my focus is on the signal:noise ratios; I'm mostly worried about false negatives. You do a study, and conclude that a drug/intervention/variable is ineffective/unimportant because there isn't a statistical difference between the two groups. More variation within the groups makes it harder to make such conclusions. We've assumed that using females (ignoring their mensutral cycle) would add to that variability, and I was more wondering if there was evidence for that.

  • DrugMonkey says:

    I was more pondering if we did all our experiments with females by default, if there were lots of things we wouldn't discover.
    We might miss the wackaloon, violent/impulsive variant of alcoholism and think it was all about depression and anxiety.

  • D. C. Sessions says:

    I think my focus is on the signal:noise ratios; I'm mostly worried about false negatives. You do a study, and conclude that a drug/intervention/variable is ineffective/unimportant because there isn't a statistical difference between the two groups. More variation within the groups makes it harder to make such conclusions. We've assumed that using females (ignoring their mensutral cycle) would add to that variability, and I was more wondering if there was evidence for that.

    Your question is heading off in the direction of science-based medicine as distinct from evidence-based medicine. The difference is in one sense primarily economic [1].
    On the one hand, if sex-related variability can be lumped in with other population variations then you can get useful results in a single trial with a reasonable size. On the other you have a series of trials with what amount to independent non-comparable sub-populations.
    Both approaches can eventually get you what you want, but each is more expensive than the other for some cases. The trick is either telling which model best applies a priori or else sometimes accepting the cost of using the less-appropriate model.
    [1] Or would be depending on funding constraints.

  • Alex says:

    We could put our rats on the pill. I was thinking it'd cut the noise, and (as I see it) modeling women-on-hormonal-birth-control > modeling-only-men.
    Given the significant number of human females on the pill, I'd think that including a group of rats on the pill in the study would be pretty important for reasons aside from signal/noise ratios: It would also be representative of a significant cohort of the human patients that are supposed to ultimately benefit from the study.
    However, I'm a physicist, so I have no idea whether rats on the pill are a common laboratory model. Would somebody who wanted to do a study with female rats on the pill then get in trouble for proposing to use a model that is not commonly studied, or that the PI does not have sufficient experience with, or something like that?

  • leigh says:

    i've heard of several instances of sex differences studies being proposed and shot down by review committees as "overly ambitious" (those two words still piss me off from my own grant review, end story.)
    i know some people who do/did a lot of female endocrine effects work. the surgeries, sham surgeries, hormone replacement injections and/or minipump implantations, subtype selective agonists, vehicle controls, etc etc etc oh and in some cases lavage preps made answering a simple question a really fucking enormous task.
    i don't think i've heard of dose-response curves in hormone replacement. i'm not all that familiar with the sex differences literature myself, though.
    estrogen does some cool stuff in the brain- and for that it is really important that we study males and females when looking at cns phenomena. but one outwardly-simple-looking question can easily wind up as almost an entire phd dissertation. so add in the cost of a graduate student to all the existing costs (easily 4x the number of animals, surgeries, hormone replacement, animal care, facilities, supplies, equipment, etc)

  • Dr. Feelgood says:

    Yeah, I have dealt with this issue ad nauseum. I am lucky since much of my work is developmental, I can examine juvenile females without worrying about estrus cycling. But this issue is a totally stock criticism. However, as mentioned, at least there are RFAs and PAs that specifically call for examination of females. I think it might be good to establish the entire phenomenon in males, then put in an entire new grant examining the same issue in females. However, it is true, females get the shaft (...that sounded bad...).
    Dr F.

  • The NIH needs to get its head out of its ass. Its stock critiques aren't important when put into the context of this larger issue of single-sex studies on everything. The NIH, as a leader in science funding, has a responsibility to be fair in its allocation of resources. Oh, you have got to love bureaucracy.
    But DM, certainly your point about newly minted PI's pursuing the path of least resistance is dead-on. That will be a hard problem to fix. Unless we infiltrate the ranks with a bunch of new female PI's, and then some of them will be pissed enough to tackle the problem, if only for something to do on the DL.

  • Unless we infiltrate the ranks with a bunch of new female PI's, and then some of them will be pissed enough to tackle the problem, if only for something to do on the DL.
    CE: why should this be tackled only by pissed off new female PIs??

  • whimple says:

    The NIH needs to get its head out of its ass. Its stock critiques aren't important when put into the context of this larger issue of single-sex studies on everything.
    Stop blaming the NIH for this. Stock critiques are the fault of the study section members, not the NIH. The NIH's fault is allowing the study section members to so dominate the awarding of research dollars that programmatic needs suffer.

  • S. Rivlin says:

    I guess that playing the blame game on who's responsible for lack of studies on and funding for sex differences will solve nothing. It requires real actions by those who care about fairness and equality and should be approached as any other social issue - the creation of a movement that could make enough noise and put enough pressure on the US congress and the Administration to force the NIH to issue RFAs for such studies.

  • whimple says:

    ...force the NIH to issue RFAs for such studies
    Just in case this horse isn't completely dead yet...
    These RFAs aren't going to make any difference unless there is set-aside money allocated and a special review panel convened. Without set-aside money these study proposals will just keep on getting crushed by the same standing study sections that are crushing them right now.

  • RFAs by definition have set-aside funds and are always reviewed by specially constituted study sections.

  • whimple says:

    True that. Was thinking Program Announcements. These PAs need to get some teeth put in them!

Leave a Reply