via deviant art
We recently discussed a President's column at ASBMB Today by Steven McKnight in which he claimed that NIH grant review panels are contaminated with "riff-raff" who are incompetent to properly review proposals.
A Nature News piece published today notes that McKnight 'was “saddened” by suggestions that he has any gripe with young researchers or with diversity. He meant to criticize review committees as a whole, not just young scientists, he added.'
Very interesting. He was saddened, apparently, that not enough scientists understood he was calling them riff-raff! It is not only the junior scientists, it is also everybody else that he despises.
“A level of mediocrity has crept into the grant-review system,” he said. He recalled that earlier in his career, grant-review committees were packed with well-known scientists with established credentials. “Now when I look at the list, I’ll know zero names. Five or six of them will be people from fly-by-night biotech companies.” He said that he hasn’t done any quantitative research on this trend, “but I think I’m probably right”.
Let's just unpack that "I'll know zero names" business. There is an alternative hypothesis here, which is that perhaps McKnight doesn't "know" the names of people that he should.
Leaving aside whether or not McKnight actually performed any review of scientific quality per se, let us recall what happens in any age graded social situation. For my US readers in particular, I will remind you of who you "knew the names of" when you were a Freshman in High School or College versus who you "knew the names of" when you were a Senior in those respective environs. I submit to you that, as was my experience, when are looking up the social ranks, you know a heck of a lot more people than when you are looking down the social ranks.
This squares entirely with my direct experiences with my older peers in science. It is not infrequent that I refer to someone I think of as a hot Young Gun of science and the oldster has no idea who I am talking about. Even when the oldster has been impressed by the work that person has done in OtherOldster's lab but still mentally tags it to the OtherOldster. It is only with time, repetition and further excellence from the Young Gun that my acquaintance oldsters come to "see" the name of the Young Gun.
Note, this can be well into said Young Gun's independent career as an Assistant Professor.
I am not trying to excuse McKnight's snobbiness here at all. I am mentioning a common social experience that has to do with the accident of age and relative stature and has essentially nothing to do with relative merits of the people one "knows the name of".
Given that this is so common, however, you might think one would be hesitant to bray on about people's merits as a scientist based on whether your rapidly aging (and clearly not the most socially tuned) brain happens to recognize their name.
Note: Just for grins I'm reviewing the panel rosters in the
Integrative, Functional, and Cognitive Neuroscience IRG [IFCN]. These eight panels review a lot of Neuroscience grants. Admittedly I scanned quickly, and did not review the meeting rosters for ad hoc members, but I found zero individuals from biotechs. I also note that the Universities are heavily dominated by the R1s and the non-University Institutions represented are very well known Med schools and Research Institutes. I don't think I am even above the mean (or 25th percentile, frankly) in being able to recognize names across all of neuroscience.....but even scanning quickly I saw some big names all across these groups.
This may not be comprehensive data either but it sure as heck overmatches McKnight's "I think I'm probably right" comment.
Is a lab that has 12 or more concurrent postdocs really "training" them?
Is this in and of itself evidence that this is the scientific workforce?
If, say, three-quarters of them ended up in faculty appointments would this change the equation?
One does not demonstrate "genius" by being the first over the line to answer a question that a whole bunch of other people know is the question and are likewise pursuing. Particularly when the thing that lets you "win" is access to some particular datum that someone else just doesn't happen to have right now.
Genius is more credibly associated with answering questions that no other scientist even realizes is the key question yet.
Some DAOTI asked a silly question
— Christopher Weber (@chmweber) October 7, 2014
got the simple answer of "no", demanded data and was summarily mocked. For this he got all fronty.
— Christopher Weber (@chmweber) October 7, 2014
because of course he already knew the answer he wanted to hear in response to his question.
This all arose in the wake of an article in the Boston Globe about the postdoc glut that contained this hilariosity.
“They really are the canary in the coal mine,” said Marc Kirschner, a professor of systems biology at Harvard Medical School whose lab of 17 scientists includes 12 postdocs. “They decided they’d go ahead and try to understand why a cancer cell is different from a normal cell, and here they are a few years out. They knew it was a competitive situation, and they were going to work very hard, but they didn’t see the whole system was going to sour so quickly.”
I bolded for the slower reader. My initial reaction was:
If your "lab" has 12 postdocs, you don't get to act like you give a care about the future of science careers.
— Drug Monkey (@drugmonkeyblog) October 7, 2014
On to the point.
The world of "R1, TT" positions in science is incredibly diverse, yes, even within "biomedical" or just plain "biology". I repeatedly urge postdocs who feel helpless about the glut of postdocs to start by doing some research. Find out ALL of the people who have recently been hired all across the US in jobs that are somewhat remotely related to your skillset. Note, not your "interests". Your SKILLSET!
I follow this up with a call to do the same on RePORTER to find out about the vast diversity of grants that are funded by the NIH. Diversity in topic and diversity in geographical region and diversity in University or Department stature.
This is even before I tell people to get their "R1" noses out of the air and look seriously at Universities that are supposedly beneath their notice.
So what makes for a successful "competitor" for all of the jobs that are open? Is it one thing? Such as "vertically ascending eleventy systems buzzword biology science" training? That is published in Nature and derives from a 12+ postdoc lab with everyone busily trying to hump the same pantsleg?
"Everyone" here is, guess what? Your competition. And yes, if you choose to only seek out "R1, TT" jobs that are in a University that boatloads of people want to work in, applying techniques to topic domains that a dozen fellow postdocs are also doing right beside you, chasing CNS "gets" that a few scores of labs worldwide are also chasing...well, yes, you are going to be at a disadvantage if you are not training in one of those labs.
But this doesn't also mean that making all of those choices is not also putting you at a disadvantage for a "R1, TT" job if that is your goal. Because it is putting you at a disadvantage.
Vince Lombardi's famous dictum applies to academic careers.
Run to Daylight.
Seek out ways to decrease the competition, not to increase it, if you want to have an easier career path in academic science. Take your considerable skills to a place where they are not just expected value, but represent near miraculous advance. This can be in topic, in geography, in institution type or in any other dimension. Work in an area where there are fewer of you.
Given this principle, no, a big lab does not automatically confer an advantage to obtaining a tenure-track position at an R1 university. According to Wikipedia the US has 108 Carnegie-approved "Very high research activity" Universities. Another 99 are in the next bin of "high research activity" and this includes places that would be quite reasonable for someone who wants to be an actual teaching + research old school professor. I know many scientists at these institutions and they seem to be productive enough and, I assume*, happy to be actual Professors.
Would coming from a big lab be a help? Maybe. But often enough search committees at R1s (and the next bin) are looking for signs of independent thought and a unique research program. That is hard to establish in a big lab...far easier to demonstrate from a lab with one or two concurrent postdocs. Other times, the "big labs" in a field (say, Drug Abuse) are simply not structured like they are in cannon-fodder, bench-monkey, GlamourHumping, MolecularEleventy labs. Maybe this is because the overall "group" organized around the subject has Assistant Professors where those "big labs" have Nth year "postdocs". Maybe it is because this just isn't the culture of a subfield. If that is the case, then when an R1 is hiring in your domain, they aren't expecting to see a CV that competes with three other ones just like it from people sharing your lab. They are expecting to see a unique flower with easily discernible individual contribution to the last three years of work from that small lab. That type of candidate has an advantage for this particular job search.
So yeah. It is a stupid question to ask if [single unique training environment] confers an "advantage" for some thing as general as a tenure-track job at an R1 University.
I'll close with a tweet from yesterday:
The “jobs are so hard to get!” message is a little harder to listen to when so few people are applying for jobs when we offer them.
— Terry McGlynn (@hormiga) October 6, 2014
and a followup
@hormiga Right? We have two jobs right now, and less than a dozen apps each. Gah.
— Rachael French (@DrRachaelF) October 6, 2014
This all reminds me of a famous Twitter "independent scientist" jackhole who applied to a few elite Universities, couldn't get an offer and summarily declared all of science to be broken, corrupt, crowded with "diversity" riff raff and all sorts of other externalizing excuses. Make sure you don't fall into this trap if you are serious about succeeding in an academic career.
*actually, they say so.
At first you might think it a negative that the US Supreme Court refused to hear five gay marriage cases this term.
The Supreme Court on Monday turned away appeals from five states looking to prohibit gay marriage, effectively legalizing same-sex marriage in those states and likely others -- but also leaving the issue unresolved nationally.
The justices rejected appeals from Indiana, Oklahoma, Utah, Virginia and Wisconsin. The court's order immediately ends delays on gay marriage in those states.
Couples in six other states -- Colorado, Kansas, North Carolina, South Carolina, West Virginia and Wyoming -- also should be able to get married in short order. Those states would be bound by the same appellate rulings that were put on hold pending the Supreme Court's review. That would make same-sex marriage legal in 30 states and the District of Columbia.
So they haven't "finished the job". So what. The nationwide trend on this is clear and we will have only the smallest bumps ahead as more lower courts find that what SCOTUS did in the Federal decisions applies in their jurisdictions as well. More than half the population now lives in a jurisdiction that permits same-sex marriage. Or as more States decide to legalize same-sex marriage and nobody complains (at the legal level).
This is what happened with Loving v. Virginia too. Perhaps Roe v. Wade as well?
SCOTUS waiting for tides to turn in a particular direction within the country has precedent. And today, they announced they will not try to move that tide, they will watch it. And that is a good thing.
Today was a win for civil rights and I'm going to celebrate it as such.
Since many of you are AAAS members, as am I, I think you might be interested in an open letter blogged by Michael Balter, who identifies himself as "a Contributing Correspondent for Science and Adjunct Professor of Journalism at New York University".
I have been writing continuously for Science for the past 24 years. I have been on the masthead of the journal for the past 21 years, serving in a variety of capacities ranging from staff writer to Contributing Correspondent (my current title.) I also spent 10 years as Science’s de facto Paris bureau chief. Thus it is particularly painful and sad for me to tell you that I will be taking a three-month leave of absence in protest of recent events at Science and within its publishing organization, the American Association for the Advancement of Science (AAAS).
Yet in the case of the four women dismissed last month, no such explanation was made, nor even a formal announcement that they were gone. Instead, on September 25, Covey wrote a short email to Science staff telling us who the new contacts were for magazine makeup and magazine layout. No mention whatsoever was made of our terminated colleagues. As one fellow colleague expressed it to me: “Brr.”
Four staff dismissals that he blames on a newcomer to the organization.
I think that this collegial atmosphere continued to dominate until earlier this year, when the changes that we are currently living through began in earnest. Rob Covey came on board at AAAS in September 2013, and at first many of us thought that he was serving mostly in an advisory capacity; after all, he had a reputation for helping media outlets achieve their design and digital goals, a role he had played at National Geographic, Discovery Communications, and elsewhere. I count myself among those who were happy about many of the changes he brought about, including the redesign of the magazine, the ramping up of our multimedia presence, etc. But somewhere along the way Covey began to take on more power and more authority for personnel decisions, an evolution that has generated increasing consternation among the staff in all of Science’s departments.
New broom sweeps?
(In addition, according to all the information I have been able to gather about it, Covey was responsible for one of the most embarrassing recent episodes at Science, the July 11, 2014 cover of the special AIDS issue. This cover, for which Science has been widely excoriated, featured the bare legs [and no faces] of transgender sex workers in Jakarta, which many saw as a crass objectification and exploitation of these vulnerable individuals. Marcia McNutt was forced to publicly apologize for this cover, although she partly defended it as the result of “discussion by a large group.” In fact, my understanding, based on sources I consider reliable, is that a number of members of Science’s staff urged Covey not to use the cover, to no avail.)
Remember this little oopsie?
This will be interesting to watch, particularly if we hear more about the July 11 cover and any possible role that the individuals Balter references in this statement, "The recent dismissal of four women in our art and production departments", had in the opposition or approval argument.
Dr. Harris [Wikipedia] was a Physician Scientist prior to running for Congress
and was the PI of NIH Grant R01 GM036044-04. This grant ran from 1986-2007 and was competitively renewed three times. This, in my mind, gives us much more reason, than is usual for a Congress Critter, to hear the man out.
Congress should also mandate that the median age of first research awards to new investigators be under 40 within five years, and under 38 within 10 years. Failure to meet these benchmarks would result in penalties for the N.I.H., including possible funding cuts.
I am in full agreement. I might even want more aggressive benchmarks. Telling NIH that they must address this is fine because pretty much all stakeholders have been agreeing this is ridiculous. There is very little that I have heard in the way of any serious argument that this increasing age of first-R01 award is a good thing. Now....what I really want to see is the inter-quartile range. What we see is the median and it simply doesn't square with my seat of the pants estimate. It looks really old to me, meaning I know a lot of people that got their start in their early to mid thirties (yes, within the last 15 years, thanks) and very few that got their first R01 at 43 or older. It may be the case that the distribution is nearly Gaussian....or it could be really skewed. It would not surprise me in the least if the 25th percentile is age 36 and the 75th percentile is age 45, for example. I want to see the entire distribution, ideally, but the inter-quartile range would be a good substitute.
To make sure it meets those goals, we should insist on the development of an N.I.H.-wide strategic plan — not just for targeting younger researchers, but for prioritizing different avenues of research overall. Today we see too many grants going to things like creating a video game for moms to teach them how to get their kids to eat more vegetables, or studying the creation of a social security system in southern Mexico. Such projects may have value to some, but is creating a video game really more important than researching a cure for Alzheimer’s?
It is easy for anyone to point to some "ridiculous" grant award or study topic that revs up their base. Whether you are opposed to research into topics that are "solved" in the Republican mind with prohibition and moral tut-tutting (HIV/AIDS, drug abuse), in the Democratic mind with hippie veganism and anticorporatism (diabetes, heart diseasee) or in the waccaloon mind by denial (ban all animal research, for example) isn't really important. I can show you how stupidly irrelevant some basic research is, Sarah Palin can dismiss drosophila models or PP and St. McKnight can insist that only "vertically ascending science" is relevant to real advance. We're all wrong. The tremendous strength and success of the NIH-funded research enterprise relies intimately on the relative absence of top-down control. Investigator initiated science is the best way, of a myriad of options. Period. When we try to be "efficient" by picking winners in advance, we hinder scientific advance. It is really surprising Dr. Harris doesn't realize this, even if Representative Harris feels compelled to advance the standard right wing attack against science that discomforts their constituencies of Big Business and Social Conservative Theocrats.
The final agenda item of Rep Harris is, I believe, the true agenda. The bone thrown to young investigators is only a sweetener, I would bet. He wants to end the "tap".
For one thing, we need to eliminate a budget gimmick, known as the “tap,” that allows the Department of Health and Human Services to shift money out of the N.I.H. budget into other department efforts. The N.I.H. lost $700 million to the “tap” in 2013 alone. Instead, the money should be placed under the control of the N.I.H. director, with an explicit instruction that it go to young investigators as a supplement to money already being spent. If we don’t force the N.I.H. to spend it on young investigators, history has shown that the agency won’t.
And what is this, you ask? Datahound to the rescue:
DJMH: The HHS Secretary has the authority to transfer funds for Program Evaluation. This has been down routinely for more than a decade to fund AHRQ and other agencies and to support program evaluation at NIH.
The NIH and other Public Health Service agencies within HHS are subject to
a budget “tap” called the PHS Program Evaluation Transfer, authorized by section
241 of the PHS Act (42 U.S.C. § 238j). It is used to fund not only program
evaluation activities, but also functions that are seen as having benefits across the
Public Health Service, such as the National Center for Health Statistics in CDC and
the entire budget of the Agency for Healthcare Research and Quality. These and
other uses of the evaluation tap by the appropriators have the effect of redistributing
appropriated funds among PHS agencies. The FY2005 and FY2006 L-HHS-ED
appropriations set the tap at 2.4%, as does the FY2007 Senate bill. The House bill
returns the maximum tap to 1.0%, the level specified in the PHS Act. Since NIH has
the largest budget among the PHS agencies, it becomes the largest “donor” of
program evaluation funds and is a relatively minor recipient.
Okay....you think to yourself, why would a Republican Congressman be so het up over this? Well, if you do some judicious googling about AHRQ you find things like this.
The U.S. House Committee on Appropriations released their draft 2013 Labor, Health and Human Services funding bill. In their summary, the number one stated intent is the following:
“Defunding ObamaCare – The legislation contains several provisions to stop the implementation of ObamaCare...
One extreme cut that was thrown in the draft was not just defunding, but total termination of the Agency for Healthcare Research and Quality, a tiny agency under the Department of Health and Human Services. AHRQ has a budget of $405 million.
That looks to be more than half of the "tap" that Rep Harris wants to close. And this brings us to wonder if Dr. / Rep Harris has any opinions on the ACA?
So, Dear Reader, I confess I come away from Rep Harris' Op-Ed with a feeling that the true agenda here is a very familiar right-wing Republican one that goes after part of the Affordable Care Act and attempts to gain additional direct say over what grants the NIH funds. The part about supporting younger scientists is merely a convenient ploy to sweeten the deal and attract the unwary. I don't believe it is the true purpose here.
Again, according to the Peer Review Notes for September 2014, CSR of the NIH says that applications are up by 10%.
“Total numbers of applications going to CSR study sections have surged about 14 percent," said CSR Director Dr. Richard Nakamura. “The NIH Office of Extramural Research reports about a 10 percent increase in research project grant applications across NIH.”
The difference in the two number is because "CSR is reviewing a slightly larger portion of NIH applications (79%) now than before.", the balance are reviewed in study sections managed by ICs themselves.
Why the bump in applications?
The CSR appears to be blaming this slight increase on the revision of the policy regarding resubmitting previously unfunded applications. As you know, if your revised version (A1) of a proposal is not funded, you may now resubmit it as a "new" application, making no mention whatever of the fact it was previously reviewed.
“It’s clear a large part of this increase is due to NIH removing limits on resubmitting the same research idea,” he said. “The new policy was designed to keep alive worthy ideas that would have been funded had the NIH budget kept up with inflation.”
Obviously, success rates will go down since I see very little chance the budget is going to increase any time soon. The only possible bright spot would be if the recent award of BRAIN Initiative largesse frees up the regular funds within the general framework of neuroscience that would otherwise be won by these folks. I am not holding my breath on that one.
This bump is hitting around the time of the beginning of the fiscal year when there is no appropriation from Congress, as usual. So grants submitted in the summer (first possible after the policy change) will be reviewed this fall and sent to Advisory Councils in January. My assumption is that we will still be under a Continuing Resolution and many ICs will be conservative, per their usual practice.
So anyone who has proposals in for the upcoming Council round has a bit of extra stress ahead. Tougher competition at review and uncertainty of funding all the way through Council. Probably start hearing news about scores on the bubble in March if we're lucky.
The really interesting question is whether this is a sustained trend (and does it really have anything to do with the A2asA0 policy shift).
I bet it will be short lived, IF it has anything to do with that change. Maybe just that one round or at best two rounds and we'll have cleared out that initial exuberance, is my prediction.
The Peer Review Notes for September 2014 contains a list of things you should never write when reviewing grants.
Some of them are what we might refer to as Stock Critique type of statements. Meaning that they don't just appear occasionally during review. They are seen constantly. A case in point:
7. “This R21 application does not have pilot data, which should be provided to ensure the success of the project.”
Which CSR answers with:
R21s are exploratory projects to collect pilot data. Preliminary data are not required, although they can be evaluated if provided.
What kind of namby-pamby response is this? They know that the problem with R21s is that reviewers insist they should have preliminary data or, at the least only give good scores to the applications that have strong preliminary data. They bother to put this up on their monthly notes but do NOTHING that will have any effect. Here's my proposed response: "We have noticed reviewers simply cannot refrain from prioritizing preliminary data on R21s so we will be forbiding applicants from including it". Feel free to borrow that, Dr. Nakamura.
“This is a fishing expedition.”
It would be better if you said the research plan is exploratory in nature, which may be a great thing to do if there are compelling reasons to explore a specific area. Well-designed exploratory or discovery research can provide a wealth of knowledge.
This is another area of classic stock criticism of the type that may, depending on your viewpoint, interfere with getting the desired result. As indicated by the answer, CSR (and therefore NIH) disagrees with this anti-discovery criticism as a general position. Given how prevalent it is, again, I'd like to see something stronger here instead of an anemic little tut-tut.
One of these is really good and a key reminder.
“The human subject protection section does not spell out the specifics, but they already got the IRB approval, and therefore, it is ok.”
IRB approval is not required at this stage, and it should not be considered to replace evaluation of the protection plans.
And we can put IACUC in there too. Absolutely. There is a two tiered process here which should be independent. Grant reviewers take a whack at the proposed subject protections and then the local IACUC takes a whack at the protocol associated with any funded research activities. It should be a semi-independent process in which neither assumes that the approval from the other side of the review relieves it of responsibility.
Another one is a little odd and may need some discussion.
“This application is not in my area of expertise . . . “
I find that reviewers say this in discussion but have never seen it in a written critique (even during read phase before SRO edits eliminate such statements).
The response is not incorrect...
If you’re assigned an application you feel uncomfortable reviewing, you should tell your Scientific Review Officer as soon as possible before the meeting.
...but I think there is wiggle room here. Sometimes, reviewers are specifying that they are only addressing the application in a particular way. This is OKAY! In my experience it is rare that a given application has three reviewers who are stone cold experts in every single aspect of the proposal. The idea is that they can be primary experts in some part or another. And, interestingly given the recent statements we discussed from Dr. McKnight, it is also okay if someone is going at the application from a generalist perspective as well. So I think for the most part reviewers say this sort of thing as a preamble to boxing off their areas of expertise. Which is important for the other panel members who were not assigned to the application to understand.