McKnight doubles down on "riff raff" on NIH grant review panels

Oct 08 2014 Published by under Careerism, Tribe of Science

We recently discussed a President's column at ASBMB Today by Steven McKnight in which he claimed that NIH grant review panels are contaminated with "riff-raff" who are incompetent to properly review proposals.

A Nature News piece published today notes that McKnight 'was “saddened” by suggestions that he has any gripe with young researchers or with diversity. He meant to criticize review committees as a whole, not just young scientists, he added.'

Very interesting. He was saddened, apparently, that not enough scientists understood he was calling them riff-raff! It is not only the junior scientists, it is also everybody else that he despises.

“A level of mediocrity has crept into the grant-review system,” he said. He recalled that earlier in his career, grant-review committees were packed with well-known scientists with established credentials. “Now when I look at the list, I’ll know zero names. Five or six of them will be people from fly-by-night biotech companies.” He said that he hasn’t done any quantitative research on this trend, “but I think I’m probably right”.

Let's just unpack that "I'll know zero names" business. There is an alternative hypothesis here, which is that perhaps McKnight doesn't "know" the names of people that he should.

Leaving aside whether or not McKnight actually performed any review of scientific quality per se, let us recall what happens in any age graded social situation. For my US readers in particular, I will remind you of who you "knew the names of" when you were a Freshman in High School or College versus who you "knew the names of" when you were a Senior in those respective environs. I submit to you that, as was my experience, when are looking up the social ranks, you know a heck of a lot more people than when you are looking down the social ranks.

This squares entirely with my direct experiences with my older peers in science. It is not infrequent that I refer to someone I think of as a hot Young Gun of science and the oldster has no idea who I am talking about. Even when the oldster has been impressed by the work that person has done in OtherOldster's lab but still mentally tags it to the OtherOldster. It is only with time, repetition and further excellence from the Young Gun that my acquaintance oldsters come to "see" the name of the Young Gun.

Note, this can be well into said Young Gun's independent career as an Assistant Professor.

I am not trying to excuse McKnight's snobbiness here at all. I am mentioning a common social experience that has to do with the accident of age and relative stature and has essentially nothing to do with relative merits of the people one "knows the name of".

Given that this is so common, however, you might think one would be hesitant to bray on about people's merits as a scientist based on whether your rapidly aging (and clearly not the most socially tuned) brain happens to recognize their name.

Note: Just for grins I'm reviewing the panel rosters in the
Integrative, Functional, and Cognitive Neuroscience IRG [IFCN]. These eight panels review a lot of Neuroscience grants. Admittedly I scanned quickly, and did not review the meeting rosters for ad hoc members, but I found zero individuals from biotechs. I also note that the Universities are heavily dominated by the R1s and the non-University Institutions represented are very well known Med schools and Research Institutes. I don't think I am even above the mean (or 25th percentile, frankly) in being able to recognize names across all of neuroscience.....but even scanning quickly I saw some big names all across these groups.

This may not be comprehensive data either but it sure as heck overmatches McKnight's "I think I'm probably right" comment.

56 responses so far

  • Dave says:

    What a terrible, terrible article that is in Nature. I love the CNN-style attempt at keeping it 'balanced' with some quotes form that dbag riff-raff oncologist Emil Lou at the end.

    Just call it for what it is Nature. What he said was ridiculous and he needs to go. Simple as. Nature is fucking spineless.

  • Paul Brookes says:

    The "I think I'm probably right" line really tells us everything we need to know about this guy's attitude. I only joined ASBMB to get cheaper page charges for JBC - paid for itself in a single paper. With McKnight at the helm I won't be renewing.

  • StrangeSource says:

    Wow, that's sad. I'm bothered by more than the name calling.. I didn't know McKnight's name myself, but stalking him a little, it's clear he's a science superstar. Part of me wants to acknowledge that he has a point in there somewhere... I have a soft spot for snobs. But he shoots himself in the foot (repeatedly) with his total lack of clues. It was the success of he and his greybeard buds that grew the scientific enterprise. It was they that wrote the grandiose grants that required armies of grad students and post-docs to execute. It's like the pharaoh standing atop the pyramid, looking around at hundreds of workers wielding chisels and pushing granite blocks, swishing your crook and flail saying, "you just *can't* find good help anymore."

  • drugmonkey says:

    Dave- Perhaps Nature is not "spineless" but rather agrees with McKnight or at least sees itself as catering to a demographic that agrees with McKnight?

  • qaz says:

    Has it ever occurred to this character that maybe he's actually out of touch with the times and the rest of us have moved on beyond where he is? Maybe he's the riff raff?

    I know that every person on every study section I have been on has been a professor (of some rank) at a major institution doing work that *I* knew of.

    I wonder if this McKnight guy knew the names of the three people who won the Nobel prize this week.

  • Amazing that no one at ASBMB managed to tell McKnight to put down the shovel and stop digging!

  • Jim Woodgett says:

    Paul, likely the primary rationale for ASBMB membership (and true of several other societies).

    Must use that line (I've no data but I'm probably right") in my next grant application. Am sure it'll go down well.

  • Philapodia says:

    This just solidifies my view of McKnight's lack of leadership and his lack of understanding for how his comments reflect on ASBMB and the greater scientific community. If someone at the helm of a visible scientific society doesn't have the political acumen to think before he speaks (or at least think of the ramifications of his words), perhaps he shouldn't be in that position.

    Datahound, could you enlighten us on how the presidency of ASBMB is determined? Is it by all of the members, self-nomination, or is the President selected by a small group of individuals?

  • drugmonkey says:

    The fact that when Nature drew him out, McKnight came up with more easily falsifiable pseudo-data "five or six...from...biotech companies" is instructive. This is why when authoritah figures spout out some good sounding BS, it is key to question them. Let them play out the rope. That way you can find out if they might really have a point or if they are totally making stuff up out of whole cloth.

  • drugmonkey says:

    and I rush to add...I am NOT agreeing with McKnight that employment at a biotech, fly-by-night or other, is of any particular value in estimating scientific acumen.

  • Philapodia says:

    Was he looking at a SBIR/STTR panel maybe? Those dagnab translational types keep messing thing up!

    "They're in it for the money not the science!"
    -Bill Harding, "Twister"

  • chemicalbilology says:

    Seriously--I've met several people at study sections for technology and drug development-type grants from biotech companies, and they have all been highly experienced and offered crucial insight in their relative areas of expertise. Like I said before, McKnight's attitude is really insulting no matter whether you're junior or not, and is inappropriate for someone in his role .

  • sk says:

    McKnight says "grant-review committees were packed with well-known scientists with established credentials". What does "established credentials" mean? Quite apart from the fact that "well-known" is a very subjective term as DM points out, why should one assume that well-known scientists are better reviewers than not-so-well-known? Are there any data that support this hypothesis?

  • zb says:

    "He said that he hasn’t done any quantitative research on this trend, “but I think I’m probably right”."


    Maybe his grants and manuscripts end that way, too. "I haven't done any quantitative research on this hypothesis, 'but I think I'm probably right.'"

  • Philapodia says:

    "established credentials" = someone whose work has validated my vertically ascending hypotheses

    "Well-known" = drinking buddy I've shot the shitte with many times before at meetings

  • dsks says:

    "He said that he hasn’t done any quantitative research on this trend, “but I think I’m probably right”.

    The scientific rigor of this fellow is staggering. If only we had people on the study sections with standards as high as this. The "riff raff" would be doomed.

  • dsks says:

    Zb said,

    Maybe his grants and manuscripts end that way, too. "I haven't done any quantitative research on this hypothesis, 'but I think I'm probably right.'""

    You can't refudiate the truthiness, zb.

  • drugmonkey says:

    Great scientists come up with ideas and look for confirming evidence later, dsks. Only the schlubs need data first.

  • Dave says:

    Dave- Perhaps Nature is not "spineless" but rather agrees with McKnight or at least sees itself as catering to a demographic that agrees with McKnight?

    Maybe. But I think the article is just another example of the media's obsession with 'fair and balanced' when, in fact, in this case there is absolutely no 'other side of the debate'.

  • Comradde PhysioProffe says:

    Nature is trolling for page views.

  • Noncoding Arenay says:

    "grant-review committees were packed with well-known scientists with established credentials"

    Oh, you mean your big boys club? Referring to one of my comments on DM's previous post on the McKnight issue, I'd like to draw attention to the likes of not-well-known-but-clearly-talented scientists such as Yitang Zhang as a direct counter to this assumption.

    I agree with every comment here. It is almost comical how these thoughts did not cross his mind. Or maybe it demonstrates the (privileged? prejudiced? bigoted?) wall that might have emerged in his head, which prevents him from thinking along these lines.

  • Dave says:

    I think Nature is turning anti-Drugmonkey.

  • datahound says:

    Philapodia: The Presidency of ASBMB is determined by an open election of all members. The candidates are chosen by a nominating committee. I do not know what percentage of the membership actually votes.

  • Established PI says:

    I do agree that McKnight dug a deeper hole for himself - he should have done his homework after shooting from the hip the first time. But I also agree that study sections vary in quality and there have been some where I just scratched my head and had to try to figure out what on earth some of the members have ever done. The problem is that McKnight overstates, paints with a vastly broad brush and doesn't come up with any specifics that could even masquerade as facts.

  • Philapodia says:

    Thanks Datahound. Just curious how the position gets filled.

  • Hermitage says:

    Oh yes, we've never seen an uninformed member of his particular demographics come to a conclusion based on, you know, inherent rightness.

    I hope he's never a reviewer for these poor, incompetent, riff-raff scientists he can't be arsed to learn the names of.

  • Davis Sharp says:

    To quote a baby boomer cartoon character, which is much better drawn and voiced than today's riff-raff cartoon characters, "What a maroon!"

  • dsks says:

    "But I also agree that study sections vary in quality and there have been some where I just scratched my head and had to try to figure out what on earth some of the members have ever done."

    Yeah, but the question is what fraction of the reviews we get are generally and objectively Out There? And what fraction of these are genuinely the only reason a given proposal was shot down? Everybody has a few anecdotes, but take that against the number of applications they've submitted and my bet is that the incompetent reviewer remains a rarity; albeit one that gets blown out of proportion simply because the odd shitty review sticks in the mind more than the 20 fair reviews that came before it.

    (...Unless of course you're as unlucky as I routinely am and can legitimately complain that 100% of the reviewers on your rejected proposals have been barrel-scraping, backward-looking charlatans, poseurs and assorted riff-raffians who wouldn't know true genius if it slapped them upside the head with a lemon wrapped around a large gold brick stamped in pink lettering with, "GENIUS"... but generally speaking, a rarity...)

    Even if there is an uptick in the fraction of shitty reviewing of late, it's likely as Datahound suggested that this has less to do with competence so much as reviewers having an increasingly difficult time discriminating between high quality proposals (like mine).

  • drugmonkey says:

    And for the Out There reviews, how many are written by the ClearlyCredentialed minority?

  • Philapodia says:

    As the SRO is responsible for finding reviewers for study section, and riff-raff reviewers are the causing the intellectual decline of the biomedical research-industrial complex, this argument implies that SROs are the reason that the Scienz is doomed. Ergo, SROs are to blame for everything (Unless you're the SRO on the study section my grants go to. You're wonderful, you crazy kid you.).

  • Joe says:

    I recently heard an SRO talking about the difficulty of getting some people to serve on study section. In particular, well-funded established investigators that have little to gain by spending time on study section. They are already successful at working the system, and apparently feel no obligation to give back to the system that is rewarding them. NIH has considered requiring (willingness for) study section service as a condition of accepting an award.

  • halcyon says:

    All this talk about the ubiquity of riff-raff is really fueling my already rampant case of imposter syndrome.

  • drugmonkey says:

    McKnight types would be hilarious to have on study sections. Bring them on! (Not the ones I use though)

  • E rook says:

    My experience with industry scientists has been that they are concerned with timelines, feasibility, real world impact, applicability of research. They have been highly intelligent and discerning. I think their input on NIH grants would be of high value. I don't have quantitative data, but I'm pretty sure I'm right. Further, I check the roster of my SS of interest for the past 3 cycles, including ad hoc members, not one is from industry. I recognized the names of about 70% of them and the rest I looked up on Pubmed, teh google, and Reporter. Lo and behold, I learned something new. My SS has a diverse group on it, wrt career levels, institution types, and even some NIH folks. To me, it looks appropriately populated. I don't have quantitative data on this, but I'm pretty sure I'm right. McKnight seems to have made the mistake of punching down. This demonstrates lack of leadership, lack of wisdom, and bulliness (which is a marker for lack of competence). I don't have quantitative data on this, but ...

  • The study section I serve on has a number of members who on paper could be like McKnight: chasing vertically ascending science, glamorous CVs, etc. But they are all decent folks, and do a good fair job; not flaming dumpster fire assholes like McKnight. And yes, I do attribute this to the SRO being truly outstanding at selecting the right people and shepherding the review process.

    As far as incentivizing highly accomplished senior people to serve, do not underestimate the value of having your grants reviewed by member conflict SEPs. There are structural features of scoring behavior as a consequence of their being percentiled to the all-CSR base that can be hugely beneficial.

  • drugmonkey says:

    I also attribute it to the fact that when someone actually serves on a study section it becomes increasingly difficult to maintain some of the sorts of claims made by McKnight and other people who are dissatisfied with the process. Actually doing the job has a way of making you appreciate the outcome of the peer-review process at CSR. You may not always like it, but it can disabuse you of the notion that it is conducted by evil incompetents out to get your applications, personally, for demented reasons.

  • @Davis Sharp
    Nah, Bugs Bunny predates the Boomers -- he's pre-WWII, although I'm sure the Boomers enjoyed him just as Gen-X and presumably the Millennials do.

  • Davis Sharp says:

    Why would I ruin a perfectly snarkiferous comment with fact-checking?

  • Philapodia says:

    "The study section I serve on... I do attribute this to the SRO being truly outstanding at selecting the right people"

    He saw past his blustery exterior and uncovered the special snowflake that is CPP.

  • John says:

    Give poor Dr. McKnight a break. Probably, still bitter that he got kicked out of the glamour filled halls of the private science club down the street in Chevy Chase without even one renewal. Can't be easy to walk back with your tail between your legs to a drab Bethesda conference room to slum it with the mediocre, unknown masses on study section. Oh what he would do to sip just one more drop of beer in the gilded rathskeller of Kandel and Horvitz, hoping they would one day acknowledge him by name.

  • Crystaldoc says:

    Could CPP or DM please elaborate on "the value of having your grants reviewed by member conflict SEPs. There are structural features of scoring behavior as a consequence of their being percentiled to the all-CSR base that can be hugely beneficial"??

    What structural features?

  • drugmonkey says:

    HHMI is full of riffraff these days John. I haven't done the quantitative research but I'm pretty sure I'm right.

  • In regular study sections, scoring behavior is overwhelmingly dominated by the command to "spread your scores", which means that you can only give a very small number of outstanding scores. This is because the scores are percentiled against themselves, and if you give a bunch of super scores, then you compress the percentiles. In an SEP, the scores are percentiled against all CSR, which means that the scores you give have no influence on the percentiling, which means that the study section can give all 10s, and every grant will get a 1%ile. This frees the reviewers to compress scores all they want and to give as many outstanding scores as they want.

  • drugmonkey says:

    And the SRO says nothing?

  • crystaldoc says:

    What determines whether a grant from a member of a study section gets reviewed by a member conflict SEP or by a different but closely related study section in the same IRG?

  • drugmonkey says:

    My impression is that there has been a lot of effort to minimize SEPs lately. I'd say it only goes to SEP when they can't get a fit elsewhere. But this is only from indirect evidence.

  • The SROs of SEPs that are percentiled against the all-CSR base behave differently than SROs of standing study sections that are percentiled against only themselves. Obviously, they tell you to score fairly, but there is a much smaller number of grants to review, and the overriding structure isn't one of ranking apps, which is how it is in a standing study section.

    As far as getting assigned to an SEP, the way to guarantee it is to submit your grant too late for it to be handled by a regular study section for that council round. If you read the rules for continuous submission, it gives some date certain that guarantees peer review in time for each particular council round. However, if you talk to the review staff in your IRG, they will (or at least mine always do) tell you a substantially later date for submission which will be in time for consideration in that council round. That later date is much too late for assignment to a regular study section.

  • The Other Dave says:

    Smart established professors stay away from study section. It's hard work, you don't need the 'service to the profession' credentials, and funding rates are so low that mostly you'll just get blamed for ruining other people's lives. Stay home and get science done.

  • datahound says:

    John and Drugmonkey: In 1991, Steve McKnight left his HHMI position to become Chief Scientific Officer at Tularik, a drug development company founded in the San Francisco Bay Area by Bob Tjian and David Goeddel. Steve McKnight was recruited as Chair of Biochemistry at UT Southwestern in 1996. Tularik was subsequently purchased by Amgen in 2004.

  • drugmonkey says:

    So he himself is biotech company riffraff? That is awesome.

  • MoBio says:

    @ the Other Dave

    "Smart established professors stay away from study section. "

    I guess that makes me a 'dumb established professor' as I and many 'established' colleagues regularly pitch in.

  • MoBio says:

    @the Other Dave:

    Check out the roster from the October 2014 of MOLECULAR NEUROPHARMACOLOGY AND SIGNALING STUDY SECTION

  • MoBio says:

    Riff raff indeed

Leave a Reply