However, you will be interested to hear the new ASBMB President's analysis of the real problem in science today.
Two or three decades ago, this system was effective, yet I now judge it to be flawed. What went wrong? I submit that the demise of the review process can be attributed to two changes. First, the quality of scientists participating in CSR study sections two to three decades ago was, on average, superior to the quality of study section participants today. Second, study sections have become highly specialized such that they narrowly define a differentiated club of biomedical research.
Hrm, hrm, I submit to you, hrm hrm, that I and my fellow Great Men of Science used to get our grants funded every time we submitted them. Now things have changed! Clearly there is a problem that is simple to diagnose. Read on.
Among all problems leading to devolution of CSR study sections, the elephantine expansion of the biomedical research complex tops the list. Biomedical research in the 1960s and 1970s was a spartan game.
Wait, what? Does this guy know it is 2014? That would make it three to four decades ago...but who is counting.
First, the average scientist today is not of the quality of our predecessors; it’s a bit analogous to the so-called “greatest generation” of men and women of the United States who fought off fascism in World War II compared with their baby boomer children. Biomedical research is a huge enterprise now; it attracts riff-raff who never would have survived as scientists in the 1960s and 1970s. There is no doubt that highly capable scientists currently participate in the grant-review process. Likewise, unfortunately, study sections are undoubtedly contaminated by riff-raff.
Hmmmm. Okay so let me just think about this. The greatest change in the population of Professor-scientists since the 1070s is....hmmm. Women? Persons underrepresented in the sciences (and academia)? Those from a less-well-off background? All of the bloody above?
Good LORD what a snob.
A second cause of the demise in study section quality can be attributed to the fact that it is a thankless task balanced by few benefits. Three to four decades ago, it was a feather in one’s cap to be appointed to an NIH study section. When I joined the molecular cytology study section in the 1980s, Bruce Alberts was chairman, and the committee included the likes of Tom Pollard and all kinds of superb scientists. To spend three meetings per year with an esteemed group of scientists was both inspirational to me as a young scientist and of tangible value to my maturation as a researcher.
Less than one decade ago I went on study section and it was considered by all of my local peers (and members of my field external to my University) to be quite a feather in my cap. I also had the opportunity to spend three meetings a year with an esteemed group of scientists in my field. It was inspirational to me and of tangible value to my maturation as a scientist.
Wow. It is almost like nothing has changed.
Finally, as study sections have become ever more dedicated to thin slices of the biomedical landscape, participants are exposed to less and less science outside of the narrowly defined disciplines covered by their individual study sections. As such, one rarely learns much of anything new by participating in an NIH study section meeting.
I can't speak to every study section but I did have the good fortune to be appointed to one with a broad mission. Broad across IC assigned for possible funding, broad in study subject population / models and broad in scientific questions being asked. From my perspective, of course. Not sure how Dr. McKnight would see it but I'm willing to argue the merits of that study section on the stats.
let’s consider what might be expected from a grant review committee composed largely of second-tier scientists with limited knowledge of the breadth of biology and medicine. I propose that these committees are equally good at ensuring that the worst and best applications never get funded. They can see a terrible grant wherein the science is flawed and the investigator has no track record of achievement. These committees are, unfortunately, equally good at spotting and excluding the most creative proposals – the grant applications coming from inspired scientists whose research is damned because it is several steps ahead of the curve and damned because it comes from an applicant not blessed with club membership.
as they say on Wikipedia, .
Now, to the second of two evils: the evolution of scientific clubs. Back when we used to “walk miles to school,” the scientific meetings we attended had some level of breadth.
Ok. If this is an intentional troll, this is the tell right here. Right? Right?
Fast-forward 30 years, and what do we now have? The typical modern biomedical meeting spends a week on a ridiculously thin slice of biology. There are entire meetings devoted to the hypoxic response pathway, sirtuin proteins, P53, mTor or NFkB. If a scientist studies some aspect of any of these domains, he or she absolutely has to attend these mindless meetings where, at most, some miniscule increment of advancement is all to be learned.
Is it possible that science really has gotten more complex so that low hanging fruit cannot be scooped up by a generalist, gentleman dilettante anymore? I mean, it couldn't be that, could it?
Damn the fool who does not attend these meetings: The consequence is failure to maintain club membership. And why is club membership of such vital importance? Yes, precisely, there is nearly a one-to-one correspondence between these clubs and CSR study sections.
Actually, the study section I spent the most time reviewing for contained LOTS of people that tend to attend meetings that I do not attend. I met many of them for the first time on study section. So the "one-to-one correspondence" simply doesn't square with my experience. Oh, and btw, I have ad-hoc'd on panels that are related but have a more defined focus than the one I served as an appointed member. Guess what? Also a dearth of "one-to-one correspondence" with any particular scientific meetings I or they attend.
So, to wrap up, this guy has zero data for his assertions, they come from the expected direction of unearned privilege of those who got their starts in the 1960s and early 1970s and his claims about modern study section are at considerable odds with my own experiences.
I will be fascinated to see if my readers share his view on the modern study section experience.
ETA: A check of Dr. McKnight's funded grants from the NIH suggests he is no stranger to the Special Emphasis Panel.
UPDATE 10/08/14: A Nature News piece notes that McKnight 'was “saddened” by suggestions that he has any gripe with young researchers or with diversity. He meant to criticize review committees as a whole, not just young scientists, he added.'
h/t: Some twitter troll who shall not be identified unless s/he so choses.