On 'Rescuing US biomedical research from its systemic flaws'

Apr 14 2014 Published by under Fixing the NIH, NIH

I was alerted on the Twitts

to a Perspective by Bruce Alberts (former Science EIC), Marc Kirschner (BSD), Shirley Tilghman (working tirelessly on "fix the NIH" committees) and Harold Varmus (former NIH Directior, current NCI Director). In Rescuing US biomedical research from its systemic flaws the authors recognize the current dismal state of affairs and issue several calls to action.

You will find nearly everything that has been put on the table here at this blog, at Rock Talking blog and, really, everywhere the discussions are serious about NIH-funded biomedical research. I find much to like in here and of course I disagree with several of their points and/or solutions.

Nevertheless, the overall goals are pretty good. They want to evolve to some more HHMI-like stability of funding, an equilibrium of participating scientists to the available budgets and to generally keep the grant process on improving scientific productivity rather than hampering it.

Overall there are good things being said here. But the specifics will be key. They argue for funding more graduate students on fellowships to give more control to the federal government. They phrase this in terms of "quality control" but they overlook using the bully pulpit to demand training programs actually reduce their output of PhDs! On the postdoctoral side of the over supply issue, they call for raising postdoctoral salaries until staff scientists are an attractive alternative. Then they call for increasing the ratio of staff scientist positions. I agree wholeheartedly but am not sure raising postdoc salaries is the way to go. There needs to be some additional mechanisms to provide a carrot to the expansion of the staff scientist category.

The Perspective also goes after the incentives local Universities, Med Schools and Research Institutes have to employ soft-money faculty. Proposals include going after soft-money job types, indirect cost recovery on faculty salary and some obscure but important stuff about including new construction costs in the IDC calculation.

Their recommendations for peer review changes are a mixed bag. They want to dismantle geographical diversity requirements for study section panels (O Rly?) and increase the number of oldsters on the panels. There is some of the usual blah-blah about risk taking, innovation, longer term outcome expectation and identifying the "strongest candidates for support".

What I don't like about these proposals is the stench of elitism that encircles them. As I have said repeatedly, one of the design virtues (even if imperfectly realized) of the NIH system is the respect for the democracy of ideas. In theory, anyone can propose a fundable study by coming up with a great idea that simply must be done. The alternative that is being proposed here is that we select good scientists at the beginnings of their careers and that is it. They get the money even if someone else has a better idea. Because that someone else may not have been selected in the beginning and will now never be able to get into the racket.

The racket will be dominated by pedigree, you better believe it. Not good.

There is one statement about peer review that I find hilarious:

Senior scientists with a wide appreciation for different fields can play important roles by counteracting the tendency of specialists to overvalue work in their own field.

...because "senior scientists" never "overvalue work" based on their biases, right? Please. I've said this many times in the context of grant review. Everyone has biases. EVERYONE! The only tried and true solution is the competition of biases which requires diversity. We need junior scientists in the mix. We need people from a diversity of institution types. We need people of diverse backgrounds and interests. We need a diversity of scientific approaches, orientations and interests. All competing on a more or less equal footing. It is the only way to minimize biases of review.

__
UPDATE/PostScript: I think these authors also fall into the usual trap of thinking that they know who the best scientists are and that if we could just get the money in their hands then the best science would result. It is the trap of thinking we can actually predict where the advances will come from. Read the history of optogenetics. Did it require Deisseroth? Would we have gotten there eventually? Would Deisseroth even have been funded under the New World Order envisioned by the Alberts et al Perspective piece? Has the infinitely more translatably useful DREADD technology been underdeveloped because of the shadow cast by optogenetics? In the New World Order might DREADD have been prevented entirely?

52 responses so far

  • Anon Female Scientist says:

    I have to agree with your take on the "oldsters" and "senior scientists" and elitism - IME, they all just seem to promote the folks that are already part of their circles. I tend to find the younger scientists to be more democratic. Hear ya on diversity, in general, too.

  • drugmonkey says:

    I'm not arguing panels should be all youngsters either. The point is that we ALL have biases and some of them are associated with our various categories. Diversity on as many axes as possible is the best way to minimize bias.

  • Anon Female Scientist says:

    I agree.

  • Viroprof says:

    "Providing resources to those scientists who are most likely to make important contributions over the course of their career is essential for the optimal use of research funds."

    How do we really know who is the most likely to make important contributions over their career? Last I looked, no one at the study sections I've been at had a crystal ball and could accurately divine the future. Perhaps metrics like one or more first author glamour mag article, working in Boston, and/or being the golden boy/girl protege of a Nobel laureate? Or do have to have an continuous R01 for at least 15 years to prove that your ideas are worthwhile? Seems like this will just concentrate the wealth with the haves and screw over everyone else.

  • Pinko Punko says:

    In terms of what is currently wrong with the system, the lack of a star chamber of elites determining who lives and dies based on their inherent biases is probably pretty low on the list. I need to read the piece but on a distant view this seems narcissistic and blinkered. In the absence of increased funds and some sort of cap on students and mechanism for longer term employment of PhDs then it's all just a way to move deck chairs around. What does their method of identifying top investigators entail? Sounds like a lot more 5s for investigator and environment subscores that are unjustufied on any real basis other than "not in club of people regularly validated by Glamour Mag editors"

  • Pinko Punko says:

    I suspect the idea of connecting graduate students to fellowships not projects will be a backdoor way of entrenching an upper class. It seems likely that the criteria recognized for identifying students to fund will sync nicely with the attributes of students competitive for the top programs. This would shrink the student pool nicely because it represent a second bar for lower tier investigators to achieve in addition to getting their grants funded. Especially if there is no geographical diversity allowed, essentially this idea is pretty nefarious. Early stage investigators are always dinged on F grants because of lack of track record so students get cosponsors. The better the cosponsor etc....just think about the ramifications of many of these proposals. I think under the surface this is a pretty big f*ck you to a large chunk of biomedical academia and I also think there are some severe blind spots here.

  • Agree with other comments, but I'll add one more: the staff scientist concept is screwed up. Having been a staff scientist, it's pretty clear to me that the way they envision staff PhDs (long-term association with a PI) will work well for science spouses, but doesn't work for others.

    Underlying this flaw is the belief that universities (versus research centers) are always the best places to do science. If there are funding increases, then maybe some of these additional funds should go towards centers (intra- and extramural) that offer career tracks for staff scientists. Alternatively, NIH should explicitly state that they believe the academic lab/single heroic PI is the wave of the future and just be done with it.

    I'll add a second problem with the article: ultimately, it's the too many mouths at the trough problem (to use a phrase...). We have to come up with reasons to deny people funding (and careers in academic-style science). No one wants to explicitly state that.

    Probably will have to fisk the article....

  • Eli Rabett says:

    As Eli has said before (here too), the problem of overproduction of PhDs is at the top tier institutions with large groups and lots of people. The faculty there, especially in the basic medical sciences, need to do more teaching and get paid more for their teaching so they are not forced to field football teams of grad students and post-docs.

    As to the geographical thing the authors need to learn how to count votes in Congress. A lot of those votes are there for NIH because of the local med schools.

  • Viroprof says:

    "Targets of policy change should include the full reimbursement to amortize loans for new buildings, the payment of indirect costs on faculty salaries, and the provision that allows 100% of faculty salaries to be supported on research grants"

    This was another interesting suggestion in the article. To me this says that the the federal government should be nice and make universities/research centers/med schools have even less skin in the game. It's like when your kid kicks a puppy and you give them candy while telling them not to do it again.

  • AcademicLurker says:

    obscure but important stuff about including new construction costs in the IDC calculation.

    Very important. The zeal of medical centers for putting up fancy new research buildings whether there's a need for them or not has been a significant driver of the runaway growth of biomed research. If the NIH starts to firmly discourage the practice it will, IMO, be a very good thing.

  • Jonathan says:

    Mike - go for it! I was seriously tempted yesterday but can't really given my current day job.

    I guess it's good that they're advocating for some fixes some of us (*cough*me*cough*) have been calling for for years, but at the same time, the chances of Congress budgeting five years ahead or changing the law so that non-US citizens can apply for NIH training awards are simply non-existent, and even less likely next year after the GOP take the Senate as well as the House.

    This bit did make me laugh:

    "Also, to ensure the highest standards of excellence, we propose that objective outside reviews be commissioned at regular intervals to monitor both the value of established programs and the quality of agency implementations."

    Isn't that the exact purpose of each IC's Advisory Council? You know, the outside experts who come in three times a year and ask probing questions about current and future research programs?

    Viroprof, I think you're reading that wrong, they're saying those are current policies that should be changed.

  • Viroprof says:

    @Jonathan
    Perhaps I did read it wrong. I'm only on my second cup of coffee and am still a bit fuzzy. I was under the impression that you can't currently pay 100% of faculty salary via NIH funding. Are there exceptions?

  • dsks says:

    All that guff about "diversifying" graduate education hand-wringing is exasperating. The primary purpose of training graduates in research science should be to fill research scientist positions, whether academic or industry. If the graduates wish to branch out elsewhere into what should be genuinely "alternative" careers that's their prerogative. Any pressure by the NIH to bloat PhD training in order to cater for non-research positions will have no effect other than to hinder the productivity of grad students who are already, imho, significantly hindered by bullshit coursework requirements and teaching. As it is, it's highly questionable that a substantial portion of these "alternative" careers that many graduates are falling into (rather than actively choosing) really require a 4 yr PhD as opposed to a Masters and on-the-job training.

    Implementing policies that substantially raise the bar on the quality of grad students, and insodoing reducing enrollment, is the best course of action. Although, it's important that the genuine intent is to regulate the quality of the students and programs, rather than just use this as a way to control the allocation of bench labor. The cynic suspects that this proposed control over graduate supply is more about making sure the elites continue to get a steady supply of semi-competent gummint-subsidized labor while the cuts are felt elsewhere. The problem with that is that the excellence of a grad program in terms of good training and mentorship does not necessarily correlate with the research prestige of a department or institution; graduates within a small lab at x, y, z university of flyover country very possibly getting a higher quality of training than some poor bugger lost in a lab of 25 staff who can barely remember what their actually PI looks like.

  • SG says:

    One can sum up many of their recommendations by saying that Institutions need to have more skin in the game. If you want a new building, fine, but you pay for All of it. You want more faculty, fine but you have to pay a significant fraction of their tenured salary and you do not get any indirect costs reimbursement for the salary NIH pays. You want to have lots of grad students. Fine, but each of them have to be treated as a student not an expendable college football player.

  • anonymous postdoc says:

    I'm sorry, but the two (different) misspellings of DREADD in a row is killing me.

    I enjoy the hilarious implication that senior scientists aren't specialists with less awareness of what is going on in other subfields than average. A tiny minority of senior PIs are perhaps the sort of emperors who are constantly branching out in new directions because people are seeking out their stature to obtain more attention for the findings by making the Tzar a senior author on their paper. But lots and lots are just really dug deep in their own niche.

    People are spread thin these days. No one has time to read all the papers made in their subfield and everything would be better and more wonderful if they could just get rid of the "lesser" scientists making all these "extra" papers by making sure the limited funding goes to the people who get a Nature paper, not to the people who quietly are trying to inform you about whether the findings of the Nature paper are even replicable.

  • Jonathan says:

    Curtailing US graduate school recruitment is only a partial fix though, because there is an elastic supply of cheap foreign labor just dying to get the fuck out of their home countries and come over to the US.

  • Mike Holloway says:

    Heads up! The "PhD shortage" crowd is still out there. Already some spin out there rephrasing Varmus and Alberts as stating that more funding will take care of everything: http://phys.org/news/2014-04-biomedical-unsustainable.html

  • Joe says:

    I didn't really see a mechanism in the article for increasing staff scientists and decreasing grad students. Are they going to say you can't fund a grad student on an R01, but have to fund them with training grants? Are they going to say that for every grad student in your R01 budget you have to also have a staff scientist? It's going to take some such rule to convince people to move from hiring smart, hungry students to hiring expensive, less-driven staff.

  • drugmonkey says:

    Fixed the DREADD spellings, thanks.

    joe- the staff sci issue was contrasted with postdocs, not grad students. mechanism was to increase the postdoc pa until staff sci was somehow more attractive to the PI.

    Jonathan- yep. which is why taking one bite of the apple isn't going to work. and we all know how large institutions like to make partial stabs at a solution. not optimistic.

    SG- exactly but, oh boy, the stick is going to have to be a big and broad one to have any effect.

    dsks- that alt-careers stuff is now the way that the entrenched interests manage to avoid admitting that the exploitation racket exists. since they are loathe to give that up, they are looking for ways to excuse it now that the dismal-job-prospects issue is so firmly on the table.

    Eli- Varmus has no excuse. this guy is a total washington warrior. he knows better.

  • Dave says:

    Boring. Nothing really new here, and nothing that hasn't already been written on this blog multiple times over the last few years. But it's in PNAS, so.....must be better. Their "ideas" are stock, unimaginative, and mostly unrealistic. This whole mess is like tax reform, climate change, or healthcare reform policy. You can't reform this system without major winners and losers, so we should just stop pretending that we can (see 'alt careers' nonsense). Lots of people will get majorly hurt if we are serious about this, and the pain will disproportionately fall on the young. People will lose jobs, and there isn't anywhere for most of them to go. Fact.

    The only way any of this shit works is if universities/medical schools go back to paying the bulk of investigator salaries and the NIH funds and supplements the research, kind of like what the NSF does. In other words, the traditional tenure-type/funding system. If this happens, by default everything else will eventually fall into place. Universities will cull soft-money jobs, soft-money techs, students and perma-docs/techs. This will negatively regulate graduate school enrollment, post-doc recruitment blah blah blah. But the universities need to get on board. How does that happen while the gravy train is still moving, albeit slowly? I have no idea, but they could never do it without cutting a huge number of jobs. I'm afraid everything else we do or suggest is just delaying this inevitability. I think there is enough NIH money in the system to keep it going for many years, but sooner or later.......

    Their chat about staff scientists is admirable and all, but who the fuck is going to pay their salaries and bennies? The universities? Or the NIH? How are 'permanent' staff scientists any different from perma-docs, or super-techs that exist now? Is it just a title change? A promise of 'stability'? This doesn't solve any problem, it just puts a nice dress on it and calls it something else.

  • Joe says:

    @Dave "The only way any of this shit works is if universities/medical schools go back to paying the bulk of investigator salaries .... If this happens, by default everything else will eventually fall into place. Universities will cull soft-money jobs, soft-money techs, students and perma-docs/techs. This will negatively regulate graduate school enrollment, post-doc recruitment blah blah blah. "
    I don't really buy that making schools and institutes cough up the money for salaries is going to greatly affect grad student numbers and post-doc numbers, unless you mean it will bring the Great Cull and there will be less of biomedical science in general. Given a limitless supply of bright foreigners who want a PhD and to be paid while getting it, PIs are going to continue hiring students. Ditto for post-doc training.

  • Dave says:

    ...unless you mean it will bring the Great Cull and there will be less of biomedical science in general

    Basically.

    ...limitless supply of bright foreigners who want a PhD

    I think the supply of cheap and exploitable foreign post-docs and PhD candidates is probably limitless, yes. The supply of bright and talented foreign students and post-docs is definitely not. The supply of foreign bright and talented students and post-docs who have a realistic shot at a career in academic research in the US is even smaller.

    PIs are going to continue hiring students. Ditto for post-doc training

    Not if they don't have grants and/or a job.

  • AcademicLurker says:

    unless you mean it will bring the Great Cull...

    Seconding what Dave said. I believe DM has made the point on several occasions that most medical schools don't have the money to start paying a significant part of their soft $$ PIs' salaries. That was never part of their business model.

    Cull indeed...

  • rxnm says:

    ""Providing resources to those scientists who are most likely to make important contributions over the course of their career is essential for the optimal use of research funds."

    Breathtaking arrogance, elitism, and failure to understand why the NIH has succeeded to the extent that it has.

    These people are The Problem.

  • anonymous postdoc says:

    God damn it, the thing that really makes me crazy is that all of these policy suggestions are, well, you know, that's just like, their opinion, man.

    Where is the DATA that peer review is improved by having more senior panel members? Oh wait, that would involve testing peer review outcomes and finding out the emperor has no clothes. Where is the DATA that people who have more resources available to them are uniquely qualified geniuses that provide research breakthroughs in a linear proportion to their funding? Oh wait, the available data directly contradicts that assertion.

    The parts of this that need to get done (PhD, postdoc, and research assistant professor employment reform) are the things there is no political will at the NIH to accomplish. The things that are elitist gladhanding "reforms" of peer review could easily be accomplished by fiat. This article, by creating a false equivalence between these issues, could be used to support the latter under the generic banner of "reform", allowing everyone to continue ignoring the former.

  • DrugMonkey says:

    The funny thing here is that Varmus is an Institute Director. He selects grants for funding and is free to pick up low-scoring apps out of order and skip over high-scoring proposals. If this is all so easy to predict excellence, why hasn't he busied himself being the change he wants to see at NCI? Shouldn't cancer be cured already?

  • dsks says:

    "Seconding what Dave said. I believe DM has made the point on several occasions that most medical schools don't have the money to start paying a significant part of their soft $$ PIs' salaries. That was never part of their business model."

    When I was job hunting I had a conversation with - well, maybe more of a lecture from - the head of the anesthesiology dept at a prestigious midwest med school where their vision of future funding for thee research arm of the dept was to draw it primarily from the clinical revenue generated by a predominantly MD/MDPhD faculty, with federal funding being treated as purely supplementary. I have no idea how realistic that vision was (I mean, of course anesthesiologists bring in some serious green, but surely more towards research is less in their pocket?), but it did seem like a reasonable approach by which med schools could maximize value for money on the salaries of their research staff by making sure they were employed as clinicians also (which I would imagine brings in a lot more money than tuition fees and instruction can, hence a preference for research/clinical faculty versus research/teaching professors).

  • Dave says:

    I have no idea how realistic that vision was (I mean, of course anesthesiologists bring in some serious green, but surely more towards research is less in their pocket?), but it did seem like a reasonable approach

    Already happening to a significant degree at my place, at least according to several financial reports and talks that I have read/attended. What is also happening is that clinicians who were doing unfunded research are being told to stop doing that and focus on patient care 100% instead. This is of course to maximize revenue generation, which is then being used to support research by the basic faculty.

    The question is how long will the MDs tolerate this situation, given that they can make substantially more money privately. Institutions risk alienating their clinical faculty with this behavior.

  • DJMH says:

    One of the funniest things about the paper is that the contributing editor is at the Salk institute, a soft-money place that stands to lose a lot if some of these (needed) reforms are implemented. Wonder what the discussions will be like at the next faculty meeting there.

    I still say it would be easiest just to include IDC in grant budgets, so the price of being at a high IDC institute is transparent and comes directly out of the PI's bottom line. Seems like a quick route to motivating indirect reductions. Oh and preventing med schools from double dipping on salaries would also be good.

  • lurker says:

    Too little, too late, and all blather.

    Only Kirschner is the real deal BSD that I thought interesting, given that he's currently pulling almost $TWO Million/year in NIH grants, and he's got to be near double the age of many noob ass profs. That's 8 mod R01's there. Verma is just the figurehead top editor of PNAS, and whether he's just rubberstamping this or not, he also has ~$1.3M/year in NIH grants (5 mod R01's).

    All the others are not running labs anymore, so I don't see them as part of the problem, but Tilghman and others had raised this specter more than a decade ago, when all this perversity was already apparent. But she was outrightly dismissed by the BSDs, so the status quo persists. This article is lip-service from the BSDs to say: "See, we said something for you poor schmucks, now continue starving..."

    The only real "remedy" will be an imminent deep random cull that natural selection would have intended.

  • DrugMonkey says:

    Varmus was NIH Director in the past and is now NCI Director. He's part of the problem!

  • rxnm says:

    And I'm sure when Kirschner was an assistant prof, he was desperate for attention and advice from 70 year-olds who were going to solve his problems for him.

  • Read the history of optogenetics. Did it require Deisseroth?

    I hadn't even heard of the field until now, but I found it interesting that according to Wikipedia, the idea was originally Crick's. I thought the Salk-era neurobiologist Crick was wasting his time in your opinion.

  • drugmonkey says:

    My problemo with Crick during the Salk era is encapsulated in The Astonishing Hypothesis. He came to cognitive neuroscience as essentially a first year grad student and everything was amazing. but because he was a Nobel Laureate......

  • drugmonkey says:

    The Astonishing Hypothesis was distinctly not "Astonishing" either. the underlying theme was a gigantic straw-man.

  • dsks says:

    "but I found it interesting that according to Wikipedia, the idea was originally Crick's"

    Read it again. Richard Fork at Bell Labs was blasting snail neurons with lasers and entertaining the idea of using this to map neural circuits 30 yrs beforehand.

  • dsks says:

    correction, 20 yrs not 30 yrs.

  • drugmonkey says:

    Shoulders of Giants....and other lesser creatures.

  • Dave says:

    ^off topic

  • Morgan Price says:

    As a taxpayer, I can see how having scientists spend less time on grantsmanship and more time doing science might be a good idea. (Has any research systems tried a lottery?) But how am I going to get more research bang for my buck by paying postdocs more? I don't get it.

  • odyssey says:

    But how am I going to get more research bang for my buck by paying postdocs more? I don't get it.

    Obviously you won't. The recommended "fixes" would work well for the large, well-funded labs (despite the part about the NIH looking at how much each big lab receives in funding). The rest of us would have to fight to the death in a massive, ongoing Research Funding Thunderdome. In many ways we already have been for a while. The proposed "fixes" would just codify it.

  • @drugmonkey
    Yes, the title was a bit unfortunate -- much like Dennet's "Consciousness Explained" it basically set itself up for mockery, as one could say that they weren't astonished and consciousness wasn't explained.

  • GAATTC says:

    The rumors were true... No more two strikes and you're out...

    http://nexus.od.nih.gov/all/2014/04/17/blog-on-nih-policy-notice-14-074/

  • […] After writing this I noticed that Drugmonkey said it before me, and more articulately […]

  • JMZ says:

    "But how am I going to get more research bang for my buck by paying postdocs more?"
    -Several reasons. If the postdoc becomes a more comfortable, stable position, postdocs, who are highly (if in limited scope) trained individuals, won't be in such a rush to flee it for "real" jobs (like consulting, patent law, etc). A fourth or fifth year postdoc is an experimental juggernaught, if they're not riddled with anxiety about how to feed their family. Even if they're sticking with academics, they'll do more science if they can take their time and not try to pad their CVs while simultaneously sending them to every job vacancy they see. Finally, just increasing the average tenure of a postdoc (by decreasing incentive to leave) will result in more highly trained, productive people who have developed a sense of how to do science and which projects to pursue.

    On a related point, by reducing the pressure to flee the postdoc, you're reducing the incentive to fudge or flat out forge results. I don't think this is a major issue yet, but if the current trends hold, you'll see more and more papers full of nonsense get thrown out there as postdocs just want to get their first author out and move on.

    For what my late comment is worth, I think increasing the postdoc salary to a decent wage (60k) and creating some stability for the position is the easiest and most efficacious solution to several of the problems in the US academic biomedical system.
    In addition to the points above, it would begin to ease the number of postdocs down (if finances remained constrained), which would help alleviate the competition for grants. By doing so, since giant postdoc farm labs would be most effected, this would also restore some parity to the US academic research programs. I'm at HMS, and I've seen how these labs operate, and its not always to science's benefit. Sometimes their resources are great at pulling off really cool science, however; often they just jump on a field and use their resources to outcompete and scoop small labs already working on similar projects, effectively wasting research money and time.
    I'd also suggest that NRSF and other fellowships be endowed with substantially more money for actual research expenses to be used at the postdoc's discretion. Something on the order of 20k that could fund a small project or initial experiment. Right now there's very little obligation on the PI to actually follow through on a fellowship winning idea. While this wouldn't lessen PI autonomy, it would allow the postdoc some ability to develop preliminary data for his fellowship and convince a PI its worth following up on.

  • CMDA says:

    Morgan Price commented on April 16th (above): As a taxpayer, I can see how having scientists spend less time on grantsmanship and more time doing science might be a good idea. (Has any research systems tried a lottery?)

    The lottery idea is interesting. Two investigators who think a lot about issues surrounding the current funding and science environment propose just such a lottery in the op-ed pages of the Wall Street Journal. The April 14th, 2014, article may be behind the WSJ's paywall; it can be found by doing a Google search for its title: "Taking the Powerball Approach to Funding Medical Research: Winning a government grant is already a crapshoot Making it official by running a lottery would be an improvement." By Ferric C. Fang And Arturo Casadevall.

    They may be just throwing the idea out there to stir discussion. I applaud them for doing this in such a public forum. Issues relating to how public funding for research is distributed and to how efficiently such funds are used concern everyone, not just the readership of science journals and blogs.

  • drugmonkey says:

    I think the "lottery" analogy/dismissal/complaint can be taken a mite too far. I think you will find most people having a cutoff...albeit not in the same place. Such as "anything up to 25%ile is indistinguishable". Which is a far cry from saying we should just have a lottery of the whole pool of applications. Also, there is a degree to which the various levels of review maintain the distribution of scientific topics, models, focus, etc. A pure lottery would be biased towards the most numerous applications, right? So it would be all cures for mouse cancer, all the way down, with a unfettered lottery.

  • CMDA says:

    I agree. I can think of other problems as well with a simple lottery. With some refinements, however, perhaps the idea would fly.

  • E-rock says:

    That's like saying to have a lottery for tenure track positions..... I agreed there should be more positions, grants awarded, etc. but a lottery system would be ruinous. Scholarlship, and science & science funding in particular, relies on maintaining high standards through a vetting process. You're free to have all the crazy ideas and say what you want, but it goes out into the world and our peers judge it, and someone funds it. As much as I decry old-chumy biases, a system of judgement and a (transparent) selection is necessary. It feels like lottery because we acknowledge the role of luck, timing, etc. I feel like some sort of blinded review of the science behind a proposal would be good, some accountability on the part of reviewers would be nice, perhaps a more active role of CSR for handling biases inherent in humans judging other humans. To the real problem though, graduate training needs to be reformed. Phd students need to be more than lab monkeys and advisors need to do better at preparing them for something other than casting gels, use their motivation to get good science done, but allow (and help) them prepare for jobs outside of the pyramid. Departments, chairs, and academic senates need to make the big changes to programs, and funding agencies need to incentivize them (more sticks, fewer carrots).

  • JMZ says:

    "To the real problem though, graduate training needs to be reformed. Phd students need to be more than lab monkeys and advisors need to do better at preparing them for something other than casting gels, use their motivation to get good science done, but allow (and help) them prepare for jobs outside of the pyramid."
    -Actually, that's a good corollary to what I said about postdocs getting more resources for actual science. If we want to help get the best people in the PI positions, then giving them early access to funds so as to prove themselves is one easy way the NIH could do this.
    As it stands now, someone that would make a great PI might get easily stymied because they chose a particularly micromanaging boss that didn't let them stray from the lab doctrine/expertise/etc. Picking a worthwhile PI with the right balance of knowledge and laissez faire attitude isn't really a skill you need to be a PI, but its definitely a skill you need to get through a PhD and postdoc with your sanity intact.
    So many people just write the NRSA grants to get their salaries paid, regardless of feasability, scope or impact. This leaves novelty and lab rep as the sole remaining criteria for judging these grants, not really a trait we want to encourage in the next gen of PIs.

  • E-rock says:

    A zillion years ago when I was a student, POs came to informally visit our T32 program. We sat at a conference room table and they asked us to go around explaining our career goals, maybe 2/15 of us had a concise, specific answer. We were so focused on science in the moment that we hadn't considered the future (bad mentoring, frankly). That T32 didn't go on to be renewed......but did that stop the grad program from increasing enrollment?

Leave a Reply