Archive for the 'NIH Careerism' category

These ILAF types just can't help sounding selfishly elitist, can they?

Good Gravy.

One David Korn of the Massachusetts General Hospital and Harvard Medical School has written a letter to Nature defending the indirect cost (IDC; "overhead") rates associated with NIH grants. It was submitted in response to a prior piece in Nature on IDC which was, to my eye, actually fairly good and tended to support the notion that IDC rates are not exorbitant.

But overall, the data support administrators’ assertions that their actual recovery of indirect costs often falls well below their negotiated rates. Overall, the average negotiated rate is 53%, and the average reimbursed rate is 34%.

The original article also pointed out why the larger private Universities have been heard from loudly, while the frequent punching-bag smaller research institutes with larger IDC rates are silent.

Although non-profit institutes command high rates, together they got just $611 million of the NIH’s money for indirect costs. The higher-learning institutes for which Nature obtained data received $3.9 billion, with more than $1 billion of that going to just nine institutions, including Johns Hopkins University in Baltimore, Maryland, and Stanford (see ‘Top 10 earners’).

Clearly Dr. Korn felt that this piece needed correction:

Aspects of your report on US federal funding of direct research costs and the indirect costs of facilities and administration are misleading (Nature 515, 326–329; 2014).

Contrary to your claim, no one is benefiting from federal largesse. Rather, the US government is partially reimbursing research universities for audit-verified indirect costs that they have already incurred.

Ok, ok. Fair enough. At the very least it is fine to underline this point if it doesn't come across in the original Nature article to every reader.

The biomedical sciences depend on powerful technologies that require special housing, considerable energy consumption, and maintenance. Administration is being bloated by federal regulations, many of which dictate how scientists conduct and disseminate their research. It is therefore all the more remarkable that the share of extramural research spending on indirect costs by the US National Institutes of Health (NIH) has been stable at around 30% for several decades.

Pretty good point.

But then Korn goes on to step right in a pile.

Negotiated and actual recovery rates for indirect costs vary across the academic community because federal research funding is merit-based, not a welfare programme.

You will recognize this theme from a prior complaint from Boston-area institutions.

“There’s a battle between merit and egalitarianism,” said Dr. David Page, director of the Whitehead Institute, a prestigious research institution in Cambridge affiliated with MIT.

Tone deaf, guys. Totally tone deaf. Absolutely counter-productive to the effort to get a majority of Congress Critters on board with support for the NIH mission. Hint: Your various Massachusetts Critters get to vote once, just like the Critters from North and South Dakota, Alabama and everywhere else that doesn't have a huge NIH-funded research enterprise.

And why Korn chooses to use a comment about IDC rates to advance this agenda is baffling. The takeaway message is that he thinks that higher IDC rates are awarded because His Awesome University deserves it due to the merit of their research. This totally undercuts the point he is trying to make, which is presumably "institutions may be private or public, urban or rural, with different structures, sizes, missions and financial anatomies.".

I just don't understand people who are this clueless and selfish when it comes to basic politics.

23 responses so far

NIGMS will now consider PIs' "substantial unrestricted research support"

According to the policy on this webpage, the NIGMS will now restrict award of its grants when the applicant PI has substantial other research support. It is effective as of new grants submitted on or after 2 Jan, 2015.

The clear statement of purpose:

Investigators with substantial, long-term, unrestricted research support may generally hold no more than one NIGMS research grant.

The detail:

For the purposes of these guidelines, investigators with substantial, long-term, unrestricted support (“unrestricted investigators”) would have at least $400,000 in unrestricted support (direct costs excluding the principal investigator’s salary and direct support of widely shared institutional resources, such as NMR facilities) extending at least 2 years from the time of funding the NIGMS grant. As in all cases, if NIGMS funding of a grant to an investigator with substantial, long-term, unrestricted support would result in total direct costs from all sources exceeding $750,000, National Advisory General Medical Sciences Council approval would be required

This $400,000 limit, extending for two years would appear to mean $200,000 per year in direct costs? So basically the equivalent of a single additional R01-worth of direct cost funding?

I guess they are serious about the notion that two grants is fine but three-R01-level funding means you are a greedy commons-spoiling so-and-so.

51 responses so far

More on NIGMS's call for "shared responsibility"

Jan 07 2015 Published by under Fixing the NIH, NIH, NIH Careerism, NIH funding

The post from NIGMS Director Lorsch on "shared responsibility" (my blog post) has been racking up the comments, which you should go read.

As a spoiler, it is mostly a lot of the usual, i.e., Do it to Julia!

But two of the comments are fantastic. This one from anonymous really nails it down to the floor.

More efficient? My NOA for my R01 came in a few weeks ago for this year, and as usual, it has been cut. I will get ~$181,000 this year. Let’s break down the costs of running a typical (my) lab to illustrate that which is not being considered. I have a fairly normal sized animal colony for my field, because in immunology, nothing gets published well without knockouts and such. That’s $75,000 a year in cage per diem costs. Let’s cover 20% of my salary (with fringe, at 28.5%), one student, and one postdoc (2.20 FTE total). Total salary costs are then $119,800. See, I haven’t done a single experiment and my R01 is gone. How MORE efficient could I possibly be? Even if we cut the animals in half, I have only about $20,000 for the entire year for my reagents. Oh no, you need a single ELISA kit? That’s $800. That doesn’t include plates? Hell, that’s another $300. You need magnetic beads to sort cells, that’s $800 for ONE vial of beads. Wait, that doesn’t include the separation tubes? Another $700 for a pack. You need FACS sort time? That’s $100 an hour. Oh no, it takes 4 hours to sort cells for a single experiment? Another $400. It’s easy to spend $1500 on a single experiment given the extreme costs of technology and reagents, especially when using mice. Then, after 4 years of work, you submit your study (packed into a single manuscript) for publication and the reviewers complain that you didn’t ALSO use these 4 other knockout mice, and that the study just isn’t complete enough for their beloved journal. And you (the NIH) want me to be MORE efficient? I can’t do much of anything as it is.

Anyone running an academic research laboratory should laugh (or vomit) at the mere suggestion that most are not already stretching every penny to its breaking point and beyond.

This is what is so phenomenally out of touch with Lorsch's concentration on the number of grants a PI holds. Most of us play in the full-modular space. Even for people with multiple grants that have one that managed to get funded with a substantial upgrade from full-mod, they are going to have other ones at the modular limit. And even the above-modular grants often get cut extra compared with the reductions that are put on the modular-limit awards.

The full-modular has not been adjusted with inflation. And the purchasing power is substantially eroded compared with a mere 15 years ago when they started the new budgeting approach.


[this graph depicts the erosion of purchasing power of the $250K/yr full-modular award in red and the amount necessary to maintain purchasing power in black. Inflation adjustment used was the BRDPI one]

Commenter Pinko Punko also has a great observation for Director Lorsch.

The greatest and most massive inefficiency in the system is the high probability of a funding gap for NIGMS (and all other Institute) PIs. Given that gaps in funding almost always necessitate laying off staff, and prevent long-term retention of expertise, the great inefficiency here is that expertise cannot possibly be “on demand”. I know that you are also aware that given inflation, the NIH budget never actually doubled. There has likely been a PI bubble, but it is massively deflating with a huge cost.

The lowest quantum for funding units in labs is 1. Paylines are so low, it seems the only way to attempt to prevent a gap in funding is to have an overlap at some point, because going to zero is a massive hit when research needs to grind to a halt. It is difficult to imagine that there is a large number of excessively funded labs.

While I try to put a positive spin on the Datahound analysis showing the probability of a PI becoming re-funded after losing her last NIH award, the fact is that 60% of PIs do not return to funding. A prior DataHound post showed that something between 14-30% of PIs are approximately continuously-funded (extrapolating generously here from only 8 years of data). Within these two charts there is a HUGE story of the inefficiency of trying to maintain that funding for the people who will, in the career-long run, fall into that "continuously funded" category.

This brings me to the Discussion point of the day. Lorsch's blog post is obsessed with efficiency. which he asserts comes with modestly sized research operations, indexed approximately by the number of grant awards. Three R01s being his stated threshold for excessive grants even though he cites data showing that $700K per year in direct costs is the most productive* amount of funding- i.e., three grants at a minimum.

I have a tale for you, Dear Reader. The greatly abridged version, anyway.

Once upon a time the Program Staff of ICx decided that they were interested in studies on Topic Y and so they funded some grants. Two were R01s obtained without revision. They sailed on for their full duration of funding. To my eye, there was not one single paper that resulted that was specific to the goals of Topic Y and damn little published at all. Interestingly there were other projects also funded on Topic Y. One of them required a total of 5 grant applications and was awarded a starter grant, followed by R01 level funding. This latter project actually produced papers directly relevant to Topic Y.

Which was efficient, Director Lorsch?
How could this process have been made more efficient?

Could we predict PI #3 was the one that was going to come through with the goods? Maybe we should have loaded her up with the cash and screw the other two? Could we really argue that funding all three on a shoestring was more efficient? What if the reason that the first two failed is that they just didn't have enough cash at the start to make a good effort on what was, obviously, a far from easy problem to attack.

Would it be efficient to take this scenario and give PI #3 a bunch of "people-not-projects" largesse at this point in time because she's proved able to move the scientific ball on this? Do we look at the overall picture and say "in for a penny, in for a pound"? Or do we fail to learn a damn thing and let the productive PI keep fighting against the funding cycles, the triage line and what not to keep the program going under our current approaches?

It may sound like I am leaning in one direction on this but really, I'm not. I don't know what the answer is. The distribution of success/failure across these three PIs could have been entirely different. As it happens, all three are pretty dang decent scientists. The degree to which they looked like they could kick butt on Topic Y at the point of funding their respective projects definitely didn't support the supremacy of PI#3 in the end analysis. But noobs can fail too. Sometimes spectacularly. Sometimes, as may have been the case in this little tale, people can fail because they simply haven't grown their lab operations large enough, fast enough to sustain a real program, particularly when one of the projects is difficult.

I assume, as usual, that this narrow little anecdote is worth relating because these are typical scenarios. Maybe not hugely common but not all that rare either. Common enough that a Director of an IC should be well aware.

When you have an unhealthy interest in the grant game, as do I, you notice this stuff. You can see it play out in RePORTER and PubMed. You can see it play out as you try to review competing-continuation proposals on study section. You see it play out in your sub-fields of interest and with your closer colleagues.

It makes you shake your head in dismay when someone makes assertions that they know how to make the NIH-funded research enterprise more efficient.

UPDATE: I realized that I should really should say that the third project required at least five applications since I'm going by the amended status of the *funded* awards. It is unknown if there were unfunded apps submitted. It is also unknown if either of the first two PIs tried to renew the awards and couldn't get a fundable score. I think I should also note that the third project was awarded funding in a context that triggers on at least three of the major "this is the real problem" complaints being bandied in Lorsch's comments section. The project that produced nothing at all, relevant or not, was awarded in a context that I think would align with these complainants "good" prescription. FWIW.

__
*there are huge problems even with this assessment of productivity but we'll let that slide for now.

66 responses so far

Your Grant In Review: Thought of the Day

Jan 07 2015 Published by under Grant Review, NIH Careerism

I've said it repeatedly on this blog and it is true, true, true people.

In NIH grant review, the worm turns very rapidly.

The pool of individual PIs who are appropriate to apply for, and review, NIH grants in a narrow subfield is a lot smaller than most people seem to think. Or maybe this is just my field.

My guiding belief is that the reviewer of a given grant is going to have one of her own grants reviewed by the PI of the proposal she just reviewed  in very short order. Or maybe it takes a half a decade, even more. But it will happen.

And PIs do not take kindly to jackholish reviews of their proposals.

As we all know, in this day and age it takes very little in the way of reviewer behavior to totally torpedo a grant's chances. You don't even have to be obvious about it*.

This is why I try as hard as I possibly can to ground my grant reviewing in concrete reasons for criticism.

Because I want the reviewers of my proposals to do the same. And it is the right thing to do.

We have a system of grant review that is at all times precariously balanced on a knife's edge that could slide off into Mutually Assured Destruction cycles of retaliation** at any time. And I am sure it happens in some study sections and amongst some reviewers.

Mutual Professional Respect is better. It is supported one review at a time by engaging our firmest professionalism to override the biases that we cannot help but have.

 

illustration from here.

__

*This is very likely the second hardest decision I have to make about registering a Conflict of Interest in reviewing grants. I have reviewed a lot of grants of PIs who have been on the study section panels reviewing my grants. I am pretty confident this is the case for just about anyone who has served a full term appointed on a study section and probably anyone who has reviewed with full loads in over about 3 panels as ad hoc. This in and of itself cannot be a reason to recuse yourself or they would never get anything reviewed. And as my Readers know, I am very firm in my belief that it is a fool's errand to try to game out which reviewers were on your proposals and which ones were...critical.

**And, gods above, pre-emptive counter-striking.

 

2 responses so far

The new normal

Jan 05 2015 Published by under Careerism, NIH, NIH Careerism

An interesting comment from Anonymous :

A friend of mine is getting his PhD this year. His mentor was awarded tenure last year. She had an RO1 and an R21 and enough pubs, reputation, etc., so that she was a "slam dunk," according to some of her colleagues in the field, at least. When my friend joined her lab, she had 7 students -- a mix of masters and doctoral students. When he graduates this year, there will be only 1 doctoral student left in the lab. His mentor hasn't succeeded in getting more grants, so I guess she hasn't hired anyone else because of that?

What I'm wondering is this: given the current climate, is this normal? Or is she really in trouble? Has she done it wrong? I always assumed that by the time you got tenure, your lab would be humming along.

 

I think this is pretty normal these days:

We are in an era of boom and bust instability when it comes to NIH funding. It is the very rare flower indeed, in my estimation, that will be completely free of the cycle in the coming decade or two.

As always, my view is quite possibly colored by my experiences. But I have seen the boom and bust cycle play out across a large number of labs. Some of my close acquaintance. Some labs that I know only through the grant review process. Some labs that happen to make it to the scuttlebutt news channel for some reason or other.

It usually plays out like this. "Yeah, Dr. So-and-so is really well funded.....what? What do you mean they are on the ropes? [checks RePORTER]..how in the hell did THAT happen". ....Two years later "Oh phew, glad to see So-and-so got another grant. ....what? TWO grants? and an R21? how in the hell did THAT happen?"...

Repeat.

Normal, but the PI is still in trouble. How could she not be? Has she done it wrong? Probably not. Most likely she's just experiencing the variance of grant fortune as it currently exists.

This part is painful though: " I always assumed that by the time you got tenure, your lab would be humming along."

Yeah, so did we. Because when people of my approximate scientific generation were coming up through postdoc we saw the generation of Assistant Professors just above us struggling. But then as we were finishing postdocs and starting our own Assistant Professor stints, we saw the next-older generation transition to a cruise mode. A time where they got their renewals without too much hassle*. They got their second or even third grants and maybe a few got an R37 extension. This made the struggles we went through as newbie applicants to the NIH a bit easier to stomach. Hazing ritual. Sure, we can stand this, and then we'll REALLY get stuff cranking in the lab later.

Instead the budget went stagnant just as we were reaching that stage of our careers. And then the powers that be went and invented the ESI boost to give affirmative action to those juuuuuust behind us.

So....we ride the roller coaster. And as things keep going in the wrong direction with NIH funding, more and more of us from all scientific cohorts/generations will experience the thrill.

__

*Yes, I realize it is all relative. I certainly had it easier than the kids these days. And the next-older generation did plenty of complaining about how hard they had it compared to the really established folks.

70 responses so far

NIGMS blog post on "shared responsibility"

Jan 05 2015 Published by under NIH, NIH Careerism, NIH funding

This is a fascinating read.

Jon Lorsch, current NIGMS Director, spreads around so much total nonsense that I just can't even deal.

And journals, professional societies and private funding organizations should examine the roles they can play in helping to rewire the unproductive incentive systems that encourage researchers to focus on getting more funding than they actually need.

Riiiight. We PIs out here in extramural land are focused on getting more grant money than we feel that we need. Because what? We enjoy grant writing? Is this guy nuts? When I feel like I have enough grant support to keep what is really a very modest operation afloat I quit writing grants! The problem is that Director Lorsch is really, really out of touch with what a PI in today's climate actually needs. "Unproductive incentive systems"? Dude, when NIGMS stops giving grants to anyone who publishes in Science, Nature or Cell, and starts beancounting Supplementary Data for reduced publication output, and punishes PIs for failing to publish data that is "scooped" or "not hot enough" etc, then maybe I will take you seriously. Jeepers. LOOK IN THE MIRROR, NIH!!!!!!!

But to achieve this increase, we must all be willing to share the responsibility and focus on efficiency as much as we have always focused on efficacy. In the current zero-sum funding environment, the tradeoffs are stark: If one investigator gets a third R01, it means that another productive scientist loses his only grant or a promising new investigator can’t get her lab off the ground. Which outcome should we choose?

better to have everyone funded at $50K and sitting around doing nothing, right?

Although certain kinds of research projects—particularly those with an applied outcome, such as clinical trials—can require large teams, a 2010 analysis by NIGMS and a number of subsequent studies of other funding systems (Fortin and Currie, 2013; Gallo et al., 2014) have shown that, on average, large budgets do not give us the best returns on our investments in basic science.

The "2010 analysis" has been discussed here, I recall. It's flawed. It fails to recognize the cost of a Glamour Pub- love or hate, we have to admit that it takes a rich lab to play in that arena. One pub to the accountants has like 6-10 pubs worth of time/effort and probably data (buried in the Supplemental Materials). It fails to recognize there are going to be some scientific advances that simply cannot be accomplished for less. It fails to recognize the "efficiencies" and lack thereof associated with continued funding versus the scary roller coaster of a funding gap.

and Lorsch does a little neat ju-jitsu with this post. Berg's analysis concluded that $700K was the peak of productivity. That is three concurrent full modular R01s. Even with a traditional budget award the PI has to have two of them awarded at $350K per year (and we know it really means more than that because of cuts) to hit this level. So the finger pointing at the investigator who "gets a third R01" doesn't even square with his own citation on "efficiency", now does it?

Furthermore, the larger a lab gets, the more time the principal investigator must devote to writing grants and performing administrative tasks, further reducing the time available for actually doing science.

Good GRAVY man! Do you have any idea what the hell time it is on PI street? A one-grant lab PI is constantly on the edge of disaster and the PI is always writing furiously to increase that funding level to where she can finally breathe for six months. Protocols and registrations most often are one per lab, so that "administrative tasks" claim is also nonsense. It is the smaller PI who spends more effort per-grant on keeping the approvals in place.

These and other lines of evidence indicate that funding smaller, more efficient research groups will increase the net impact of fundamental biomedical research: valuable scientific output per taxpayer dollar invested.

[citation needed]

It may sound truthy but it is not at all obvious that this is the case. "More efficient" is tautological here but the smaller=efficient is not proven. At all. Especially when you are talking about the longer term- 30 years of a career. There is also the strong whiff of Magic Unicorn Leprechaun money about this comment.

My main motivation for writing this post is to ask the biomedical research community to think carefully about these issues.

You know what I would really like to ask? For the NIH to actually think carefully about these issues. Starting with Director Lorsch.

But reshaping the system will require everyone involved to share the responsibility.

Somehow I doubt he means this. Is there any evidence that NIGMS actually denies the super awesome PIs their extra R01s? Is there any evidence they do anything more than handwring about HHMI investigators with NIGMS grants? Is there any evidence they mean to face down the powerful first and make them equal to the rest of the drones before they take it out of the hide of the less-powerful? HA!

h/t: PhysioProf

23 responses so far

McKnight: "Wait whut? There are data? Really? Maybe I'd better cool it..."

Dec 05 2014 Published by under Fixing the NIH, NIH, NIH Careerism

The sidebar to McKight's column at ASBMB Today this month is hilarious.

Author's Note

I’ve decided it’s prudent to take a break from the debate about the quality of reviewers on National Institutes of Health study sections. The American Society for Biochemistry and Molecular Biology governing council met in mid-November with Richard Nakamura, director of the NIH’s Center for Scientific Review. The discussion was enlightening, and the data presented will inform my future columns on this topic.

HAHAHHAA. Clearly it is news to McKnight that his opinions might actually be on topics for which there are data in support or contradiction? And now he has to sit down and go through actual facts to try to come up with a better argument that study sections today are populated with riff-raff who are incompetent to review science.

Never fear though, he still leaves us with some fodder for additional snickering at his....simple-minded thinking. He would like his readers to answer some poll questions...

The first question is:
Should the quality of the proposed research and researcher be the most important criteria dictating whether an NIH-sponsored grant is funded?

The response item is Yes/No so of course some 99% of the responses are going to be Yes. Right? I mean jeepers what a stupid question. Particularly without any sense of what he imagines might be a possible alternative to these two considerations as "the most important criteria". Even more hilariously since he has totally conflated the two things that are actual current items of debate (i.e., project versus person) and tie directly into his two prior columns!

The next question:
The review process used to evaluate NIH grant applications is:

has three possible answers:
essentially perfect with no room for improvement
slightly sub-optimal but impossible to improve
suboptimal with significant room for improvement

Again, simple-minded. Nobody thinks the system is perfect, this is a straw-man argument. I predict that once again, he's going to get most people responding on one option, the "suboptimal, room for improvement" one. This is, again, trivial within the discussion space. The hard questions, as you my Readers know full well, relate to the areas of suboptimality and the proposed space in which improvements need to be made.

What is he about with this? Did Nakamura really tell him that the official CSR position is that everything is hunky-dory? That seems very unlikely given the number of initiatives, pilot studies, etc that they (CSR) have been working through ever since I started paying attention about 7-8 years ago.

Ah well, maybe this is the glimmer of recognition on the part of McKnight that he went off half-cocked without the slightest consideration that perhaps there are actual facts here to be understood first?

17 responses so far

More in "NIH responds to a non-problem by creating a problem"

Dec 05 2014 Published by under Grant Review, Grantsmanship, NIH, NIH Careerism

I can't even imagine what they are thinking.

This Notice informs the applicant community of a modification for how NIH would like applicants to mark changes in their Resubmission applications. NIH has removed the requirement to identify 'substantial scientific changes' in the text of a Resubmission application by 'bracketing, indenting, or change of typography'.

Effective immediately, it is sufficient to outline the changes made to the Resubmission application in the Introduction attachment. The Introduction must include a summary of substantial additions, deletions, and changes to the application. It must also include a response to weaknesses raised in the Summary Statement. The page limit for the Introduction may not exceed one page unless indicated otherwise in the Table of Page Limits.

First of all "would like" and "removed the requirement" do not align with each other. If the NIH "would like" that means this is not just a "we don't care whether you do it or not". So why not make it a mandate?

Next up...WHY?

Finally: How in all that is holy do they really expect the applicant to ("must") summarize "substantial additions, deletions, and changes" and to "include a response to weaknesses" in just one page?

I am starting to suspect Rockey is planning on burning the OER down to the ground before leaving for greener pastures.

18 responses so far

Wait...the new Biosketch is supposed to be an antiGlamour measure? HAHAHHAHHA!!!!!

A tweet from @babs_mph sent me back to an older thread where Rockey introduced the new Biosketch concept. One "Senior investigator" commented:

For those who wonder where this idea came from, please see the commentary by Deputy Director Tabak and Director Collins (Nature 505, 612–613, January 2014) on the issue of the reproducibility of results. One part of the commentary suggests that scientists may be tempted to overstate conclusions in order to get papers published in high profile journals. The commentary adds “NIH is contemplating modifying the format of its ‘biographical sketch’ form, which grant applicants are required to complete, to emphasize the significance of advances resulting from work in which the applicant participated, and to delineate the part played by the applicant. Other organizations such as the Howard Hughes Medical Institute have used this format and found it more revealing of actual contributions to science than the traditional list of unannotated publications.”

Here's Collins and Tabak, 2014 in freely available PMC format. The lead in to the above referenced passage is:

Perhaps the most vexed issue is the academic incentive system. It currently overemphasizes publishing in high-profile journals. No doubt worsened by current budgetary woes, this encourages rapid submission of research findings to the detriment of careful replication. To address this, the NIH is contemplating...

Hmmm. So by changing this, the ability on grant applications to say something like:

"Yeah, we got totally scooped out of a Nature paper because we didn't rush some data out before it was ready but look, our much better paper that came out in our society journal 18 mo later was really the seminal discovery, we swear. So even though the entire world gives primary credit to our scoopers, you should give us this grant now."

is supposed to totally alter the dynamics of the "vexed issue" of the academic incentive system.

Right guys. Right.

13 responses so far

The new NIH Biosketch is here

Dec 02 2014 Published by under Grant Review, Grantsmanship, NIH, NIH Careerism, NIH funding

The NIH has notified us (NOT-OD-15-024) that as of Jan 25, 2015 all grant applications will have to use the new Biosketch format (sample Word docx).
[ UPDATE 12/05/14: The deadline has been delayed to apply to applications submitted after May 25, 2015 ]

The key change is Section C: Contribution to Science, which replaces the previous list of 15 publications.

C. Contribution to Science
Briefly describe up to five of your most significant contributions to science. For each contribution, indicate the historical background that frames the scientific problem; the central finding(s); the influence of the finding(s) on the progress of science or the application of those finding(s) to health or technology; and your specific role in the described work. For each of these contributions, reference up to four peer-reviewed publications or other non-publication research products (can include audio or video products; patents; data and research materials; databases; educational aids or curricula; instruments or equipment; models; protocols; and software or netware) that are relevant to the described contribution. The description of each contribution should be no longer than one half page including figures and citations. Also provide a URL to a full list of your published work as found in a publicly available digital database such as SciENcv or My Bibliography, which are maintained by the US National Library of Medicine.

The only clear win that I see here is for people who contribute to science in a way that is not captured in the publication record. This is captured by the above suggestions of non-publication products which previously had no place other than the Personal Statement. I see this as a good move for those who fall into this category.

For the regular old run-of-the-mill Biosketches, I am not certain this addresses any of the limitations of the prior system. And it clearly hurts in a key way.

One danger I see lying ahead is that the now-necessary bragging about significant contributions may trigger 1) arguments over the validity of the claim and 2) ill will about the almost inevitable overshadowing of the other people who also made related contributions. The example biosketch leads with a claim to having "changed the standards of care for addicted older adults". This is precisely the sort of claim that is going to be argumentative. There is no way that a broad sweeping change of clinical care rests on the work of one person. No way, no how.

If the Biosketch says "we're one of twenty groups who contributed...", well, this is going to look like you are a replaceable cog. Clearly you can't risk doing that. So you have risks ahead of you in trying to decide what to claim.

The bottom line here is that you are telling reviewers what they are supposed to think about your pubs, whereas previously they simply made their own assumptions. It has upside for the reviewer who is 1) positively disposed toward the application and 2) less familiar with your field but man......it really sets up a fight.

Another thing I notice is the swing of the pendulum. Some time ago, publications were limited to 15 which placed a high premium on customizing the Biosketch to the specific application at hand. This swings back in the opposite direction because it asks for Contribution to Science not Contribution to the Relevant Subfield. The above mentioned need to brag about unique awesomeness also shifts the emphasis to the persons entire body of work rather than that work that is most specific to the project at hand. On this factor, I am of less certain opinion about the influence on review.

Things that I will be curious to see develop.

GlamourMag- It will be interesting to see how many people say, in essence, that such and such was published in a high JIF journal so therefore it is important.

Citations and Alt-metrics- Will people feel it necessary to defend the claims to a critical contribution by pointing out how many citations their papers have received? I think this likely. Particularly since the "non-publication research products" have no conventional measures of impact, people will almost have to talk about downloads of their software, Internet traffic hits to their databases, etc. So why not do this for publications as well, eh?

Figures- all I can say is "huh"?

Sally Rockey reports on the pilot study they conducted with this new Biosketch format.

While reviewers and investigators had differing reactions to the biosketch, a majority of both groups agreed that the new biosketch was an improvement over the old version. In addition, both groups felt that the new format helped in the review process. Both applicants and reviewers expressed concerns, however, about the suitability of the new format for new investigators, but interestingly, investigators who were 40 years and older were more negative than those below age 40.

So us old folks are more concerned about the effects on the young than are the actual young. This is interesting to me since I'm one who feels some concern about this move being bad for less experienced applicants.

I'll note the first few comments posted to Rockey's blog are not enthusiastic about the pilot data.

71 responses so far

Older posts »