"As we contemplated possible changes, we were guided by several fundamental principles. First, while improving the system, do no harm. That is, ensure that any changes to the peer review system bring significant value and outweigh costs," said Zerhouni. "Second, continue to maximize the freedom of scientists to pursue high-risk, high-impact research. Moreover, we want to cultivate a sense that we continuously re-evaluate the peer review system to ensure that it is the best that it can be."
The Enhancing Peer Review at NIH splash page currently has a nifty timeline graphic.
The 6/6/08 report to the Advisory Council of the Director is here in pdf form.
Okay, none of this will be startling news to Readers who have been following along for the past year or so. Nevertheless, the action plan has been Issued!
The Implementation Plan Report consists of four main priorities and highlights include:
* Priority 1 -- Engage the Best Reviewers: Increase flexibility of service, formally acknowledge reviewer efforts, further compensate time and effort, and enhance and standardize training
* Priority 2 -- Improve Quality and Transparency of Reviews: Shorten and redesign applications to highlight impact and to allow alignment of the application, review and summary statement with five explicit review criteria, and modify the rating system
* Priority 3 -- Ensure Balanced and Fair Reviews Across Scientific Fields and Career Stages
o Support a minimum number of early stage investigators and investigators new to NIH, and emphasize retrospective accomplishments of experienced investigators
o Encourage and expand the Transformative Research Pathway
o Create a new investigator-initiated Transformative R01 Award program funded within the NIH Roadmap with an intended commitment of a minimum of $250 million over five years
o Continue the commitment of -- and possibly expand the use of -- the Pioneer, EUREKA, and New Innovator Awards. NIH will invest at least $750 million in these three programs over the next 5 years.
o Reduce the burden of multiple rounds of resubmission for the same application, especially for highly meritorious applications
* Priority 4 -- Develop a Permanent Process for Continuous Review of Peer Review
Sadly, I'm mired in grant reviewing stuff and preparing for a meeting so I'll have to do some diagnosis later. "Early stage investigators" is intriguing but wtf is up with "retrospective accomplishments of experience investigators"??? We're going to bloody enshrine the "gee this research plan is for garbage but dang, Prof Bluehair and Dr. Greybeard have done such great stuff that we're just going to fund them anyway"? Is THAT what this is about?
In the meantime you may visit
writedit for explication of some of the steps.
Here, the goal is to reduce resubmissions both from applicants with a high likelihood of funding based on their A0 review (hallelujah!) and from applicants with low or no likelihood of funding based on their A0 review (thank you straight talk express). This component also seeks to "rebalance success rates among A0, A1, and A2 submissions to increase system efficiency" and to include statistics on cumulative success rates as a function of score or percentile in the summary statement.
I swear there are times when I think these people are actually reading my blog!
On the issue of percent effort, an alternative approach to requiring a minimum percent effort is suggested: applicants would be required to complete a subfield in the Environment section of the application in which they indicate if they have NIH RPG (research project grant) support in excess of $1M at time of anticipated funding. If so, they must justify why additional resources are needed.
I might actually arrive at the figure of $1M in direct costs myself, if pressed to give a number.
You may also wish to visit with Jake who, in his usual naivety, notes:
I think that the bit about shortening grants and paying people to be regular attendants at study section (the meeting of peer reviewers where the assign scores to each of the grants) will certainly improve the caliber of reviewers. It would also be a great idea if you could do these reviews from afar rather than heading down to Washington. (Do they do them in places besides Washington? I don't actually know.)
Sigh. Got any evidence that this will "improve the caliber of reviewers" there buddy? Evidence there is something wrong at present and how this will address any problems? Or are you just parroting the usual lines unthinkingly?
And seriously. You blog, man. Do you really think that substituting online discussion formats for face to face meetings is a good move? Really?
On the other hand, I do understand the resistance to labeling all grants as "new." It really depends on why you think most grants aren't funded the first time. If you think that is because they weren't very good to begin with, you should be in favor of labeling all grants as new so as to more rapidly identify the best grants and get rid of the rest. On the other hand, if you think that a lot of grants aren't funded the first go because they A) weren't written well but have fundamentally good ideas or B) because funding is so limited, you need that "resubmitted" label to help your chances for the second or third go-around.
The thought is incomplete but I think he was suggesting if you think B is true, you should be against labeling all grants as "new". This overlooks the fundamental problem at hand in that B is the problem itself, not the solution to a problem. Similarly, the fact that we obsess too much over whether grants expressing fundamentally good ideas are presented in a well-written packages is the problem this is trying to address. My hope would be that this modification would be used as a tool to try to force reviewers (and Program!) to a fish-or-cut-bait strategy on proposals. Is it fundamentally of interest and sound? Will the review/revise process change anything fundamental or is it just wasting everyone's time? Those are the questions.