Applicants are starting to receive their Summary Statements from grant reviews that took place in September/October NIH study sections. The Summary Statement is the written record of the review of your grant.
It contains the written reviews of the assigned reviewers (usually two, three, or four of them), as well as the Resume and Summary of Discussion, which is a summary written by the Scientific Review Officer of the discussion of your grant by the study section. This latter section is only there if your grant was discussed--i.e., not triaged without discussion.
A colleague of mine from another institution was kind enough to share with me the Summary Statement she just received, as she found it confusing and wanted to get my take on it.
I opened the Summary Statement PDF she forwarded to me and started, as I always do, with the Resume and Summary of Discussion:
...very good proposal...broad significance...PI is very creative and productive...abundant preliminary data...considerable enthusiasm...despite numerous strengths, concerns were raised that limit enthusiasm
The concerns that were raised were solely technical issues, and the reviewers all considered those technical issues to be addressable, and they gave good suggestions for how to address them. So that sounds like the grant should receive a good score, near, but probably not within, the fundable range, right?
HAHAHAHAH! The grant received a priority score--the average of the numerical scores from 100-500 assigned by each member of the review panel to the grant--of ~200, which for this particular study section corresponded to a percentile of ~40. WTF!? Based on the Resume and Summary of Discussion, this grant should have ended up in the ~20 percentile. So what happened?
Well, as I read the written reviews of the assigned reviewers, I noticed something very interesting about one of the Investigator sections--where the reviewer is supposed to provide her opinion of the qualifications of the PI to carry out the proposed research:
Dr. So-and-So is an outstanding young investigator who has been very successful in establishing herself as an independent investigator. An important indication of her success is her high level of funding.
Ruh, roh!! High level of funding was the death knell for this application. It is nearly certain that this phrase was intentionally inserted in there by this particular assigned reviewer specifically so that all of the rest of the members of the review panel--none of whom are likely to have looked at any of the application other than the Specific Aims--would perceive that this PI does not need to have this grant funded in order for her lab to have enough funding to exist. And this perception led the entire panel to substantially downgrade the scores.
Some other colleagues have disputed this interpretation, claiming that the reviewer was just being very laudatory and complimentary to the PI in pointing out her high level of funding. I call bullshit on that.
First, this reviewer did not point out any of the other aspects of this PI's success--including, most importantly, the substantial number of published manuscripts from her lab in high-impact-factor journals. This reviewer just happened to choose to highlight out of the various outstanding accomplishments of this PI solely the fact of her high level of funding.
Second, in assigning a score to a grant, it is de jure impermissible to take into account per se the level of funding of the PI. If the level of funding arose during discussion, the SRO and/or chair of the study section would--unless they were asleep at the wheel--immediately and forcefully stop the discussion and remind the panel that this is not a permissibile consideration. The reviewer knew this, and snuck the information in as a supposed signifier of the strength of the PI's success, knowing full well that it would have exactly the opposite effect.
Having said all of this, it is important to recognize that this kind of reviewer behavior does not represent some kind of terrible personal ethical lapse on the part of this reviewer or the study section members who responded to this intentional signal by downgrading the scores. It is understandable behavior that is ethical and rational in the current funding environment if one is convinced that keeping as many labs as possible funded sufficiently to continue to exist in some form is a more important good than single-mindedly funding the best possible research.