I was very upset and frustrated by the current review system ...
My last R21 grant got a score of 182. After revision, the proposal got "unscored" ... I didn't think at the scientific review officer did a good job.
You spent a lot of time in preparing a proposal but who cares....
We really need to revise the review system...
Many applicants believe that their revised application somehow deserves an even better score than the previous version of the application and that if this does not happen this is evidence of a broken review system. That thinking is totally incorrect.
The NIH considers grants for funding on a round-by-round basis. Three times per year they will allocate a portion of their yearly budget for funding new proposals. To decide which ones to fund each round the ICs need to know the relative merit of the proposals that have been submitted for that round of funding.
They do not need to know where the proposal ranks among proposals that they funded during the 80s. They do not need to know where the proposal ranks among proposals that will be considered next round.
They do not need to know how meritorious the proposal is in an abstract scale of grant quality.
They need to know is whether this proposal better or worse than the other ones in the pool that has been submitted for the current round. Where does it rank? Which is why the percentile is so much more important than the priority score when it comes to Program considerations.
Peer reviewers who serve on NIH/CSR study sections are asked to perform the relative ranking of proposals. In fact more than just "asked" to do so. There are a number of boilerplate instructions on the topic. As the BM observed at MWE&G:
the review rules are made quite clear on this. Revised apps are NOT to be benchmarked against prior scores. All apps are to be compared primarily within round
My experience is that SROs will jump all over it if the discussion trends towards "We can't really assign a worse score than the previous version" or anything else that smacks of score-anchoring to prior rounds. (Of course, reviewers do this, they just aren't supposed to. )
So there are a number of quite reasonable scenarios for why your score could get worse, even if your application has been "objectively" improved through revision. First and foremost because there may simply be more applications that are better this round. It doesn't take many to go from scored to triaged, btw. My panel sees maybe 40-60 R01 apps so suppose that 25 are discussed. Obviously for any threshold, someone has to be the worst one discussed but just think about, say, 5 applications shifting one way or the other relative to yours. Not so hard to imagine is it? That from round to round the variance in theoretical objective merit should easily change by a mere 5 proposals?
Second, of course, because there is variance in the within-round review process depending on the specific reviewers assigned to your proposal and the pools of applications that have been assigned to those reviewers. People are loathe to believe that their good score was a result of the other proposals in a given reviewer's pile happening to be unusually bad that round. It happens. So let us think about doing a resampling process on your application. Take three different sets of reviewers, four or five different plausible sets of grant assignments for each of those reviewers and I guarantee you would see substantial variation in the outcome for a given app. And not variation from a 180 to a 183, neither. So how does the applicant know that the score they received is not simply on the extreme good side of the distribution of likely scores they could have received? They do not.
I beg of you applicants who have not sat on study section yet. Think about what you are up against and get realistic about the process. you are not special. You are not smarter than everyone else. You do not write better grants. You are not more deserving. People are not out to get you specifically.
It is going to be helpful for your mental health and blood pressure if you stop viewing every disappointing grant score as a personal attack and evidence that the system is irretrievably broken.