It’s been nearly 15 years since criterion scores were introduced in the peer review process, but each new generation of applicants encounters a similar point of confusion about how these scores are used.
To arrive at your overall impact score, reviewers first consider the core review criteria: significance, innovation, investigator, approach, environment, as well as any additional funding opportunity-specific requirements. They then assign an overall impact score, which takes into account the review criteria, but weights their importance however each reviewer chooses. This deliberate approach anchors assessments and encourages consistent use of the scoring range.
Thus, while criterion scores and overall impact scores are both numerical values, there is no mathematical formula linking the two.
NIAID funds the research it deems most likely to exert a sustained, powerful influence on its field. An overall impact score reflects the review criteria as is appropriate for each opportunity and application.
Therefore, an application does not need to be exceptionally strong in all of the review criteria to get an exceptional overall impact score. For example, reviewers may give an application an outstanding overall impact score due to its high significance and feasibility, even though it has only a good innovation score because (for example) the project uses current state-of-the-art approaches. Alternatively, a critical flaw in the study design may cast doubt on the likelihood that you can execute a proposal with very good significance and innovation, resulting in a marginal overall impact score.
Notes on the Scoring Range for Grant Applications
Criterion scores range from 1 to 9, where 1 is the best. Overall impact scores range from 10 to 90, where 10 is the best.
For investigator-initiated R01 applications, NIAID converts overall impact scores into percentiles to rank applications relative to others scored by the same study section.
An application is “not discussed” and does not receive an overall impact score if reviewers assess it to have a relatively low level of competitiveness for the available funding. Such applications still include preliminary criterion scores in the Summary Statement. Setting some applications aside as not discussed streamlines the review process.
In the case of multicomponent applications (e.g., P01s and U19s), check whether synergy is highlighted in the opportunity’s Section V. Application Review Information. For those that do, strong synergy among projects can lead to an overall impact score for the entire proposal that is greater than the scoring average of the individual components.
Criterion scores are provided for all applications. However, only applications that are discussed receive overall impact scores. The criterion scores reflect the views of the assigned reviewers while your overall impact score reflects the scores of all the panel members who voted. Therefore, it’s important to read the resume and summary of the discussion in the summary statement as well as the comments from individual reviewers.
If you decide to resubmit an application, consider the original application’s criterion scores and reviewer comments as you write your application to help earn a better overall impact score on the resubmission. By understanding which criteria scored poorly, you can better target areas for improvement.
For a longer discussion of the mechanics of peer review, read Scoring & Summary Statements.
As you may know, NIH is preparing for a transition from using five scored review criteria to three review factors, as explained at Simplifying Review of Research Project Grant Applications. Regardless, overall impact scores will continue to function in the same way—reflective of the main review criteria, but not derived by algorithm from them.