In 2016, NIH announced the Rigor and Reproducibility policy to better ensure researchers use unbiased and well-controlled experimental design, methodology, analysis, interpretation, and reporting of results. Further, the policy encourages scientific integrity, public accountability, and social responsibility.
Additionally, NIH requires formal instruction in rigorous experimental design and transparency to enhance reproducibility for institutional training (T), individual career development (K), and individual fellowship (F) applications.
NIH’s policies on rigor and reproducibility center on four key facets for research project grant applications:
Scientific Premise
What It Is: Research that forms the basis for the proposed research question(s).
What To Do: Describe the general strengths and weaknesses of the prior research by others that you are citing as crucial to supporting your application. In considering the general strengths and weaknesses of this prior research, include attention to the rigor of the previous experimental designs as well as the incorporation of relevant biological variables and authentication of key resources. If a prior study being cited has limitations, describe how those limitations are being addressed in the proposed research.
Scientific Rigor
What It Is: The strict application of the scientific method to ensure robust and unbiased experimental design, methodology, analysis, interpretation, and reporting of results.
What To Do: Include full transparency in reporting experimental details so that others may reproduce and extend the findings. Describe sample size calculations, how animals or human subjects are allocated to experimental groups, whether and how the study was blinded, as well as the statistical methods used, etc.
Biological Variables
What They Are: Factors relevant to the experimental design of the study such as sex, weight, age, and underlying health conditions.
What To Do: Explain how biological variables factor into research designs, analyses, and reporting in vertebrate animal and human studies. Note: If you are proposing to study only one sex, you must provide strong justification from the scientific literature, preliminary data, or other relevant considerations.
Authentication
What It Is: Key biological and/or chemical resources, e.g., cell lines, specialty chemicals, antibodies, and other biologics.
What To Do: Provide an authentication plan that is one page or less and includes only a description of the methods proposed to authenticate key resources before use and at regular intervals, if appropriate.
These four aspects are further explained on NIH’s Rigor and Reproducibility in NIH Applications: Resource Chart.
Peer reviewers assess research project grant applications by considering the following questions from NIH’s reviewer guidelines:
- Is there a strong scientific premise for the project?
- Have the investigators presented adequate plans to address relevant biological variables, such as sex and age, for studies in vertebrate animals or human subjects?
- Have the investigators presented strategies to ensure a robust and unbiased approach, as appropriate for the work proposed?
Scientific rigor contributes to the Approach criterion for research project grant applications and the Research Plan criterion for mentored career development award applications, as well as the overall impact score. Authentication plans are discussed after scoring; inadequate plans must be addressed before an award is issued.
With those assessment criteria in mind, NIH’s Enhancing Reproducibility Through Rigor and Transparency page is the best place to start when building the Research Plan of your next application. Pay close attention to the listed examples within the Resources for Preparing Your Application section.
Once awarded a grant, you will provide updates on those same aspects, describing how your research ensures reproducibility, when you submit an annual Research Performance Progress Report (RPPR).
If you are encountering reproducibility as a topic for the first time, review NIH’s Training Modules on the topic. Several other helpful resources to check out are Online Training Resources from NIH’s Office of Behavioral and Social Sciences Research, a video series titled Pragmatic and Group-Randomized Trials in Public Health and Medicine from NIH’s Office of Disease Prevention, and Clearinghouse for Training Modules to Enhance Data Reproducibility from the National Institute of General Medical Sciences.
Finally, you can read applications on our Sample Applications page. Those using FORMS-D, -E, -F, or -G application packages will demonstrate successful consideration of rigor and transparency.