An ad hoc committee of the National Academies of Sciences, Engineering, and Medicine explored the issues of reproducibility and replication in scientific and engineering research, focusing on defining reproducibility and replicability, and examining the extent of non-reproducibility and non-replicability. The resulting report offers definitions of reproducibility and replicability and examines the factors that may lead to non-reproducibility and non-replicability in research. While reproducibility is straightforward and should generally be expected, replicability is more nuanced, and in some cases a lack of replicability can aid the process of scientific discovery. The report provides recommendations to researchers, academic institutions, journals, and funders on steps they can take to improve reproducibility and replicability in science.
CLIC Resource Center
The CLIC Resource Center is a place for CTSA Program members to share relevant and useful guides, manuals and other tools with the full consortium. This page aims to encourage collaboration and resource-sharing so that hubs across the program don’t need to ‘recreate the wheel’ every time they’re faced with a challenge or question.
These resources may be related to administration, informatics, team science or communication, among other subject matters. Similar to our Events and News pages, all CLIC website users are invited to submit resources, but must be logged into the site to access the submission form.
Reproducibility and Replicability in Science
Penn State CTSI
Integrity in Scientific Research Videos
This video series will introduce key topics and ethical dilemmas in the responsible conduct of research. We hope it will prompt the exchange of information and discussion about research integrity principles among undergraduate/graduate students, post-doctoral fellows, scientists, and administrators. These videos were filmed at Georgetown University with the coordination and assistance of the REKS component of their CTSA Program hub, and can be used by all CTSA Program investigators and trainees.
The National Research Mentoring Network
The NRMN provide researchers across all career stages in the biomedical, behavioral, clinical and social sciences with the evidence-based mentorship and professional development programming that emphasizes the benefits and challenges of diversity, inclusivity and culture.
Programs include mentor training, mentee training, facilitator training, grant writing coaching and a guided virtual mentorship called MyMentor.
Resources include mentor certification, facilitator certification, professional development webinars and a social networking platform called MyNRMN.
National Institutes of Health, Grant Number: 5U54GM119023
Principles of Community Engagement - Second Edition
Clinical and Translational Science Awards Consortium Community Engagement Key Function Committee Task Force on the Principles of Community Engagement
Principles of Community Engagement (Second Edition) provides public health professionals, health care providers, researchers, and community-based leaders and organizations with both a science base and practical guidance for engaging partners in projects that may affect them. The principles of engagement can be used by people in a range of roles, from the program funder who needs to know how to support community engagement to the researcher or community leader who needs hands-on, practical information on how to mobilize the members of a community to partner in research initiatives. In addition, this primer provides tools for those who are leading efforts to improve population health through community engagement.
This primer was developed by the CDC and updated by the CE KFC taskforce
Download a pdf of the full document, view chapters online or order a print copy.
Publication: Evaluation metrics for biostatistical and epidemiological collaborations
Research Design (BERD) Key Function Committee of the Clinical and Translational Science Awards (CTSA) Consortium
Evaluation metrics for biostatistical and epidemiological collaborations
Rubio DM, Del Junco DJ, Bhore R, Lindsell CJ, Oster RA, Wittkowski KM, Welty LJ, Li YJ, Demets D; Biostatistics, Epidemiology, and Research Design (BERD) Key Function Committee of the Clinical and Translational Science Awards (CTSA) Consortium.
Stat Med. 2011 Oct 15;30(23):2767-77. doi: 10.1002/sim.4184. Epub 2011 Feb 1. Erratum in: Stat Med. 2012 Mar 15;31(6):600.
A consortium-endorsed publication related to Biostatistics, Epidemiology, and Research Design (BERD)
Clinical Trials in 15 Minutes
Penn State Clinical and Translational Science Institute
Katie Bode-Lang, Director of Education and Quality Management, Office for Research Protections at Penn State, covers the NIH definition of Clinical Trials in 15 minutes.
Un-Meeting Event Guide
A comprehensive guide to help anyone involved with the CTSA Program host their own successful Un-Meeting. The Guide includes elements such as logisitc planning, identifying your team, promotion and communication best practices, budgeting templates and more.
Translational Science Benefits Model
Washington University Institute of Clinical and Translational Sciences
The Translational Science Benefits Model (TSBM) is a framework designed to support institutional assessment of clinical and translational research outcomes to measure clinical and community health impacts beyond bibliometric measures. The TSBM includes 30 specific and potentially measurable indicators that reflect benefits that accrue from clinical and translational science research such as products, system characteristics, or activities. Development of the TSBM was based on literature review, a modified Delphi method, and in-house expert panel feedback. Three case studies illustrate the feasibility and face validity of the TSBM for identification of clinical and community health impacts that result from translational science activities. Future plans for the TSBM include further pilot testing and a resource library that will be freely available for evaluators, translational scientists, and academic institutions who wish to implement the TSBM framework in their own evaluation efforts.
Regulatory Guidance for Academic Research of Drugs and Devices (ReGARDD)
University of North Carolina at Chapel Hill Translational & Clinical Science Institute
ReGARDD provides academic researchers with the regulatory tools and resources necessary to successfully navigate the pathway from discovery to clinical implementation of new drugs, biologics and medical devices.
The ReGARDD affiliates are comprised of regulatory affairs specialists and experts from North Carolina Institutions that receive funding from the NIH Clinical and Translational Science Awards (CTSA) Program. Currently these institutions are the University of North Carolina at Chapel Hill, Duke University, Wake Forest School of Medicine, and RTI International.
Combining the regulatory talents from the North Carolina CTSAs and the research triangle park (RTP) area enables sharing of ideas, lessons learned, historical information, and the development of successful strategies to assist the academic researcher in navigating an increasingly complex regulatory environment.
Data and Safety Monitoring Board Training Manual
Tufts Clinical and Translational Science Institute
The CTSA Program Collaborative Data and Safety Monitoring Board (DSMB) Workgroup identified a need to provide guidance, training and resources in DSMB practices for investigator-initiated research studies. To promote clinical and translational research, the CTSA Program Collaborative DSMB Workgroup, supported by NCATS, produced an online DSMB Training Manual with a focus on investigator-initiated studies. This manual provides information and training for Principal Investigators, DSMB members, IRB members, biostatisticians, and research staff on how to work effectively with DSMBs.