There are line graphs available for the data. From the CM-PRISM form, enter and save your data, then click the "View Chart" button - the graph will display in a separate browser window. This feature will evolve over time.
Common Metrics Initiative Frequently Asked Questions
Starting with 2020 data, the informatics scripts were written to collect data for one year and the five previous years.
Please submit your questions and suggestions to the CLIC team (email@example.com) or submit them through the CLIC website. CLIC will forward these items to NCATS.
Metrics - General
The CM-PRISM data is de-identified and aggregated and provided to NCATS in a report. Since this data is de-identified, NCATS cannot use this report to make funding decisions.
Hubs receive a report that shows consortium aggregated data and their own individual data.
The Operational Guidelines were updated to indicate CTSA program hubs should use the calendar year for each of the Common Metrics (January 1 - December 31).
Data and Program Summaries are due annually on the last business day of August.
Metrics - Careers
You should be collecting data on the number and percentage of underrepresented persons. You do not need to break it down into three categories. Please see NIH's Interest in Diversity statement at https://grants.nih.gov/grants/guide/notice-files/NOT-OD-20-031.html
At this time, women are not considered underrepresented persons. See the NIH definition: https://diversity.nih.gov/about-us/population-underrepresented
It is not possible to have a KL2 Program without a U54 CTSA Program award.
No, these are examples of what it means to be engaged in research, given these different roles. Hubs may want information regarding an individual’s primary role in research for their own evaluation, but it is not reported for the common metrics.
Please include only CTSA program-funded KL2 Scholars and TL1 Trainees when reporting Common Metric data.
NCATS has clarified that non-CTSA grant funded scholars and trainees who participate in your KL2 or TL1 program should not be included.
If any demographic characteristic is missing, then the data for the individual should be removed from the numerator and denominator.
Graduates should be added to the denominator for the calendar year that they finish their program.
Use the year that they started the grant.
These are examples of what people in those roles would be doing to assist hubs in capturing “engaged” data.
The template, which is optional, is designed to assist hubs in collecting data for KL2/TL1 scholars and trainees. Hubs are not required to collect data from previous years - use the template to collect data only for the year that you are reporting/entering data in CM-PRISM.
This information is not currently required for the Common Metrics.
The gender categories have been changed to Male, Female, and Other - at this time these are the only options. This issue will continue to be addressed as the metric evolves.
Lost to follow-up designation is for any Scholars/Trainees for whom their current research status cannot be determined. Hubs should continue to try to follow up with these Scholars/Trainees for 5 years before dropping them from the dataset. Those who are lost to follow-up after 5 years are not included in the denominator, for common metrics reporting but hubs should continue to try to track them, as they may re-enter clinical and translational research.
Yes, remove them from the numerator and denominator of the metric.
Yes, this is considered to still be engaged in research.
There is a specified percentage of time defined for "engaged in research" - please see the Technical Description section of the Operational Guideline.
This metric reflects all research.
Following completion of CTSA-funded training as a KL2 scholar, they are eligible to be counted for the metric. If they are engaged in further training by another K award, they are considered engaged in research. However, if a scholar leaves the KL2 program without completing the full training program requirements, they are excluded.
Use the scholar’s most recent classification.
Please use the definition provided by NCATS: https://grants.nih.gov/grants/guide/notice-files/NOT-OD-20-031.html
Graduates are to be assessed for their engagement in clinical and translational research annually.
The list of activities that indicate engagement in research are examples only and not criteria.
No, they should not be counted until they finish grad school.
No, they should not be included until they finish their residency.
This metric reflects all research.
Yes, they should be counted. The goal of the program is to place scholars and trainees in research positions.
Yes, if they are going back to medical school (or another clinical degree program like PharmD, DPT, etc.). These individuals are currently in "training" and are not engaged in research. Counting them in the currently engaged numerator would affect the hubs' percentages in a negative way.
If they are not directly funded by NCATS then they are excluded when reporting the metric.
At this time, use gender identification at the time of application.
The total number of program graduates over time is cumulative. It gets updated once a year by adding the new number of graduates to the previous total and updating the denominator for the metric. The numerator, the number and percent of graduates from the denominator who are currently engaged in clinical and translational research is assessed each year.
eRA will be a source of information but it won’t be a complete source, because they are not all inclusive; there are activities that are outside of the system.
Refer to the definitions of being engaged in research.
We are using the NIH definitions. The Operational Guideline has a link to more information about these definitions. The most up to date versions of the Operational Guidelines can be found on the Established Common Metrics page.
This is the link to the most current NIH definition: https://diversity.nih.gov/about-us/population-underrepresented
Use the NIH definition: https://diversity.nih.gov/about-us/population-underrepresented
Scholars/Trainees should be followed for at least five years.
CLIC conducts qualitative analysis on the narratives. This information then serves as the foundation for the next year’s Insights to Inspire program. This program fosters innovation and ideas, and encourages collaboration across the consortium. Click here to learn more.
Scenario: So for our pre-doctoral trainees, they'll finish medical school, do an internship in residency so they are in training for all that time, let's say they come on as faculty and they do a career development award from the NIH. Technically, if they have any kind of K award they're considered in training. So we can't count them.
Response: Any individual who receives a new career development award is counted as being engaged in research.
Exclusion Criteria: Trainees and Scholars who are still in training or who have left the program without completing the full training program requirements. Program alumni who are still in residency or other degree-seeking programs are also excluded.
Using the denominator of "cumulative number of graduates" more accurately reflects the percentage of women and underrepresented persons (URPs) engaged in research.
Metrics – Informatics
Yes, hubs can enter qualitative data for up to three topics.
While only one Program Summary is required for the Informatics and Career metrics, hubs are encouraged to enter additional topics. This information is shared by the Insights to Inspire program with the consortium as a method of sharing actionable intelligence.
The metric includes all patients in the research data warehouse – inpatient, outpatient, ER, etc.
Tools are available on the CLIC website: https://clic-ctsa.org/common-metrics-initiative
Total # of unique patients in the data warehouse
Total # of unique encounters in the data warehouse
Total # of unique patients with age >12 in the data warehouse
The scripts are updated each year to "pull" the correct dates. Starting with 2020 data, the scripts were written to collect data for one- and five-year periods.