Do Common Metrics Add Value? Perspectives from NIH Clinical and Translational Science Awards (CTSA) Consortium Hubs

Introduction

The Clinical and Translational Science Awards (CTSA) Consortium, a network of academic health care institutions with CTSA hubs, is charged with improving the national clinical and translational research enterprise. The CTSA Consortium and the NIH National Center for Advancing Translational Sciences implemented the Common Metrics Initiative comprised of standardized metrics and a shared performance improvement framework. This paper summarizes hubs’ perspectives on its value during the initial implementation.

Methods

Value was assessed across 58 hubs. Survey items assessed change in perceived ability to manage performance and advance clinical and translational science. Semi-structured interviews elicited hubs’ perspectives on meaningfulness and value added of the Common Metrics Initiative and hubs’ recommendations.

Results

Hubs considered their abilities to manage performance to have improved, but there was no change in perceived ability to advance clinical and translational science. The initiative added value by providing a formal structured process, enabling strategic conversations, facilitating improvements in processes, providing external impetus for improvement, and providing justification for funds invested. Hubs were concerned about usefulness of the metrics chosen and whether the value added was sufficient relative to effort required. Hubs recommended useful benchmarking, disseminating best practices and promoting peer-to-peer learning, and expanding the use of data to inform the initiative.

Conclusions

Implementing Common Metrics and a performance improvement framework yielded concrete short-term benefits, but concerns about usefulness remained, particularly considering the effort required. The Common Metrics Initiative should focus on facilitating cross-hub collaboration around metrics that address high-priority impact areas for individual hubs and the Consortium. 

Authors
Lisa C. Welch1,2, Andrada Tomoaia-Cotisel3, Hong Chang1,2, Peter Mendel3, Jason M. Etchegaray3, Nabeel Qureshi3, Marguerite Fenwood-Hughes1, Anshu Parajulee1, Harry P. Selker1,2

1 Tufts Clinical and Translational Science Institute, Tufts University, Boston, MA
2 Institute for Clinical Research and Health Policy Studies, Tufts Medical Center, Boston, MA
3 RAND Corporation, Santa Monica, CA
Journal
Journal of Clinical and Translational Science
Publication Date
DOI
https://doi.org/10.1017/cts.2020.565
Linked Authors
Lisa
Welch
Director of Evaluation, Tufts Clinical and Translational Science Institute
Marguerite Fenwood Hughes
Fenwood Hughes
Project Manager
Harry
Selker
Dean of Tufts Clinical and Translational Science Institute (CTSI)