- News
- Mike’s Blog
- CLIC News Roundup
- CTSA Ansible
What is the real-world impact of our network? That is a question the Clinical and Translational Science Awards Program has been trying to answer since its inception. Over the past four years, the Center for Leading Innovation and Collaboration has pitched in on that effort, knitting together several tried and trusted evaluation methods to develop a new Stakeholder-Engaged Research Impact Evaluation model, or SERIE.
In its 15 years, the CTSA Program has employed a range of methods to assess its progress toward “accelerat[ing] the process of turning observations in the laboratory, clinic and community into interventions that improve the health of individuals and the public.” But none of these methods were designed to evaluate impact at a network level.
Research networks, like the CTSA Program, tend to struggle to define their real-world impact and often opt to quantify outputs like publications, grants and clinical trial enrollment as proxy measures of their success.
Employing both quantitative and qualitative evaluation methods, CLIC’s new SERIE model can help research networks overcome these barriers to better measure and communicate their impact. By fostering stakeholder engagement, SERIE helps networks define their impact as the real-world changes that are valued by the people, organizations and communities involved in and affected by the network.
“There is no impact without stakeholders who define what is meaningful,” said CLIC Co-Director Martin Zand, M.D., Ph.D. “Consulting with stakeholders is a crucial first step to measuring and then communicating impact.”
The SERIE model evolved and was tested over the past four years by the CLIC. Building on the Common Metrics Initiative and adopting its software platform, CLIC incorporated three contextual, real-world impact assessment methods into SERIE: the Translational Science Benefits Model (TSBM), the RE-AIM framework and logic models.
Using logic models, SERIE distinguishes outputs (like publications and grants) from real-world impacts defined by stakeholders. SERIE also expands upon the TSBM developed at Washington University in St. Louis, to assess the real-world medical, public health, economic and policy benefits of clinical and translational science networks, rather than individual institutions. And the RE-AIM framework (which stands for reach, efficacy, adoption, implementation, and maintenance) helps SERIE evaluate the dissemination and implementation of evidence-based interventions.
The principles behind these four models have been distilled into the five steps of SERIE: engage, define, collect, analyze, and communicate. Using SERIE, networks engage their various stakeholders to define their potential impacts and to identify the qualitative and quantitative metrics that will help measure those impacts. Metric data are then gathered and analyzed and the resulting insights on real-world impacts are communicated back to stakeholders.
Though the CLIC has tested SERIE across the CTSA Program, this model needs further validation with other large translational science networks. The CLIC hopes to continue that work soon.
- Evaluation
- Stakeholder Engagement
- Research Evaluation
- Impact
- CLIC