CLIC - CTSA Informatics Common Metric Post-Pilot Survey

Purpose

To accelerate translational research and enhance collaboration and opportunities for discovery through interoperable data.

Executive Summary

The goal of this Informatics Common Metric is to enhance collaboration and opportunities for discovery through interoperable data. Informatics provides critical tools, methods and resources to accelerate translational research. The forthcoming Informatics Common Metric aims to improve discovery opportunities within and among CTSA Program hubs through metrics that encourage data repository quality and harmonization across hubs.

A team of informatics and evaluation experts developed the metric definition and Operational Guidelines. The Common Metrics Implementation team at the Center for Leading Innovation and Collaboration (CLIC) began the pilot activities in August 2017 through March 2018. Sixteen hubs participated in the pilot, representing diversity such as: hub size, single vs. multi-institution, and data model (i2b2/ACT, i2b2/TriNetX, OMOP, PCORnet).

Based on post-pilot assessment including hub interviews, a number of key findings were identified to inform the recommendations for Consortium-wide implementation. These included: (1) Providing scripts to the hubs was a critical addition to the metric that was viewed positively by the hubs, as it reduced duplicative efforts. (2) The amount of script editing required varied by data model and hub. (3) Informatics Metric was seen as valuable to all although new to some hubs. (4) Not all hubs had data for all required domains, (e.g. observations) although all the domains were seen as valuable. (5) Age of the data was a significant concern to the hubs – the value of including data older than
10 years was questioned. (6) For some hubs influence on improving their data warehouse was limited as not all hubs view themselves as able to influence the management and/or collection of the clinical data. (7) RBA process was viewed as valuable to strategic management although there were challenges with the Clear Impact Scorecard. (8) Hubs indicated it would be helpful to have guidance on metric team formation and engagement.

Prior to Consortium scale-up, CLIC suggests the following recommendations: (1) Modify the Operational Guidelines to refine data collection and clarify scope of data. (2) Expand CLIC technical support to the hubs including facilitating access to script subject matter expert. (3) Develop Implementation Guide specific to the metric. (4) Market the
Informatics metric to demonstrate value and future potential. (5) Revise the structure of the Scorecard to reduce redundancies.

Communication of the pilot findings and network scale-up plans will be presented to key Consortium stakeholder groups over the next couple of months. Scale-up
implementation to the Consortium will start upon approval from the NCATS leadership.

Distributed Date
Files

Please log in to access this content

Instrument Fields
Order Instrument Field
1

Your Institution

2

Please provide your Name/Title

3

When your hub initially volunteered to be an Informatics Metric Pilot site, which data model did your hub propose to use for the pilot?

4

Did you use this data model for the pilot?

5

If you did not use the data model for the pilot, please explain why.

6

Which data model did you use for the pilot?

7

Please explain why you used the data model you chose for the pilot.

8

When this metric is implemented across the CTSA Program, will you use the same data model that you used for the pilot?

9

If you will not use the same data model that you used for the pilot when this metric is implemented or you are not sure, please explain why.

10

If you will not use the same data model that you used for the pilot when this metric is implemented, what model would you plan to use?

11

Please provide other comments related to your hub's choice of data model for this metric pilot.

12

Please provide your Name/Title

13

Was it easy to access the script from the NCATS CTSA-Metrics GitHub site?

14

If it was not easy to access the script from the NCATS CTSA-Metrics GitHub site, please explain why.

15

Did your team have any other challenges working with the NCATS CTSA-Metrics GitHub site?

16

If your team had any other challenges working with the NCATS CTSA-Metrics GitHub site, would a short tutorial on GitHub be useful?

17

If you believe your team would find or might find a short tutorial on GitHub useful, does your team have any suggestions as to how to make this easier?

18

Did you post questions and/or information on the GitHub site?

19

If you posted questions and/or information, how easy was it to post information on the GitHub site?

20

Was the question resolved within 1-2 business days?


Submitted by Jacqueline Attia on Thu, 02/27/2020 - 13:23