Insights to Inspire: The 4 Stages of a Successful Pilot Funding Publication Program
As a key currency of the scientific community, publications are often used as a measurement of success for researchers and research institutions. Published research allows an expansive community of scientists, healthcare professionals, government workers and more to make data-driven decisions that will improve population and patient health outcomes. The Clinical and Translational Science Award (CTSA) Program, funded by the National Center for Advancing Translational Science (NCATS), seeks to drive the continued development of research through pilot funded projects that will yield tangible medical and population health interventions, which can then be disseminated through scientific literature.
In order to maximize the consortium’s impact, NCATS created the Common Metrics Initiative (CMI). The vision for the CMI is to use specific, common metrics to demonstrate and improve the impact of the CTSA Program. The CMI, implemented by the Center for Leading Innovation and Collaboration (CLIC), uses this set of metrics to help focus network and hub activities on making significant, measurable improvements in translational science and research, and workforce development.
Included in this set of metrics is the Pilot Funding Publication Common Metric. This metric assesses the total number of scholarly publications resulting from pilot funded research at a CTSA Program hub each year. In 2019, the Common Metrics Initiative released a Multi-Year Report that identifies the hubs with the most improvement in each of the three metrics for the measurement period of 2016-2017. A selection of the hubs that saw the greatest improvement in their Pilot Funding Publication metric shared their strategies for making improvements across the four key stages of a successful Pilot Funding Publication Program and their advice for how others can implement similar changes.
Stage 1: Fund the right projects
To create this improvement, CTSI of Southeast Wisconsin took several steps to address their pilot funding process. One of the areas they focused on was how to select the right projects to receive funding in the first place. “We created a more cohesive and extensive review process that strategically assesses the potential of each project,” said Dr. Reza Shaker, Principal Investigator and Director, CTSI of Southeast Wisconsin. “We also increased the number of reviewers for each application, emphasizing the importance of quality projects, not just the quantity of them.”
As University of Miami looked to make improvements to their Pilot Funding Publication metric, they zeroed in on their approach to selecting the right projects to fund. “We have a rigorous review process in place and as a result we end up funding very high-quality research projects that often lead to high-quality publications,” said Dr. Dalton Dietrich, Co-Investigator and Pilot Program Director, University of Miami. University of Miami also emphasizes the importance of being upfront with applicants about the expectation of a scholarly outcome. By setting that expectation early and then checking in with researchers regularly throughout their project, University of Miami keeps the goal of publication at the forefront of awardees’ minds.
University of Miami also found that, when possible, if they support pilots that focus on new or emerging health topics, there is greater potential to secure sizeable subsequent funding and additional publication by tapping into increased interest around a “trending” topic. “During the Zika outbreak in Florida, we awarded several innovative pilot projects that focused on the issue and this led to large amounts of additional federal and state funding for these projects as well as additional publications,” said Rosalina Das, Senior Evaluation Manager, University of Miami. Despite being a smaller hub, such “the greater the risk, the greater the reward” strategies can also be applied at larger hubs to deliver similar results.
Stage 2: Support the researcher on his/her path to publication
University of Miami also attributes their improved Pilot Funding Publication Common Metric to the creation of multiple touchpoints with pilot funded researchers as they carry out their project. The hub has created a reputation among their pilot funded awardees as a resource for ongoing research support, grant writing guidance, letters of support, peer reviews, and more. Additionally, pilot funded researchers are given clear points of contact at the CTSI who they can reach out to with questions or concerns, making it easy to address issues when they arise and minimize any downtime in the research.
Michelle Miller and her team at Scripps Research Translational Institute took a similar approach to improving their hub’s Pilot Funding Publication Common Metric saying, “by personalizing our interactions with pilot funded awardees, we were able to develop a rapport with them, making them feel comfortable enough with our team to air concerns and ask questions before major issues derailed their project.”
To prevent struggling projects from falling through the cracks, Weill Medical College of Cornell’s CTSC Common Metrics committee leveraged the investigator survey tool, PROMPTR. The team created a “needs page” that is sent to pilot funded researchers while their project is in-process. The form asks a series of questions to better understand the researcher’s expectations for their project and what challenges they foresee. The page is then reviewed by the hub’s PI. “The needs page generates a report for us that lists projects that are currently in the works, including descriptions and status updates, so that we can review and identify potential issues and offer interventions” said Elizabeth Wood, Co-Director, Biomedical Informatics, Weill Medical College of Cornell. For more information on how to leverage PROMPTR’s needs pages at your own hub, please view this webinar overview from Weill Medical College of Cornell and CLIC.
Stage 3: Ensure CTSA Program grant citation
Not only do multiple checkpoints with pilot funded awardees allow PIs to troubleshoot for potential issues with struggling projects, they also provide an opportunity to remind researchers about citing the CTSA grant number. Without proper citation of the grant number, not only is that hub not receiving credit for their role in funding the research, it makes it much less likely that evaluators will be able to find the publication when reporting on the hub’s Common Metrics. “We make it a priority to share grant numbers early in an awardee’s project,” said Michelle Miller, Operations Manager, Scripps Research Translational Institute. “By providing researchers with the grant number up front and then sending frequent reminders, we were able to make significant strides with the accuracy of their citation.”
Stage 4: Find the publication once it gets published
While it may seem simple, tracking publications that result from pilot funded projects can often be the most challenging part of securing a strong Pilot Funding Publication Common Metric. To help address this, the team at Weill Medical College of Cornell, led by Elizabeth Wood, developed PAIRS, a tool within WebCamp that proactively identifies publications potentially resulting from CTSC projects. It does so by issuing a broad search of PubMed and matching publications to pilot project titles based on search terms. “For each automated search, the tool generates a report that allows the user to review all publications found and either confirm it as an outcome of pilot funding, deny it as an outcome or ask the system to generate an email to the PI requesting them to review whether it is the result of a pilot funded project from the CTSC,” said Wood. “By automating our search for publications resulting from pilot funded projects, we’re able to more accurately reflect the results of our hub.” PAIRS is available to all CTSA Program hubs. To learn more about the tool, please view this webinar hosted by CLIC and Weill Medical College of Cornell.
PAIRS can streamline processes for both large and small hubs. While larger hubs tend to have a greater quantity of projects and publications to keep track of, smaller hubs are often looking for ways to automate and reallocate manpower to other areas. For example, Michelle Miller, Operations Manager for Scripps Research Translational Institute says, “we are proud of the improvements we’ve made to our metric in the last few years and as we look for ways to continue that progress, automation is a top priority.”
CTSA Program hubs rely on appropriate citations of their CTSA grant numbers as a critical performance measure when reporting on annual productivity. As such, it is crucial that hubs have a system in place that supports projects on their path to publication from start to finish. By prioritizing each of these key stages to developing pilot funded publications, hubs can fuel research growth at their institution and beyond.