Evaluation of implementation outcomes for the IMPACT implementation support platform

Abstract

Background: The IMPACT implementation support platform is a measurement and feedback system specifically designed to scale evidence-based programs and practices (EBPPs) and support high-quality implementation. The purpose of this study was to evaluate use of the IMPACT software system in terms of its acceptability, appropriateness, feasibility, and likelihood of adoption. Methods : Seventy-eight school-based providers (across 49 schools) participated in this study. Demographic and background information was collected in the fall of 2019. Thereafter, providers delivered the Social Skills Group Intervention (S.S.GRIN) EBPP with students as usual and used IMPACT to enter and track progress and process data for their S.S.GRIN groups. In the spring of 2020, providers completed a series of ratings to evaluate their experience with IMPACT. Providers' usage of IMPACT during the study was also tracked. Results : Providers' ratings of IMPACT for each implementation outcome were significantly higher than average ( p < 0.001), with Satisfaction, Superior Innovation, and Interface Quality being especially positive. There was a significant interaction effect where ratings were higher for providers who reported disliking implementation data tracking for S.S.GRIN at the start of the study. Greater usage was significantly associated with higher ratings of providers’ capacity for tracking implementation data and a higher likelihood of recommending IMPACT to others. No significant differences in the patterns of results were found for demographic subgroups. Conclusion : IMPACT was seen as acceptable, feasible, and appropriate by school-based S.S.GRIN providers who reported likely continued use (adoption) of the innovation. IMPACT was particularly well received by those who started the study with negative impressions of implementation data tracking for S.S.GRIN. This study supports the potential utility and value for supporting ongoing implementation of school-based programs. Future research is needed to evaluate IMPACT for other EBPPs and in other service delivery settings to determine the generalizability of this study’s findings.