Implementation research and practice最新文献

Background: Despite the ubiquity of multilevel sampling, design, and analysis in mental health implementation trials, few resources are available that provide reference values of design parameters (e.g., effect size, intraclass correlation coefficient [ICC], and proportion of variance explained by covariates [covariate R ²]) needed to accurately determine sample size. The aim of this study was to provide empirical reference values for these parameters by aggregating data on implementation and clinical outcomes from multilevel implementation trials, including cluster randomized trials and individually randomized repeated measures trials, in mental health. The compendium of design parameters presented here represents plausible values that implementation scientists can use to guide sample size calculations for future trials.

Method: We searched NIH RePORTER for all federally funded, multilevel implementation trials addressing mental health populations and settings from 2010 to 2020. For all continuous and binary implementation and clinical outcomes included in eligible trials, we generated values of effect size, ICC, and covariate R² at each level via secondary analysis of trial data or via extraction of estimates from analyses in published research reports. Effect sizes were calculated as Cohen d; ICCs were generated via one-way random effects ANOVAs; covariate R² estimates were calculated using the reduction in variance approach.

Results: Seventeen trials were eligible, reporting on 53 implementation and clinical outcomes and 81 contrasts between implementation conditions. Tables of effect size, ICC, and covariate R² are provided to guide implementation researchers in power analyses for designing multilevel implementation trials in mental health settings, including two- and three-level cluster randomized designs and unit-randomized repeated-measures designs.

Conclusions: Researchers can use the empirical reference values reported in this study to develop meaningful sample size determinations for multilevel implementation trials in mental health. Discussion focuses on the application of the reference values reported in this study.

Background: Shifting organizational priorities can negatively affect the sustainment of innovations in community settings. Shifting priorities can present barriers to conducting clinical research in schools if a misalignment occurs between school district priorities and the aims of the study. Often this misalignment occurs due to a shift during the period between when the study is submitted for funding and when research activities begin. Participatory research approaches can be employed to restore alignment between study processes and school district priorities. The purpose of the study is to describe data from a shared process with district partners. The shared process resulted in modifications to the main study's implementation processes and strategies in order to restore alignment with evolving school priorities while remaining faithful to the aims of the study.

Method: Data originated from qualitative interviews conducted with 20 school district and school personnel in a large urban school district. Qualitative themes were organized into categories based on a social-ecological school implementation framework. Data from team meetings, meetings with school district administrators, and emails served to supplement and verify findings from interview analyses.

Results: Themes included barriers and facilitators at the macro-, school-, individual-, team-, and implementation quality levels. Adaptations were made to address barriers and facilitators and restore alignment with school district priorities. Most adaptations to study processes and implementation strategies focused on re-training and providing more information to school district coaches and school-based staff. New procedures were created, and resources were re-allocated for the larger study.

Conclusions: Findings were discussed in relation to the implementation literature in schools. Recommendations for sustaining strong collaboration among researchers and school partners are provided.

Background: There is a growing interest in practice-based implementation research, yet too often research prioritizes and is most successful in academic settings. During a national implementation trial to evaluate the effectiveness of Collaborative Care for co-occurring opioid use and mental health disorders, we lost three of our 11 participating implementation sites, all representing community sites.

Method: To better understand needed supports for implementation trial participation, we conducted exit interviews (n = 5) with key staff at these community sites. Interview transcripts were double-coded and analyzed using Rapid Assessment Process. Qualitative themes were iteratively reviewed by the study team.

Results: Three themes emerged characterizing challenges for community sites, including that: (1) research threatens sites' most precious resource-staff; (2) staff lack comfort with and skills for research; and (3) research participation in its current form does not offer a clear return on investment.

Conclusions: Learnings from this work illuminate some of the barriers community sites face when trying to participate in multisite implementation research. An undercurrent of participant perspectives was the belief that community sites like theirs are just not set up to successfully participate in clinical trial research, including population-based implementation trials. Future implementation trials should consider strategies that disrupt traditional approaches, increasing the equitable inclusion of diverse practice settings in implementation research.

Preventing and treating mental health and substance use problems requires effective, affordable, scalable, and efficient interventions. The multiphase optimization strategy (MOST) framework guides researchers through a phased and systematic process of developing optimized interventions. However, new methods of systematically incorporating information about implementation constraints across MOST phases are needed. We propose that early and sustained integration of community-engaged methods within MOST is a promising strategy for enhancing an optimized intervention's potential for implementation. In this article, we outline the advantages of using community-engaged methods throughout the intervention optimization process, with a focus on the Preparation and Optimization Phases of MOST. We discuss the role of experimental designs in optimization research and highlight potential challenges in conducting rigorous experiments in community settings. We then demonstrate how relying on the resource management principle to select experimental designs across MOST phases is a promising strategy for maintaining both experimental rigor and community responsiveness. We end with an applied example illustrating a community-engaged approach to optimize an intervention to reduce the risk for mental health problems and substance use problems among children with incarcerated parents.

Background: This study examined the implementation determinants of a culturally grounded, school-based drug prevention curriculum in rural Hawai'i. Test development and validation procedures were used to examine the impact of implementation barriers and facilitators of the curriculum in public or charter middle/intermediate schools on Hawai'i Island.

Method: A five-phase, mixed-methods approach toward test development and validation was used. These phases included item generation (Phase 1), item refinement and selection (Phase 2), item reduction (Phase 3), reliability testing (Phase 4), and validity testing (Phase 5). Educational administrators, teachers, and staff employed by the Hawai'i State Department of Education (HIDOE) participated in the study.

Results: Phases 1 and 2 yielded 50 implementation barriers and 27 implementation facilitators that were evaluated by 204 HIDOE administrators, teachers, and staff. Factor analysis of the barrier items indicated a four-factor solution: (1) Innovation Barriers, (2) HIDOE State-Level Barriers, (3) Teacher-Level Barriers, and (4) Administrator-Level Barriers. Mean comparisons indicated that several barrier and facilitator items differentiated teachers from administrators in the sample.

Conclusions: This study contributes to the implementation measurement literature, specifically in the areas of mental health and substance use. It also highlights the importance of addressing multiple contextual levels in the implementation of culturally focused prevention interventions.

Plain language summary title: Examining Implementation Barriers and Facilitators for School-Based Prevention in Hawai'i. Plain Language Summary Compared with other major ethnic groups, Native Hawaiian and Pacific Islander (NHPI) youth have substantially higher rates of substance use and higher adverse mental and physical health effects related to their use. Despite these disparities, educational and community-based practitioners have long struggled with implementing and sustaining promising substance use interventions for NHPI youth. This study contributes to our understanding of measuring implementation barriers and facilitators for substance use prevention for NHPI youth in rural school settings. It contributes to the field of health disparities and health equity promotion, by addressing calls for research to understand factors affecting successful implementation of prevention programs. This is critical toward achieving health equity for underrepresented and vulnerable populations, such as NHPI and rural youth.

Background: Sustained delivery of evidence-based treatments (EBTs) is essential to addressing the public health impacts of youth mental health problems, but is complicated by the limited and fragmented funding available to youth mental health service agencies. Supports are needed that can guide service agencies in accessing sustainable funding for EBTs. We conducted a pilot evaluation of the Fiscal Mapping Process, an Excel-based strategic planning tool that helps service agency leaders identify and coordinate financing strategies for their EBT programs.

Method: Pilot testing of the Fiscal Mapping Process was completed with 10 youth mental health service agencies over a 12-month period, using trauma-focused cognitive-behavioral therapy or parent-child interaction therapy programs. Service agency representatives received initial training and monthly coaching in using the tool. We used case study methods to synthesize all available data (surveys, focus groups, coaching notes, document review) and contrast agency experiences to identify key findings through explanation building.

Results: Key evaluation findings related to the process and outcomes of using the Fiscal Mapping Process, as well as contextual influences. Process evaluation findings helped clarify the primary use case for the tool and identified the importance-and challenges-of engaging external collaborators. Outcome evaluation findings documented the impacts of the Fiscal Mapping Process on agency-reported sustainment capacities (strategic planning, funding stability), which fully explained reported improvements in outcomes (extent and likelihood)-although these impacts were incremental. Findings on contextual factors documented the influence of environmental and organizational capacities on engagement with the tool and concerns about equitable impacts, but also the view that the process could usefully generalize to other EBTs.

Conclusions: Our pilot evaluation of the Fiscal Mapping Process was promising. In future work, we plan to integrate the tool into EBT implementation initiatives and test its impact on long-term sustainment outcomes across various EBTs, while increasing attention to equity considerations.

Background: Implementation strategies are theorized to work well when carefully matched to implementation determinants and when factors-preconditions, moderators, etc.-that influence strategy effectiveness are prospectively identified and addressed. Existing methods for strategy selection are either imprecise or require significant technical expertise and resources, undermining their utility. This article outlines refinements to causal pathway diagrams (CPDs), a method for articulating the causal process through which implementation strategies work and offers illustrations of their use.

Method: CPDs are a visualization tool to represent an implementation strategy, its mechanism(s) (i.e., the processes through which a strategy is thought to operate), determinants it is intended to address, factors that may impede or facilitate its effectiveness, and the series of outcomes that should be expected if the strategy is operating as intended. We offer principles for constructing CPDs and describe their key functions.

Results: Applications of the CPD method by study teams from two National Institute of Health-funded Implementation Science Centers and a research grant are presented. These include the use of CPDs to (a) match implementation strategies to determinants, (b) understand the conditions under which an implementation strategy works, and (c) develop causal theories of implementation strategies.

Conclusions: CPDs offer a novel method for implementers to select, understand, and improve the effectiveness of implementation strategies. They make explicit theoretical assumptions about strategy operation while supporting practical planning. Early applications have led to method refinements and guidance for the field.

Background: Effective interventions need to be implemented successfully to achieve impact. Two theory-based measures exist for measuring the effectiveness of implementation strategies and monitor implementation progress. The Normalization MeAsure Development questionnaire (NoMAD) explores the four core concepts (Coherence, Cognitive Participation, Collective Action, Reflexive Monitoring) of the Normalization Process Theory. The Organizational Readiness for Implementing Change (ORIC) is based on the theory of Organizational Readiness for Change, measuring organization members' psychological and behavioral preparedness for implementing a change. We examined the measurement properties of the NoMAD and ORIC in a multi-national implementation effectiveness study.

Method: Twelve mental health organizations in nine countries implemented Internet-based cognitive behavioral therapy (iCBT) for common mental disorders. Staff involved in iCBT service delivery (n = 318) participated in the study. Both measures were translated into eight languages using a standardized forward-backward translation procedure. Correlations between measures and subscales were estimated to examine convergent validity. The theoretical factor structures of the scales were tested using confirmatory factor analysis (CFA). Test-retest reliability was based on the correlation between scores at two time points 3 months apart. Internal consistency was assessed using Cronbach's alpha. Floor and ceiling effects were quantified using the proportion of zero and maximum scores.

Results: NoMAD and ORIC measure related but distinct latent constructs. The CFA showed that the use of a total score for each measure is appropriate. The theoretical subscales of the NoMAD had adequate internal consistency. The total scale had high internal consistency. The total ORIC scale and subscales demonstrated high internal consistency. Test-retest reliability was suboptimal for both measures and floor and ceiling effects were absent.

Conclusions: This study confirmed the psychometric properties of the NoMAD and ORIC in multi-national mental health care settings. While measuring on different but related aspects of implementation processes, the NoMAD and ORIC prove to be valid and reliable across different language settings.

Background: Schools are critical venues for supporting LGBTQ+ youth well-being. Implementing LGBTQ-supportive practices can decrease experiences of stigmatization, discrimination, and victimization that lead to adverse mental health outcomes like anxiety, depression, and suicidality. However, schools are also subject to a wide range of outer-context pressures that may influence their priorities and implementation of LGBTQ-supportive practices. We assessed the role of emergent outer-context determinants in the context of a 5-year cluster randomized controlled trial to study the implementation of LGBTQ-supportive evidence-informed practices (EIPs) in New Mexico high schools.

Method: Using an iterative coding approach, we analyzed qualitative data from annual interviews with school professionals involved in EIP implementation efforts.

Results: The analysis yielded three categories of outer-context determinants that created challenges and opportunities for implementation: (a) social barriers related to heterocentrism, cisgenderism, and religious conservatism; (b) local, state, and national policy and political discourse; and (c) crisis events.

Conclusions: By exploring the implications of outer-context determinants for the uptake of LGBTQ-supportive practices, we demonstrate that these elements are dynamic-not simply reducible to barriers or facilitators-and that assessing outer-context determinants shaping implementation environments is crucial for addressing LGBTQ health equity.

Background: Rising rates of adolescent depression in the wake of COVID-19 and a youth mental health crisis highlight the urgent need for accessible mental healthcare and prevention within primary care. Digital mental health interventions (DMHIs) may increase access for underserved populations. However, these interventions are not well studied in adolescents, nor healthcare settings. The purpose of this study was to identify barriers and facilitators to screening and recruitment activities for PATH 2 Purpose (P2P): Primary Care and Community-Based Prevention of Mental Disorders in Adolescents, a multi-site adolescent depression prevention trial comparing two digital prevention programs within four diverse health systems in two U.S. states.

Method: This qualitative study is a component of a larger Hybrid Type I trial. We conducted semi-structured key informant interviews with clinical and non-clinical implementers involved with screening and recruitment for the P2P trial. Informed by the Consolidated Framework for Implementation Research (CFIR), interviews were conducted at the midpoint of the trial to identify barriers, facilitators, and needed adaptations, and to gather information on determinants that may affect future implementation.

Findings: Respondents perceived the P2P trial as valuable, well aligned with the mission of their health systems. However, several barriers were identified, many of which stemmed from influences outside of the healthcare settings. Universal and site-specific outer setting influences (COVID-19 pandemic, youth mental health crisis, local community conditions) interacted with Inner Setting and Innovation domains to create numerous challenges to the implementation of screening and recruitment.

Conclusion: Our findings emphasize the need for ongoing, comprehensive assessment of dynamic inner and outer setting contexts prior to and during implementation of clinical trials, as well as flexibility for adaptation to unique clinical contexts. The CFIR is useful for assessing determinants during times of rapid inner and outer setting change, such as those brought on by the COVID-19 pandemic, youth mental health crisis, and the corresponding exacerbation of resource strain within healthcare settings.

Clinical trial registration: PATH 2 Purpose: Primary Care and Community-Based Prevention of Mental Disorders in Adolescents https://www.clinicaltrials.gov/study/NCT04290754.