Validating a Pragmatic Measure of Evidence-Based Practice (EBP) Delivery: Therapist Reports of EBP Strategy Delivery and Associations with Child Outcome Trajectories

Abstract

Pragmatic measures of evidence-based practice (EBP) implementation can support and evaluate implementation efforts. We examined the predictive validity of therapist reports of EBP strategy delivery for children’s mental health outcomes. Data were obtained from 1,380 sessions with 248 children delivered by 76 therapists in two county systems. Children (M_age=11.8 years, SD = 3.7) presented with internalizing (52%), externalizing (27%), trauma (16%), and other (5%) concerns. Therapists reported their delivery of EBP strategies on a revised version of the EBP Concordant Care Assessment (ECCA; Brookman-Frazee, et al., Administration and Policy in Mental Health and Mental Health Services Research, 48, 155–170, 2021) that included 25 content (e.g., parenting, cognitive behavioral) and 12 technique strategies (e.g., modeling, practice/role-play). On average, 5.6 ECCA session reports (SD = 2.3) were obtained for each client, and caregivers reported symptoms on the Brief Problem Checklist (Chorpita, et al., Journal of Consulting and Clinical Psychology, 78(4), 526–536, 2010) at baseline, weekly over two months, and again at four months. Multilevel models examined whether the mean extensiveness of each EBP strategy predicted trajectories of child outcomes. More individual technique (6 of 12) than content strategies (1 of 25) were associated with outcome trajectories. For techniques, more extensive use of Performance Feedback and Live Coaching and less extensive use of Addressing Barriers were associated with greater declines in total symptoms, and more extensive use of Establishing/Reviewing Goals, Tracking/Reviewing Progress, and Assigning/Reviewing Homework was associated with declines in externalizing symptoms. For content, more extensive use of Cognitive Restructuring was associated with declines in total symptoms. In addition, higher average extensiveness ratings of the top content strategy across sessions was associated with greater declines in total and externalizing symptoms. Therapist-reported delivery of some EBP strategies showed evidence of predictive validity and may hold utility in indexing quality of care.