Looking under the hood of a hybrid two-way texting intervention to improve early retention on antiretroviral therapy in Malawi: an implementation fidelity evaluation.

Abstract

Background: While key to interpreting findings and assessing generalizability, implementation fidelity is underreported in mobile health (mHealth) literature. We evaluated implementation fidelity of an opt-in, hybrid, two-way texting (2wT) intervention previously demonstrated to improve 12-month retention on antiretroviral therapy (ART) among people living with HIV (PLHIV) in a quasi-experimental study in Lilongwe, Malawi.

Methods: Short message service (SMS) data and ART refill visit records were used to evaluate adherence to 2wT content, frequency and duration through the lens of the Conceptual Framework for Implementation Fidelity. Message delivery and 2wT participant interactions were considered across four core 2wT components: 1) weekly motivational SMS messages; 2) proactive SMS appointment reminders; 3) SMS reminders after missed appointments; and 4) interactive messaging with 2wT staff about transfers and appointment rescheduling. Using mixed-effects logistic regression models adjusted for participant demographics, we examined the effect of core 2wT component fidelity on a) on-time appointment attendance and b) timely return to care after a missed appointment, presenting adjusted odds ratios (aORs) and 95% confidence intervals (CIs).

Results: The 468 2wT participants had a median of 52 study weeks (interquartile range [IQR] 34 - 52) with 6 ART appointments (IQR 4-7) of which 2 (IQR 1 - 3) were missed. On average, participants received a motivation message for 75% (IQR 56%-83%) of enrolled weeks, a reminder before 83% (IQR 67%-100%) of appointments, and after 67% (IQR 0%-100%) of missed appointments. Participants reported 9 transfers and rescheduled 46 appointments through 2wT prompts; 196 appointments were changed via unprompted interaction. Participants with 10% higher expected motivation message delivery were more likely to attend clinic appointments on time (aOR: 1.08; 95%CI: 1.01 - 1.16, p = 0.03). Receiving and responding to an appointment reminder in any way were also associated with increased on-time appointment attendance (aOR: 1.35; 95%CI: 1.03 - 1.79, p = 0.03 and aOR: 1.47, 95%CI: 1.16 - 1.87, p = 0.001, respectively). No associations were found for 2wT messages and timely return to care following a missed appointment.

Conclusion: Greater 2wT implementation fidelity was associated with improved care outcomes. Although implementation fidelity monitoring of mHealth interventions is complex, it should be integrated into study design.