Development and usability testing of an online support tool to identify models and frameworks to inform implementation.

Abstract

Background: Theories, models and frameworks (TMFs) are useful when implementing, evaluating and sustaining healthcare evidence-based interventions. Yet it can be challenging to identify an appropriate TMF for an implementation project. We developed and tested the usability of an online tool to help individuals who are doing or supporting implementation practice activities to identify appropriate models and/or frameworks to inform their work.

Methods: We used methods guided by models and evidence on implementation science and user-centered design. Phases of tool development included applying findings from a scoping review of TMFs and interviews with 24 researchers/implementers on barriers and facilitators to identifying and selecting TMFs. Based on interview findings, we categorized the TMFs by aim, stage of implementation, and target level of change to inform the tool's algorithm. We then conducted interviews with 10 end-users to test the usability of the prototype tool and administered the System Usability Scale (SUS). Usability issues were addressed and incorporated into the tool.

Results: We developed Find TMF, an online tool consisting of 3-4 questions about the user's implementation project. The tool's algorithm matches key characteristics of the user's project (aim, stage, target change level) with characteristics of different TMFs and presents a list of candidate models/frameworks. Ten individuals from Canada or Australia participated in usability testing (mean SUS score 84.5, standard deviation 11.4). Overall, participants found the tool to be simple, easy to use and visually appealing with a useful output of candidate models/frameworks to consider for an implementation project. Users wanted additional instruction and guidance on what to expect from the tool and how to use the information in the output table. Tool improvements included incorporating an overview figure outlining the tool steps and output, displaying the tool questions on a single page, and clarifying the available functions of the results page, including adding direct links to the glossary and to complementary tools.

Conclusions: Find TMF is an easy-to-use online tool that may benefit individuals who support implementation practice activities by making the vast number of models and frameworks more accessible, while also supporting a consistent approach to identifying and selecting relevant TMFs.