Development of a Human Factors–Based Guideline to Support the Design, Evaluation, and Continuous Improvement of Clinical Decision Support

Abstract

Objective

To develop a vendor-agnostic, human factors (HF)-based guideline to guide the design, evaluation, and continuous improvement of clinical decision support (CDS).

Participants and Methods

The study used a 2-phased iterative approach between June 2022 and June 2024. Phase 1 involved a search for relevant industry standards and literature and consultation with multidisciplinary subject matter experts. Phase 2 involved a workshop with 30 health care and academic stakeholders to evaluate face validity and perceived usefulness of the initial section of the guideline. Participants were asked if the guideline met their expectations, to report on usefulness and ease of use and to suggest areas for improvement.

Results

Phase 1 resulted in a compilation of accessible, best practice, and context-appropriate HF guidance for CDS design and optimization. The guideline supports users in determining whether use of CDS is appropriate, and if yes, CDS options and design guidance. During phase 2, the guideline addressed 15 of participants’ 19 expectations for a CDS guideline. Participants said the guideline was helpful, comprehensive, easy to use, and provided step-by-step guidance, boundaries, and transparency around CDS decisions. Participants recommended strengthening guidance around the need to understand system capabilities and the technical burden or complexity of CDS, and further guidance on how to approach CDS optimization using the guideline.

Conclusion

The 2-phased iterative development and feedback process resulted in the development of an HF-informed guideline to provide consolidated, accessible, and current best practice guidance on the appropriateness of CDS and CDS options, as well as designing, evaluating, and continuously improving CDS. Future work will evaluate the impact and implementation of the guideline in real-world settings.