Anticipatory moral distress in machine learning-based clinical decision support tool development: A qualitative analysis

Abstract

Ongoing interest in machine learning systems include the emerging capability to integrate electronic health records to develop clinical decision support (CDS) tools that improve medical care, diagnostics, and therapy. Such CDS tools, which can handle a large quantity of data sources, can advise clinicians and amplify insights on diverse patient risk factors, from physiological challenges to psychosocial vulnerabilities. Despite a growing interest, there are various challenges that hinder the successful use of CDS tools in clinical practice. Among these, a key challenge is hesitance or resistance among end-users to take up tools and integrate their use into practice. The current inquiry applied a framework of the symbolic interaction of participatory experience-based co-design and used an interpretive descriptive approach to analysis of qualitative data, investigating the ethical issues brought to light by clinicians participating in three participatory experience-based co-design focus groups, as a part of the initial development of a CDS tool for detecting risk factors for adverse health outcomes in outpatient obstetric care at a single academically affiliated medical institution. Findings revealed that participants describe their anticipated symbolic relationship with a ML-based CDS tool as either promising or morally distressing. Anticipatory moral distress includes three separate sub-categories: 1) clinical conflict with clinical assessment and judgment, 2) partial conflict with comprehensive clinical considerations, and 3) resource conflict with structural barriers related to care delivery. Future work should include utilizing participatory experience-based co-design with end users to identify relevant context and institution-specific priorities and concerns from the beginning of CDS tool development and to continue co-design throughout the development process.