Human-centered AI as a framework guiding the development of image-based diagnostic tools in oncology: a systematic review

Background

Artificial intelligence diagnostic tools (AIDTs) in oncology show high image classification accuracy but limited clinical adoption. Their adoption could be enhanced by (i) using user feedback during the software design, (ii) demonstrating that AIDTs improve the user’s decisions, and (iii) providing explanations of AI decisions tailored to the user, three aspects central to human-centered AI (HCAI). This review assesses these three aspects in AIDTs for oncology in general, exemplifying its concepts in the established field of skin cancer diagnostics as a specific use case.

Materials and methods

We carried out three Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) searches using PubMed and ScienceDirect, limiting the results to articles published from 2019 to 2024. The first search focused on articles that used user feedback to develop AIDTs. The second search addressed whether AIDT improves dermatologists’ decisions. The third search targeted explainable AI in skin cancer.

Results

Five studies incorporated user feedback in AIDT design for cancer. Zooming in on AIDT for skin cancer, nine studies (3/37 in 2019, 3/93 in 2023) indicated that AIDTs improve dermatologists’ decisions in experimental (n = 5) and clinical settings (n = 1). Explainable AI was common in skin cancer diagnostics (n = 26), with papers assessing the user’s preference for explainable AI (XAI) methods or the impact of XAI on the user’s trust in AI diagnosis.

Conclusions

User feedback has been used to develop AIDTs tailored to clinicians’ needs. Evidence shows that AIDTs can improve clinicians’ decisions. This, combined with XAI, increases clinicians’ trust in AIDTs, potentially favoring their widespread usage.