An ethics assessment tool for artificial intelligence implementation in healthcare: CARE-AI

The deployment of artificial intelligence (AI)-powered prediction models in healthcare can lead to ethical concerns about their implementation and upscaling. For example, AI prediction models can hinder clinical decision-making if they advise different diagnoses or treatments by sex and gender or by race and ethnicity without clear justification. Recent guidance (such as the WHO guidance on ethics and governance of AI for health and the Dutch guideline on AI for healthcare) and legislation (such as the European Union AI Act and the White House Executive Order on Safe, Secure, and Trustworthy Development and Use of AI in United States) have outlined important principles for the implementation of AI, including ethical considerations^1,2. Health systems have responded by establishing governance committees and processes to ensure the safe and equitable implementation of AI tools³. However, there is currently no assessment tool that can identify and mitigate ethical issues during the implementation of AI prediction models in healthcare practice, including for public health.

The development and validation of AI prediction models has benefited from detailed reporting and risk-of-bias tools, such as TRIPOD+AI⁴ and PROBAST (with its forthcoming AI extension) for fairness and bias control and CLAIM⁵ for data privacy, security and interpretability of AI imaging studies. However, when planning the implementation of a rigorously developed and well-performing AI prediction model in healthcare practice, existing recommendations and guidance on ethics are sparse and lack operational detail. For example, the DECIDE-AI reporting guideline⁶ contains a small number of ethics-related recommendations for early clinical evaluation of AI concerning equity, safety and human-AI interaction, and FUTURE-AI⁷ provides recommendations based on six principles (fairness, universality, traceability, usability, robustness and explainability) in model design, development, validation and deployment. A bioethics-centric delivery science toolkit for responsible AI implementation in healthcare is needed⁸.