Sex, ethnicity, and race data are often unreported in artificial intelligence and machine learning studies in medicine

Abstract

The use of artificial intelligence (AI) programs in healthcare and medicine has steadily increased over the past decade. One major challenge affecting the use of AI programs is that the results of AI programs are sometimes not replicable, meaning that the performance of the AI program is substantially different in the external testing dataset when compared to its performance in the training or validation datasets. This often happens when the external testing dataset is very different from the training or validation datasets. Sex, ethnicity, and race are some of the most important biological and social determinants of health, and are important factors that may differ between training, validation, and external testing datasets, and may contribute to the lack of reproducibility of AI programs. We reviewed over 28,000 original research articles published in the three journals with the highest impact factors in each of 16 medical specialties between 2019 and 2022, to evaluate how often the sex, ethnic, and racial compositions of the datasets used to develop AI algorithms were reported. We also reviewed all currently used AI reporting guidelines, to evaluate which guidelines recommend specific reporting of sex, ethnicity, and race. We find that only 42.47 % (338/797) of articles reported sex, 1.4 % (12/831) reported ethnicity, and 7.3 % (61/831) reported race. When sex was reported, approximately 55.8 % of the study participants were female, and when ethnicity was reported, only 6.2 % of the study participants were Hispanic/Latino. When race was reported, only 29.4 % of study participants were non-White. Most AI guidelines (93.3 %; 14/15) also did not recommend reporting sex, ethnicity, and race. To have fair and ethnical AI, it is important that the sex, ethnic, and racial compositions of the datasets used to develop the AI program are known.