Joseph E Alderman MB ChB , Maria Charalambides MB ChB , Gagandeep Sachdeva MB ChB , Elinor Laws MB BCh , Joanne Palmer PhD , Elsa Lee MSc , Vaishnavi Menon MB ChB , Qasim Malik MB ChB , Sonam Vadera MB BS , Prof Melanie Calvert PhD , Marzyeh Ghassemi PhD , Melissa D McCradden PhD , Johan Ordish MA , Bilal Mateen MBBS , Prof Charlotte Summers PhD , Jacqui Gath , Rubeta N Matin PhD , Prof Alastair K Denniston PhD , Xiaoxuan Liu PhD
{"title":"Revealing transparency gaps in publicly available COVID-19 datasets used for medical artificial intelligence development—a systematic review","authors":"Joseph E Alderman MB ChB , Maria Charalambides MB ChB , Gagandeep Sachdeva MB ChB , Elinor Laws MB BCh , Joanne Palmer PhD , Elsa Lee MSc , Vaishnavi Menon MB ChB , Qasim Malik MB ChB , Sonam Vadera MB BS , Prof Melanie Calvert PhD , Marzyeh Ghassemi PhD , Melissa D McCradden PhD , Johan Ordish MA , Bilal Mateen MBBS , Prof Charlotte Summers PhD , Jacqui Gath , Rubeta N Matin PhD , Prof Alastair K Denniston PhD , Xiaoxuan Liu PhD","doi":"10.1016/S2589-7500(24)00146-8","DOIUrl":null,"url":null,"abstract":"<div><div>During the COVID-19 pandemic, artificial intelligence (AI) models were created to address health-care resource constraints. Previous research shows that health-care datasets often have limitations, leading to biased AI technologies. This systematic review assessed datasets used for AI development during the pandemic, identifying several deficiencies. Datasets were identified by screening articles from MEDLINE and using Google Dataset Search. 192 datasets were analysed for metadata completeness, composition, data accessibility, and ethical considerations. Findings revealed substantial gaps: only 48% of datasets documented individuals’ country of origin, 43% reported age, and under 25% included sex, gender, race, or ethnicity. Information on data labelling, ethical review, or consent was frequently missing. Many datasets reused data with inadequate traceability. Notably, historical paediatric chest x-rays appeared in some datasets without acknowledgment. These deficiencies highlight the need for better data quality and transparent documentation to lessen the risk that biased AI models are developed in future health emergencies.</div></div>","PeriodicalId":48534,"journal":{"name":"Lancet Digital Health","volume":"6 11","pages":"Pages e827-e847"},"PeriodicalIF":23.8000,"publicationDate":"2024-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Lancet Digital Health","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2589750024001468","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MEDICAL INFORMATICS","Score":null,"Total":0}
引用次数: 0
Abstract
During the COVID-19 pandemic, artificial intelligence (AI) models were created to address health-care resource constraints. Previous research shows that health-care datasets often have limitations, leading to biased AI technologies. This systematic review assessed datasets used for AI development during the pandemic, identifying several deficiencies. Datasets were identified by screening articles from MEDLINE and using Google Dataset Search. 192 datasets were analysed for metadata completeness, composition, data accessibility, and ethical considerations. Findings revealed substantial gaps: only 48% of datasets documented individuals’ country of origin, 43% reported age, and under 25% included sex, gender, race, or ethnicity. Information on data labelling, ethical review, or consent was frequently missing. Many datasets reused data with inadequate traceability. Notably, historical paediatric chest x-rays appeared in some datasets without acknowledgment. These deficiencies highlight the need for better data quality and transparent documentation to lessen the risk that biased AI models are developed in future health emergencies.
期刊介绍:
The Lancet Digital Health publishes important, innovative, and practice-changing research on any topic connected with digital technology in clinical medicine, public health, and global health.
The journal’s open access content crosses subject boundaries, building bridges between health professionals and researchers.By bringing together the most important advances in this multidisciplinary field,The Lancet Digital Health is the most prominent publishing venue in digital health.
We publish a range of content types including Articles,Review, Comment, and Correspondence, contributing to promoting digital technologies in health practice worldwide.