描述并验证用于识别 FDA 标签中不良事件变更的新型人工智能工具 LabelComp。

IF 4 2区 医学 Q1 PHARMACOLOGY & PHARMACY Drug Safety Pub Date : 2024-12-01 Epub Date: 2024-07-31 DOI:10.1007/s40264-024-01468-8
George A Neyarapally, Leihong Wu, Joshua Xu, Esther H Zhou, Oanh Dang, Joann Lee, Dharmang Mehta, Rochelle D Vaughn, Ellen Pinnow, Hong Fang
{"title":"描述并验证用于识别 FDA 标签中不良事件变更的新型人工智能工具 LabelComp。","authors":"George A Neyarapally, Leihong Wu, Joshua Xu, Esther H Zhou, Oanh Dang, Joann Lee, Dharmang Mehta, Rochelle D Vaughn, Ellen Pinnow, Hong Fang","doi":"10.1007/s40264-024-01468-8","DOIUrl":null,"url":null,"abstract":"<p><strong>Introduction: </strong>The accurate identification and timely updating of adverse reactions in drug labeling are crucial for patient safety and effective drug use. Postmarketing surveillance plays a pivotal role in identifying previously undetected adverse events (AEs) that emerge when a drug is used in broader and more diverse patient populations. However, traditional methods of updating drug labeling with new AE information have been manual, time consuming, and error prone. This paper introduces the LabelComp tool, an innovative artificial intelligence (AI) tool designed to enhance the efficiency and accuracy of postmarketing drug safety surveillance. Utilizing a combination of text analytics and a trained Bidirectional Encoder Representations from Transformers (BERT) model, the LabelComp tool automatically identifies changes in AE terms from updated drug labeling documents.</p><p><strong>Objective: </strong>Our objective was to create and validate an AI tool with high accuracy that could enable researchers and FDA reviewers to efficiently identify safety-related drug labeling changes.</p><p><strong>Results: </strong>Our validation study of 87 drug labeling PDF pairs demonstrates the tool's high accuracy, with F1 scores of overall performance ranging from 0.795 to 0.936 across different evaluation tiers and a recall of at least 0.997 with only one missed AE out of 483 total AEs detected, indicating the tool's efficacy in identifying new AEs.</p><p><strong>Conclusion: </strong>The LabelComp tool can support drug safety surveillance and inform regulatory decision-making. The publication of this tool also aims to encourage further community-driven enhancements, aligning with broader interests in applying AI to advance regulatory science and public health.</p>","PeriodicalId":11382,"journal":{"name":"Drug Safety","volume":null,"pages":null},"PeriodicalIF":4.0000,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Description and Validation of a Novel AI Tool, LabelComp, for the Identification of Adverse Event Changes in FDA Labeling.\",\"authors\":\"George A Neyarapally, Leihong Wu, Joshua Xu, Esther H Zhou, Oanh Dang, Joann Lee, Dharmang Mehta, Rochelle D Vaughn, Ellen Pinnow, Hong Fang\",\"doi\":\"10.1007/s40264-024-01468-8\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Introduction: </strong>The accurate identification and timely updating of adverse reactions in drug labeling are crucial for patient safety and effective drug use. Postmarketing surveillance plays a pivotal role in identifying previously undetected adverse events (AEs) that emerge when a drug is used in broader and more diverse patient populations. However, traditional methods of updating drug labeling with new AE information have been manual, time consuming, and error prone. This paper introduces the LabelComp tool, an innovative artificial intelligence (AI) tool designed to enhance the efficiency and accuracy of postmarketing drug safety surveillance. Utilizing a combination of text analytics and a trained Bidirectional Encoder Representations from Transformers (BERT) model, the LabelComp tool automatically identifies changes in AE terms from updated drug labeling documents.</p><p><strong>Objective: </strong>Our objective was to create and validate an AI tool with high accuracy that could enable researchers and FDA reviewers to efficiently identify safety-related drug labeling changes.</p><p><strong>Results: </strong>Our validation study of 87 drug labeling PDF pairs demonstrates the tool's high accuracy, with F1 scores of overall performance ranging from 0.795 to 0.936 across different evaluation tiers and a recall of at least 0.997 with only one missed AE out of 483 total AEs detected, indicating the tool's efficacy in identifying new AEs.</p><p><strong>Conclusion: </strong>The LabelComp tool can support drug safety surveillance and inform regulatory decision-making. The publication of this tool also aims to encourage further community-driven enhancements, aligning with broader interests in applying AI to advance regulatory science and public health.</p>\",\"PeriodicalId\":11382,\"journal\":{\"name\":\"Drug Safety\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":4.0000,\"publicationDate\":\"2024-12-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Drug Safety\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1007/s40264-024-01468-8\",\"RegionNum\":2,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"2024/7/31 0:00:00\",\"PubModel\":\"Epub\",\"JCR\":\"Q1\",\"JCRName\":\"PHARMACOLOGY & PHARMACY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Drug Safety","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1007/s40264-024-01468-8","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/7/31 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"PHARMACOLOGY & PHARMACY","Score":null,"Total":0}
引用次数: 0

摘要

导言:准确识别和及时更新药品标签中的不良反应对患者安全和有效用药至关重要。上市后监测在识别以前未被发现的不良事件(AEs)方面发挥着关键作用,这些不良事件是药物在更广泛、更多样化的患者群体中使用时出现的。然而,用新的 AE 信息更新药品标签的传统方法都是手动操作,既费时又容易出错。本文介绍了 LabelComp 工具,这是一种创新的人工智能 (AI) 工具,旨在提高上市后药品安全监测的效率和准确性。LabelComp 工具将文本分析与训练有素的变压器双向编码器表征 (BERT) 模型相结合,自动识别更新的药品标签文件中 AE 术语的变化:我们的目标是创建并验证一种高准确性的人工智能工具,使研究人员和 FDA 评审人员能够高效地识别与安全性相关的药品标签变更:我们对 87 个药品标签 PDF 对进行了验证研究,结果表明该工具具有很高的准确性,在不同的评估层级中,总体性能的 F1 分数从 0.795 到 0.936 不等,召回率至少为 0.997,在检测到的 483 个 AE 中只有一个遗漏 AE,这表明该工具在识别新的 AE 方面具有很高的效率:LabelComp 工具可为药物安全性监测提供支持,并为监管决策提供信息。该工具的发布还旨在鼓励进一步的社区驱动改进,与应用人工智能促进监管科学和公共卫生的更广泛兴趣相一致。
本文章由计算机程序翻译,如有差异,请以英文原文为准。

摘要图片

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Description and Validation of a Novel AI Tool, LabelComp, for the Identification of Adverse Event Changes in FDA Labeling.

Introduction: The accurate identification and timely updating of adverse reactions in drug labeling are crucial for patient safety and effective drug use. Postmarketing surveillance plays a pivotal role in identifying previously undetected adverse events (AEs) that emerge when a drug is used in broader and more diverse patient populations. However, traditional methods of updating drug labeling with new AE information have been manual, time consuming, and error prone. This paper introduces the LabelComp tool, an innovative artificial intelligence (AI) tool designed to enhance the efficiency and accuracy of postmarketing drug safety surveillance. Utilizing a combination of text analytics and a trained Bidirectional Encoder Representations from Transformers (BERT) model, the LabelComp tool automatically identifies changes in AE terms from updated drug labeling documents.

Objective: Our objective was to create and validate an AI tool with high accuracy that could enable researchers and FDA reviewers to efficiently identify safety-related drug labeling changes.

Results: Our validation study of 87 drug labeling PDF pairs demonstrates the tool's high accuracy, with F1 scores of overall performance ranging from 0.795 to 0.936 across different evaluation tiers and a recall of at least 0.997 with only one missed AE out of 483 total AEs detected, indicating the tool's efficacy in identifying new AEs.

Conclusion: The LabelComp tool can support drug safety surveillance and inform regulatory decision-making. The publication of this tool also aims to encourage further community-driven enhancements, aligning with broader interests in applying AI to advance regulatory science and public health.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Drug Safety
Drug Safety 医学-毒理学
CiteScore
7.60
自引率
7.10%
发文量
112
审稿时长
6-12 weeks
期刊介绍: Drug Safety is the official journal of the International Society of Pharmacovigilance. The journal includes: Overviews of contentious or emerging issues. Comprehensive narrative reviews that provide an authoritative source of information on epidemiology, clinical features, prevention and management of adverse effects of individual drugs and drug classes. In-depth benefit-risk assessment of adverse effect and efficacy data for a drug in a defined therapeutic area. Systematic reviews (with or without meta-analyses) that collate empirical evidence to answer a specific research question, using explicit, systematic methods as outlined by the PRISMA statement. Original research articles reporting the results of well-designed studies in disciplines such as pharmacoepidemiology, pharmacovigilance, pharmacology and toxicology, and pharmacogenomics. Editorials and commentaries on topical issues. Additional digital features (including animated abstracts, video abstracts, slide decks, audio slides, instructional videos, infographics, podcasts and animations) can be published with articles; these are designed to increase the visibility, readership and educational value of the journal’s content. In addition, articles published in Drug Safety Drugs may be accompanied by plain language summaries to assist readers who have some knowledge of, but not in-depth expertise in, the area to understand important medical advances.
期刊最新文献
RETRACTED ARTICLE: Long-Term Safety Analysis of the BBV152 Coronavirus Vaccine in Adolescents and Adults: Findings from a 1-Year Prospective Study in North India. A Calculated Risk: Evaluation of QTc Drug-Drug Interaction (DDI) Clinical Decision Support (CDS) Alerts and Performance of the Tisdale Risk Score Calculator. Description and Validation of a Novel AI Tool, LabelComp, for the Identification of Adverse Event Changes in FDA Labeling. Examining the Effect of Missing Data and Unmeasured Confounding on External Comparator Studies: Case Studies and Simulations. Unveiling the Burden of Drug-Induced Impulsivity: A Network Analysis of the FDA Adverse Event Reporting System.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1