{"title":"MELEP:多标记心电图诊断中可转移性的新型预测量度。","authors":"Cuong V Nguyen, Hieu Minh Duong, Cuong D Do","doi":"10.1007/s41666-024-00168-3","DOIUrl":null,"url":null,"abstract":"<p><p>In practical electrocardiography (ECG) interpretation, the scarcity of well-annotated data is a common challenge. Transfer learning techniques are valuable in such situations, yet the assessment of transferability has received limited attention. To tackle this issue, we introduce MELEP, which stands for <i>Muti-label Expected Log of Empirical Predictions</i>, a measure designed to estimate the effectiveness of knowledge transfer from a pre-trained model to a downstream multi-label ECG diagnosis task. MELEP is generic, working with new target data with different label sets, and computationally efficient, requiring only a single forward pass through the pre-trained model. To the best of our knowledge, MELEP is the first transferability metric specifically designed for multi-label ECG classification problems. Our experiments show that MELEP can predict the performance of pre-trained convolutional and recurrent deep neural networks, on small and imbalanced ECG data. Specifically, we observed strong correlation coefficients (with absolute values exceeding 0.6 in most cases) between MELEP and the actual average F1 scores of the fine-tuned models. Our work highlights the potential of MELEP to expedite the selection of suitable pre-trained models for ECG diagnosis tasks, saving time and effort that would otherwise be spent on fine-tuning these models.</p>","PeriodicalId":101413,"journal":{"name":"Journal of healthcare informatics research","volume":null,"pages":null},"PeriodicalIF":5.4000,"publicationDate":"2024-06-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11310184/pdf/","citationCount":"0","resultStr":"{\"title\":\"MELEP: A Novel Predictive Measure of Transferability in Multi-label ECG Diagnosis.\",\"authors\":\"Cuong V Nguyen, Hieu Minh Duong, Cuong D Do\",\"doi\":\"10.1007/s41666-024-00168-3\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p>In practical electrocardiography (ECG) interpretation, the scarcity of well-annotated data is a common challenge. Transfer learning techniques are valuable in such situations, yet the assessment of transferability has received limited attention. To tackle this issue, we introduce MELEP, which stands for <i>Muti-label Expected Log of Empirical Predictions</i>, a measure designed to estimate the effectiveness of knowledge transfer from a pre-trained model to a downstream multi-label ECG diagnosis task. MELEP is generic, working with new target data with different label sets, and computationally efficient, requiring only a single forward pass through the pre-trained model. To the best of our knowledge, MELEP is the first transferability metric specifically designed for multi-label ECG classification problems. Our experiments show that MELEP can predict the performance of pre-trained convolutional and recurrent deep neural networks, on small and imbalanced ECG data. Specifically, we observed strong correlation coefficients (with absolute values exceeding 0.6 in most cases) between MELEP and the actual average F1 scores of the fine-tuned models. Our work highlights the potential of MELEP to expedite the selection of suitable pre-trained models for ECG diagnosis tasks, saving time and effort that would otherwise be spent on fine-tuning these models.</p>\",\"PeriodicalId\":101413,\"journal\":{\"name\":\"Journal of healthcare informatics research\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":5.4000,\"publicationDate\":\"2024-06-15\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11310184/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of healthcare informatics research\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1007/s41666-024-00168-3\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"2024/9/1 0:00:00\",\"PubModel\":\"eCollection\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of healthcare informatics research","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1007/s41666-024-00168-3","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/9/1 0:00:00","PubModel":"eCollection","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0
摘要
在实际的心电图(ECG)解读中,缺乏注释清晰的数据是一个常见的挑战。迁移学习技术在这种情况下很有价值,但对可迁移性的评估却关注有限。为了解决这个问题,我们引入了 MELEP(Muti-label Expected Log of Empirical Predictions),这是一种用于评估从预训练模型到下游多标签心电图诊断任务的知识转移效果的方法。MELEP 具有通用性,可处理具有不同标签集的新目标数据,而且计算效率高,只需对预训练模型进行一次前向传递。据我们所知,MELEP 是第一个专门为多标签心电图分类问题设计的可转移性指标。我们的实验表明,MELEP 可以预测预先训练好的卷积和递归深度神经网络在少量不平衡心电图数据上的表现。具体来说,我们观察到 MELEP 与微调模型的实际平均 F1 分数之间具有很强的相关系数(在大多数情况下绝对值超过 0.6)。我们的工作凸显了 MELEP 在加快为心电图诊断任务选择合适的预训练模型方面的潜力,从而节省了用于微调这些模型的时间和精力。
MELEP: A Novel Predictive Measure of Transferability in Multi-label ECG Diagnosis.
In practical electrocardiography (ECG) interpretation, the scarcity of well-annotated data is a common challenge. Transfer learning techniques are valuable in such situations, yet the assessment of transferability has received limited attention. To tackle this issue, we introduce MELEP, which stands for Muti-label Expected Log of Empirical Predictions, a measure designed to estimate the effectiveness of knowledge transfer from a pre-trained model to a downstream multi-label ECG diagnosis task. MELEP is generic, working with new target data with different label sets, and computationally efficient, requiring only a single forward pass through the pre-trained model. To the best of our knowledge, MELEP is the first transferability metric specifically designed for multi-label ECG classification problems. Our experiments show that MELEP can predict the performance of pre-trained convolutional and recurrent deep neural networks, on small and imbalanced ECG data. Specifically, we observed strong correlation coefficients (with absolute values exceeding 0.6 in most cases) between MELEP and the actual average F1 scores of the fine-tuned models. Our work highlights the potential of MELEP to expedite the selection of suitable pre-trained models for ECG diagnosis tasks, saving time and effort that would otherwise be spent on fine-tuning these models.