Leveraging 3D convolutional neural network and 3D visible-near-infrared multimodal imaging for enhanced contactless oximetry.

IF 3 3区 医学 Q2 BIOCHEMICAL RESEARCH METHODS Journal of Biomedical Optics Pub Date : 2024-06-01 Epub Date: 2024-08-21 DOI:10.1117/1.JBO.29.S3.S33309
Wang Liao, Chen Zhang, Belmin Alić, Alina Wildenauer, Sarah Dietz-Terjung, Jose Guillermo Ortiz Sucre, Sivagurunathan Sutharsan, Christoph Schöbel, Karsten Seidl, Gunther Notni
{"title":"Leveraging 3D convolutional neural network and 3D visible-near-infrared multimodal imaging for enhanced contactless oximetry.","authors":"Wang Liao, Chen Zhang, Belmin Alić, Alina Wildenauer, Sarah Dietz-Terjung, Jose Guillermo Ortiz Sucre, Sivagurunathan Sutharsan, Christoph Schöbel, Karsten Seidl, Gunther Notni","doi":"10.1117/1.JBO.29.S3.S33309","DOIUrl":null,"url":null,"abstract":"<p><strong>Significance: </strong>Monitoring oxygen saturation ( <math> <mrow><msub><mi>SpO</mi> <mn>2</mn></msub> </mrow> </math> ) is important in healthcare, especially for diagnosing and managing pulmonary diseases. Non-contact approaches broaden the potential applications of <math> <mrow><msub><mi>SpO</mi> <mn>2</mn></msub> </mrow> </math> measurement by better hygiene, comfort, and capability for long-term monitoring. However, existing studies often encounter challenges such as lower signal-to-noise ratios and stringent environmental conditions.</p><p><strong>Aim: </strong>We aim to develop and validate a contactless <math> <mrow><msub><mi>SpO</mi> <mn>2</mn></msub> </mrow> </math> measurement approach using 3D convolutional neural networks (3D CNN) and 3D visible-near-infrared (VIS-NIR) multimodal imaging, to offer a convenient, accurate, and robust alternative for <math> <mrow><msub><mi>SpO</mi> <mn>2</mn></msub> </mrow> </math> monitoring.</p><p><strong>Approach: </strong>We propose an approach that utilizes a 3D VIS-NIR multimodal camera system to capture facial videos, in which <math> <mrow><msub><mi>SpO</mi> <mn>2</mn></msub> </mrow> </math> is estimated through 3D CNN by simultaneously extracting spatial and temporal features. Our approach includes registration of multimodal images, tracking of the 3D region of interest, spatial and temporal preprocessing, and 3D CNN-based feature extraction and <math> <mrow><msub><mi>SpO</mi> <mn>2</mn></msub> </mrow> </math> regression.</p><p><strong>Results: </strong>In a breath-holding experiment involving 23 healthy participants, we obtained multimodal video data with reference <math> <mrow><msub><mi>SpO</mi> <mn>2</mn></msub> </mrow> </math> values ranging from 80% to 99% measured by pulse oximeter on the fingertip. The approach achieved a mean absolute error (MAE) of 2.31% and a Pearson correlation coefficient of 0.64 in the experiment, demonstrating good agreement with traditional pulse oximetry. The discrepancy of estimated <math> <mrow><msub><mi>SpO</mi> <mn>2</mn></msub> </mrow> </math> values was within 3% of the reference <math> <mrow><msub><mi>SpO</mi> <mn>2</mn></msub> </mrow> </math> for <math><mrow><mo>∼</mo> <mn>80</mn> <mo>%</mo></mrow> </math> of all 1-s time points. Besides, in clinical trials involving patients with sleep apnea syndrome, our approach demonstrated robust performance, with an MAE of less than 2% in <math> <mrow><msub><mi>SpO</mi> <mn>2</mn></msub> </mrow> </math> estimations compared to gold-standard polysomnography.</p><p><strong>Conclusions: </strong>The proposed approach offers a promising alternative for non-contact oxygen saturation measurement with good sensitivity to desaturation, showing potential for applications in clinical settings.</p>","PeriodicalId":15264,"journal":{"name":"Journal of Biomedical Optics","volume":"29 Suppl 3","pages":"S33309"},"PeriodicalIF":3.0000,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11338290/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Biomedical Optics","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1117/1.JBO.29.S3.S33309","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/8/21 0:00:00","PubModel":"Epub","JCR":"Q2","JCRName":"BIOCHEMICAL RESEARCH METHODS","Score":null,"Total":0}
引用次数: 0

Abstract

Significance: Monitoring oxygen saturation ( SpO 2 ) is important in healthcare, especially for diagnosing and managing pulmonary diseases. Non-contact approaches broaden the potential applications of SpO 2 measurement by better hygiene, comfort, and capability for long-term monitoring. However, existing studies often encounter challenges such as lower signal-to-noise ratios and stringent environmental conditions.

Aim: We aim to develop and validate a contactless SpO 2 measurement approach using 3D convolutional neural networks (3D CNN) and 3D visible-near-infrared (VIS-NIR) multimodal imaging, to offer a convenient, accurate, and robust alternative for SpO 2 monitoring.

Approach: We propose an approach that utilizes a 3D VIS-NIR multimodal camera system to capture facial videos, in which SpO 2 is estimated through 3D CNN by simultaneously extracting spatial and temporal features. Our approach includes registration of multimodal images, tracking of the 3D region of interest, spatial and temporal preprocessing, and 3D CNN-based feature extraction and SpO 2 regression.

Results: In a breath-holding experiment involving 23 healthy participants, we obtained multimodal video data with reference SpO 2 values ranging from 80% to 99% measured by pulse oximeter on the fingertip. The approach achieved a mean absolute error (MAE) of 2.31% and a Pearson correlation coefficient of 0.64 in the experiment, demonstrating good agreement with traditional pulse oximetry. The discrepancy of estimated SpO 2 values was within 3% of the reference SpO 2 for 80 % of all 1-s time points. Besides, in clinical trials involving patients with sleep apnea syndrome, our approach demonstrated robust performance, with an MAE of less than 2% in SpO 2 estimations compared to gold-standard polysomnography.

Conclusions: The proposed approach offers a promising alternative for non-contact oxygen saturation measurement with good sensitivity to desaturation, showing potential for applications in clinical settings.

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
利用三维卷积神经网络和三维可见光-近红外多模态成像增强非接触式血氧测量。
意义重大:监测血氧饱和度(SpO 2)在医疗保健中非常重要,尤其是在诊断和管理肺部疾病方面。非接触式方法具有更好的卫生性、舒适性和长期监测能力,从而拓宽了 SpO 2 测量的潜在应用领域。目的:我们旨在利用三维卷积神经网络(3D CNN)和三维可见光-近红外(VIS-NIR)多模态成像技术,开发并验证一种非接触式血氧饱和度测量方法,为血氧饱和度监测提供一种便捷、准确、稳健的替代方法:我们提出了一种利用三维可见光-近红外多模态摄像系统捕捉面部视频的方法,通过三维 CNN 同时提取空间和时间特征来估算 SpO 2。我们的方法包括多模态图像的注册、三维感兴趣区域的跟踪、空间和时间预处理,以及基于三维 CNN 的特征提取和 SpO 2 回归:在一项涉及 23 名健康参与者的憋气实验中,我们获得了多模态视频数据,其参考 SpO 2 值范围为指尖脉搏血氧计测量的 80% 至 99%。在实验中,该方法的平均绝对误差(MAE)为 2.31%,皮尔逊相关系数为 0.64,与传统脉搏血氧仪的测量结果一致。在所有 1 秒时间点中,80% 的估计 SpO 2 值与参考 SpO 2 值的误差在 3% 以内。此外,在涉及睡眠呼吸暂停综合症患者的临床试验中,我们的方法表现出了强大的性能,与黄金标准多导睡眠图相比,Spo 2 估计值的 MAE 小于 2%:结论:所提出的方法为非接触式血氧饱和度测量提供了一种有前途的替代方法,对失饱和具有良好的灵敏度,显示出在临床环境中的应用潜力。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
CiteScore
6.40
自引率
5.70%
发文量
263
审稿时长
2 months
期刊介绍: The Journal of Biomedical Optics publishes peer-reviewed papers on the use of modern optical technology for improved health care and biomedical research.
期刊最新文献
Hyperspectral imaging in neurosurgery: a review of systems, computational methods, and clinical applications. Exploring near-infrared autofluorescence properties in parathyroid tissue: an analysis of fresh and paraffin-embedded thyroidectomy specimens. Impact of signal-to-noise ratio and contrast definition on the sensitivity assessment and benchmarking of fluorescence molecular imaging systems. Comparing spatial distributions of ALA-PpIX and indocyanine green in a whole pig brain glioma model using 3D fluorescence cryotomography. Detection properties of indium-111 and IRDye800CW for intraoperative molecular imaging use across tissue phantom models.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1