利用多级扩散特征和变换器对高光谱图像中的脑组织进行分类

IF 1.5 4区 工程技术 Q3 MICROSCOPY Journal of microscopy Pub Date : 2024-11-19 DOI:10.1111/jmi.13372
Neetu Sigger, Tuan T Nguyen, Gianluca Tozzi
{"title":"利用多级扩散特征和变换器对高光谱图像中的脑组织进行分类","authors":"Neetu Sigger, Tuan T Nguyen, Gianluca Tozzi","doi":"10.1111/jmi.13372","DOIUrl":null,"url":null,"abstract":"<p><p>Brain surgery is a widely practised and effective treatment for brain tumours, but accurately identifying and classifying tumour boundaries is crucial to maximise resection and avoid neurological complications. This precision in classification is essential for guiding surgical decisions and subsequent treatment planning. Hyperspectral (HS) imaging (HSI) is an emerging multidimensional optical imaging method that captures detailed spectral information across multiple wavelengths, allowing for the identification of nuanced differences in tissue composition, with the potential to enhance intraoperative tissue classification. However, current frameworks often require retraining models for each HSI to extract meaningful features, resulting in long processing times and high computational costs. Additionally, most methods utilise the deep semantic features at the end of the network for classification, ignoring the spatial details contained in the shallow features. To overcome these challenges, we propose a novel approach called MedDiffHSI, which combines diffusion and transformer techniques. Our method involves training an unsupervised learning framework based on the diffusion model to extract high-level and low-level spectral-spatial features from HSI. This approach eliminates the need for retraining of spectral-spatial feature learning model, thereby reducing time complexity. We then extract intermediate multistage features from different timestamps for classification using a pretrained denoising U-Net. To fully explore and exploit the rich contextual semantics and textual information hidden in the extracted diffusion feature, we utilise a spectral-spatial attention module. This module not only learns multistage information about features at different depths, but also extracts and enhances effective information from them. Finally, we employ a supervised transformer-based classifier with weighted majority voting (WMV) to perform the HSI classification. To validate our approach, we conduct comprehensive experiments on in vivo brain database data sets and also extend the analysis to include additional HSI data sets for breast cancer to evaluate the framework performance across different types of tissue. The results demonstrate that our framework outperforms existing approaches by using minimal training samples (5%) while achieving state-of-the-art performance.</p>","PeriodicalId":16484,"journal":{"name":"Journal of microscopy","volume":" ","pages":""},"PeriodicalIF":1.5000,"publicationDate":"2024-11-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Brain tissue classification in hyperspectral images using multistage diffusion features and transformer.\",\"authors\":\"Neetu Sigger, Tuan T Nguyen, Gianluca Tozzi\",\"doi\":\"10.1111/jmi.13372\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p>Brain surgery is a widely practised and effective treatment for brain tumours, but accurately identifying and classifying tumour boundaries is crucial to maximise resection and avoid neurological complications. This precision in classification is essential for guiding surgical decisions and subsequent treatment planning. Hyperspectral (HS) imaging (HSI) is an emerging multidimensional optical imaging method that captures detailed spectral information across multiple wavelengths, allowing for the identification of nuanced differences in tissue composition, with the potential to enhance intraoperative tissue classification. However, current frameworks often require retraining models for each HSI to extract meaningful features, resulting in long processing times and high computational costs. Additionally, most methods utilise the deep semantic features at the end of the network for classification, ignoring the spatial details contained in the shallow features. To overcome these challenges, we propose a novel approach called MedDiffHSI, which combines diffusion and transformer techniques. Our method involves training an unsupervised learning framework based on the diffusion model to extract high-level and low-level spectral-spatial features from HSI. This approach eliminates the need for retraining of spectral-spatial feature learning model, thereby reducing time complexity. We then extract intermediate multistage features from different timestamps for classification using a pretrained denoising U-Net. To fully explore and exploit the rich contextual semantics and textual information hidden in the extracted diffusion feature, we utilise a spectral-spatial attention module. This module not only learns multistage information about features at different depths, but also extracts and enhances effective information from them. Finally, we employ a supervised transformer-based classifier with weighted majority voting (WMV) to perform the HSI classification. To validate our approach, we conduct comprehensive experiments on in vivo brain database data sets and also extend the analysis to include additional HSI data sets for breast cancer to evaluate the framework performance across different types of tissue. The results demonstrate that our framework outperforms existing approaches by using minimal training samples (5%) while achieving state-of-the-art performance.</p>\",\"PeriodicalId\":16484,\"journal\":{\"name\":\"Journal of microscopy\",\"volume\":\" \",\"pages\":\"\"},\"PeriodicalIF\":1.5000,\"publicationDate\":\"2024-11-19\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of microscopy\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://doi.org/10.1111/jmi.13372\",\"RegionNum\":4,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"MICROSCOPY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of microscopy","FirstCategoryId":"5","ListUrlMain":"https://doi.org/10.1111/jmi.13372","RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"MICROSCOPY","Score":null,"Total":0}
引用次数: 0

摘要

脑外科手术是治疗脑肿瘤的一种广泛而有效的方法,但要最大限度地切除肿瘤并避免神经系统并发症,准确识别和分类肿瘤边界至关重要。这种精确分类对于指导手术决策和后续治疗计划至关重要。高光谱(HS)成像(HSI)是一种新兴的多维光学成像方法,它能捕捉多个波长的详细光谱信息,从而识别组织成分的细微差别,有望加强术中组织分类。然而,目前的框架通常需要对每个 HSI 重新训练模型,以提取有意义的特征,从而导致处理时间长、计算成本高。此外,大多数方法利用网络末端的深层语义特征进行分类,忽略了浅层特征中包含的空间细节。为了克服这些挑战,我们提出了一种名为 MedDiffHSI 的新方法,它结合了扩散和变换器技术。我们的方法包括训练一个基于扩散模型的无监督学习框架,以从 HSI 中提取高级和低级频谱空间特征。这种方法无需重新训练光谱空间特征学习模型,从而降低了时间复杂性。然后,我们从不同的时间戳中提取中间多级特征,使用预训练的去噪 U-Net 进行分类。为了充分探索和利用所提取的扩散特征中隐藏的丰富的上下文语义和文本信息,我们使用了频谱-空间注意力模块。该模块不仅能学习不同深度特征的多级信息,还能从中提取并增强有效信息。最后,我们采用基于变换器的有监督分类器和加权多数表决(WMV)来进行 HSI 分类。为了验证我们的方法,我们在活体脑部数据库数据集上进行了全面实验,并将分析扩展到乳腺癌的其他 HSI 数据集,以评估框架在不同类型组织中的性能。结果表明,我们的框架只需使用最少的训练样本(5%)就能达到最先进的性能,优于现有的方法。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Brain tissue classification in hyperspectral images using multistage diffusion features and transformer.

Brain surgery is a widely practised and effective treatment for brain tumours, but accurately identifying and classifying tumour boundaries is crucial to maximise resection and avoid neurological complications. This precision in classification is essential for guiding surgical decisions and subsequent treatment planning. Hyperspectral (HS) imaging (HSI) is an emerging multidimensional optical imaging method that captures detailed spectral information across multiple wavelengths, allowing for the identification of nuanced differences in tissue composition, with the potential to enhance intraoperative tissue classification. However, current frameworks often require retraining models for each HSI to extract meaningful features, resulting in long processing times and high computational costs. Additionally, most methods utilise the deep semantic features at the end of the network for classification, ignoring the spatial details contained in the shallow features. To overcome these challenges, we propose a novel approach called MedDiffHSI, which combines diffusion and transformer techniques. Our method involves training an unsupervised learning framework based on the diffusion model to extract high-level and low-level spectral-spatial features from HSI. This approach eliminates the need for retraining of spectral-spatial feature learning model, thereby reducing time complexity. We then extract intermediate multistage features from different timestamps for classification using a pretrained denoising U-Net. To fully explore and exploit the rich contextual semantics and textual information hidden in the extracted diffusion feature, we utilise a spectral-spatial attention module. This module not only learns multistage information about features at different depths, but also extracts and enhances effective information from them. Finally, we employ a supervised transformer-based classifier with weighted majority voting (WMV) to perform the HSI classification. To validate our approach, we conduct comprehensive experiments on in vivo brain database data sets and also extend the analysis to include additional HSI data sets for breast cancer to evaluate the framework performance across different types of tissue. The results demonstrate that our framework outperforms existing approaches by using minimal training samples (5%) while achieving state-of-the-art performance.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Journal of microscopy
Journal of microscopy 工程技术-显微镜技术
CiteScore
4.30
自引率
5.00%
发文量
83
审稿时长
1 months
期刊介绍: The Journal of Microscopy is the oldest journal dedicated to the science of microscopy and the only peer-reviewed publication of the Royal Microscopical Society. It publishes papers that report on the very latest developments in microscopy such as advances in microscopy techniques or novel areas of application. The Journal does not seek to publish routine applications of microscopy or specimen preparation even though the submission may otherwise have a high scientific merit. The scope covers research in the physical and biological sciences and covers imaging methods using light, electrons, X-rays and other radiations as well as atomic force and near field techniques. Interdisciplinary research is welcome. Papers pertaining to microscopy are also welcomed on optical theory, spectroscopy, novel specimen preparation and manipulation methods and image recording, processing and analysis including dynamic analysis of living specimens. Publication types include full papers, hot topic fast tracked communications and review articles. Authors considering submitting a review article should contact the editorial office first.
期刊最新文献
Towards affordable biomedical imaging: Recent advances in low-cost, high-resolution optoacoustic microscopy. TOC - Issue Information A comparison of super-resolution microscopy techniques for imaging tightly packed microcolonies of an obligate intracellular bacterium. Mechanical properties of bone cells studied by atomic force microscopy. Multicamera simultaneous total internal reflection and interference reflection microscopy.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1