Computerized Medical Imaging and Graphics最新文献_第4页

Post-hoc out-of-distribution detection for cardiac MRI segmentation 心脏MRI分割的事后非分布检测。

IF 5.4 2区医学 Q1 ENGINEERING, BIOMEDICAL

Computerized Medical Imaging and Graphics

Pub Date : 2025-01-01 DOI: 10.1016/j.compmedimag.2024.102476

Tewodros Weldebirhan Arega , Stéphanie Bricq , Fabrice Meriaudeau

In real-world scenarios, medical image segmentation models encounter input images that may deviate from the training images in various ways. These differences can arise from changes in image scanners and acquisition protocols, or even the images can come from a different modality or domain. When the model encounters these out-of-distribution (OOD) images, it can behave unpredictably. Therefore, it is important to develop a system that handles such out-of-distribution images to ensure the safe usage of the models in clinical practice. In this paper, we propose a post-hoc out-of-distribution (OOD) detection method that can be used with any pre-trained segmentation model. Our method utilizes multi-scale representations extracted from the encoder blocks of the segmentation model and employs Mahalanobis distance as a metric to measure the similarity between the input image and the in-distribution images. The segmentation model is pre-trained on a publicly available cardiac short-axis cine MRI dataset. The detection performance of the proposed method is evaluated on 13 different OOD datasets, which can be categorized as near, mild, and far OOD datasets based on their similarity to the in-distribution dataset. The results show that our method outperforms state-of-the-art feature space-based and uncertainty-based OOD detection methods across the various OOD datasets. Our method successfully detects near, mild, and far OOD images with high detection accuracy, showcasing the advantage of using the multi-scale and semantically rich representations of the encoder. In addition to the feature-based approach, we also propose a Dice coefficient-based OOD detection method, which demonstrates superior performance for adversarial OOD detection and shows a high correlation with segmentation quality. For the uncertainty-based method, despite having a strong correlation with the quality of the segmentation results in the near OOD datasets, they failed to detect mild and far OOD images, indicating the weakness of these methods when the images are more dissimilar. Future work will explore combining Mahalanobis distance and uncertainty scores for improved detection of challenging OOD images that are difficult to segment.

在现实场景中，医学图像分割模型会遇到输入图像可能以各种方式偏离训练图像的情况。这些差异可能来自图像扫描仪和采集协议的变化，甚至图像可能来自不同的模态或域。当模型遇到这些分布外（OOD）图像时，它的行为可能不可预测。因此，开发一种系统来处理这种分布外的图像，以确保模型在临床实践中的安全使用是很重要的。在本文中，我们提出了一种可用于任何预训练分割模型的post-hoc out- distribution （OOD）检测方法。我们的方法利用从分割模型的编码器块中提取的多尺度表示，并使用马氏距离作为度量输入图像与分布中图像之间的相似性的度量。分割模型在公开可用的心脏短轴电影MRI数据集上进行预训练。在13个不同的OOD数据集上评估了该方法的检测性能，这些数据集可以根据其与分布内数据集的相似性分为近、轻度和远OOD数据集。结果表明，我们的方法在各种OOD数据集上优于最先进的基于特征空间和基于不确定性的OOD检测方法。我们的方法以较高的检测精度成功地检测了近、轻度和远OOD图像，展示了使用编码器的多尺度和语义丰富表示的优势。除了基于特征的方法外，我们还提出了一种基于Dice系数的OOD检测方法，该方法在对抗性OOD检测中表现出优越的性能，并且与分割质量具有很高的相关性。对于基于不确定性的方法，尽管与近OOD数据集的分割结果质量有很强的相关性，但它们无法检测到轻度和远OOD图像，这表明这些方法在图像差异较大时的弱点。未来的工作将探索结合马氏距离和不确定性评分，以改进难以分割的具有挑战性的OOD图像的检测。

{"title":"Post-hoc out-of-distribution detection for cardiac MRI segmentation","authors":"Tewodros Weldebirhan Arega , Stéphanie Bricq , Fabrice Meriaudeau","doi":"10.1016/j.compmedimag.2024.102476","DOIUrl":"10.1016/j.compmedimag.2024.102476","url":null,"abstract":"<div><div>In real-world scenarios, medical image segmentation models encounter input images that may deviate from the training images in various ways. These differences can arise from changes in image scanners and acquisition protocols, or even the images can come from a different modality or domain. When the model encounters these out-of-distribution (OOD) images, it can behave unpredictably. Therefore, it is important to develop a system that handles such out-of-distribution images to ensure the safe usage of the models in clinical practice. In this paper, we propose a post-hoc out-of-distribution (OOD) detection method that can be used with any pre-trained segmentation model. Our method utilizes multi-scale representations extracted from the encoder blocks of the segmentation model and employs Mahalanobis distance as a metric to measure the similarity between the input image and the in-distribution images. The segmentation model is pre-trained on a publicly available cardiac short-axis cine MRI dataset. The detection performance of the proposed method is evaluated on 13 different OOD datasets, which can be categorized as near, mild, and far OOD datasets based on their similarity to the in-distribution dataset. The results show that our method outperforms state-of-the-art feature space-based and uncertainty-based OOD detection methods across the various OOD datasets. Our method successfully detects near, mild, and far OOD images with high detection accuracy, showcasing the advantage of using the multi-scale and semantically rich representations of the encoder. In addition to the feature-based approach, we also propose a Dice coefficient-based OOD detection method, which demonstrates superior performance for adversarial OOD detection and shows a high correlation with segmentation quality. For the uncertainty-based method, despite having a strong correlation with the quality of the segmentation results in the near OOD datasets, they failed to detect mild and far OOD images, indicating the weakness of these methods when the images are more dissimilar. Future work will explore combining Mahalanobis distance and uncertainty scores for improved detection of challenging OOD images that are difficult to segment.</div></div>","PeriodicalId":50631,"journal":{"name":"Computerized Medical Imaging and Graphics","volume":"119 ","pages":"Article 102476"},"PeriodicalIF":5.4,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142866125","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Adaptive fusion of dual-view for grading prostate cancer 双影像自适应融合在前列腺癌分级中的应用。

IF 5.4 2区医学 Q1 ENGINEERING, BIOMEDICAL

Computerized Medical Imaging and Graphics

Pub Date : 2025-01-01 DOI: 10.1016/j.compmedimag.2024.102479

Yaolin He , Bowen Li , Ruimin He , Guangming Fu , Dan Sun , Dongyong Shan , Zijian Zhang

Accurate preoperative grading of prostate cancer is crucial for assisted diagnosis. Multi-parametric magnetic resonance imaging (MRI) is a commonly used non-invasive approach, however, the interpretation of MRI images is still subject to significant subjectivity due to variations in physicians’ expertise and experience. To achieve accurate, non-invasive, and efficient grading of prostate cancer, this paper proposes a deep learning method that adaptively fuses dual-view MRI images. Specifically, a dual-view adaptive fusion model is designed. The model employs encoders to extract embedded features from two MRI sequences: T2-weighted imaging (T2WI) and apparent diffusion coefficient (ADC). The model reconstructs the original input images using the embedded features and adopts a cross-embedding fusion module to adaptively fuse the embedded features from the two views. Adaptive fusion refers to dynamically adjusting the fusion weights of the features from the two views according to different input samples, thereby fully utilizing complementary information. Furthermore, the model adaptively weights the prediction results from the two views based on uncertainty estimation, further enhancing the grading performance. To verify the importance of effective multi-view fusion for prostate cancer grading, extensive experiments are designed. The experiments evaluate the performance of single-view models, dual-view models, and state-of-the-art multi-view fusion algorithms. The results demonstrate that the proposed dual-view adaptive fusion method achieves the best grading performance, confirming its effectiveness for assisted grading diagnosis of prostate cancer. This study provides a novel deep learning solution for preoperative grading of prostate cancer, which has the potential to assist clinical physicians in making more accurate diagnostic decisions and has significant clinical application value.

术前准确的前列腺癌分级是辅助诊断的关键。多参数磁共振成像（MRI）是一种常用的非侵入性方法，然而，由于医生的专业知识和经验的差异，MRI图像的解释仍然受到显著的主观性的影响。为了实现准确、无创、高效的前列腺癌分级，本文提出了一种自适应融合双视图MRI图像的深度学习方法。具体来说，设计了一种双视图自适应融合模型。该模型采用编码器从两个MRI序列中提取嵌入特征：t2加权成像（T2WI）和表观扩散系数（ADC）。该模型利用嵌入特征重构原始输入图像，并采用交叉嵌入融合模块自适应融合两视图的嵌入特征。自适应融合是指根据不同的输入样本动态调整两个视图特征的融合权值，从而充分利用互补信息。在不确定性估计的基础上，对两种观点的预测结果进行自适应加权，进一步提高了分级性能。为了验证有效的多视点融合对前列腺癌分级的重要性，我们设计了大量的实验。实验评估了单视图模型、双视图模型和最先进的多视图融合算法的性能。结果表明，所提出的双视图自适应融合方法获得了最佳的分级性能，证实了其在前列腺癌辅助分级诊断中的有效性。本研究为前列腺癌术前分级提供了一种新颖的深度学习解决方案，有可能帮助临床医生做出更准确的诊断决策，具有重要的临床应用价值。

{"title":"Adaptive fusion of dual-view for grading prostate cancer","authors":"Yaolin He , Bowen Li , Ruimin He , Guangming Fu , Dan Sun , Dongyong Shan , Zijian Zhang","doi":"10.1016/j.compmedimag.2024.102479","DOIUrl":"10.1016/j.compmedimag.2024.102479","url":null,"abstract":"<div><div>Accurate preoperative grading of prostate cancer is crucial for assisted diagnosis. Multi-parametric magnetic resonance imaging (MRI) is a commonly used non-invasive approach, however, the interpretation of MRI images is still subject to significant subjectivity due to variations in physicians’ expertise and experience. To achieve accurate, non-invasive, and efficient grading of prostate cancer, this paper proposes a deep learning method that adaptively fuses dual-view MRI images. Specifically, a dual-view adaptive fusion model is designed. The model employs encoders to extract embedded features from two MRI sequences: T2-weighted imaging (T2WI) and apparent diffusion coefficient (ADC). The model reconstructs the original input images using the embedded features and adopts a cross-embedding fusion module to adaptively fuse the embedded features from the two views. Adaptive fusion refers to dynamically adjusting the fusion weights of the features from the two views according to different input samples, thereby fully utilizing complementary information. Furthermore, the model adaptively weights the prediction results from the two views based on uncertainty estimation, further enhancing the grading performance. To verify the importance of effective multi-view fusion for prostate cancer grading, extensive experiments are designed. The experiments evaluate the performance of single-view models, dual-view models, and state-of-the-art multi-view fusion algorithms. The results demonstrate that the proposed dual-view adaptive fusion method achieves the best grading performance, confirming its effectiveness for assisted grading diagnosis of prostate cancer. This study provides a novel deep learning solution for preoperative grading of prostate cancer, which has the potential to assist clinical physicians in making more accurate diagnostic decisions and has significant clinical application value.</div></div>","PeriodicalId":50631,"journal":{"name":"Computerized Medical Imaging and Graphics","volume":"119 ","pages":"Article 102479"},"PeriodicalIF":5.4,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142873356","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Head pose-assisted localization of facial landmarks for enhanced fast registration in skull base surgery 在颅底手术中，头部姿势辅助定位面部标志增强快速定位。

IF 5.4 2区医学 Q1 ENGINEERING, BIOMEDICAL

Computerized Medical Imaging and Graphics

Pub Date : 2024-12-30 DOI: 10.1016/j.compmedimag.2024.102483

Yifei Yang , Jingfan Fan , Tianyu Fu , Deqiang Xiao , Dongsheng Ma , Hong Song , Zhengkai Feng , Youping Liu , Jian Yang

In skull base surgery, the method of using a probe to draw or 3D scanners to acquire intraoperative facial point clouds for spatial registration presents several issues. Manual manipulation results in inefficiency and poor consistency. Traditional registration algorithms based on point clouds are highly dependent on the initial pose. The complexity of registration algorithms can also extend the required time. To address these issues, we used an RGB-D camera to capture real-time facial point clouds during surgery. The initial registration of the 3D model reconstructed from preoperative CT/MR images and the point cloud collected during surgery is accomplished through corresponding facial landmarks. The facial point clouds collected intraoperatively often contain rotations caused by the free-angle camera. Benefit from the close spatial geometric relationship between head pose and facial landmarks coordinates, we propose a facial landmarks localization network assisted by estimating head pose. The shared representation head pose estimation module boosts network performance by enhancing its perception of global facial features. The proposed network facilitates the localization of landmark points in both preoperative and intraoperative point clouds, enabling rapid automatic registration. A free-view human facial landmarks dataset called 3D-FVL was synthesized from clinical CT images for training. The proposed network achieves leading localization accuracy and robustness on two public datasets and the 3D-FVL. In clinical experiments, using the Artec Eva scanner, the trained network achieved a concurrent reduction in average registration time to 0.28 s, with an average registration error of 2.33 mm. The proposed method significantly reduced registration time, while meeting clinical accuracy requirements for surgical navigation. Our research will help to improving the efficiency and quality of skull base surgery.

在颅底手术中，使用探针绘制或3D扫描仪获取术中面部点云进行空间配准的方法存在几个问题。手工操作导致效率低下和一致性差。传统的基于点云的配准算法高度依赖于初始姿态。配准算法的复杂性也会延长所需的时间。为了解决这些问题，我们使用RGB-D相机在手术过程中实时捕捉面部点云。术前CT/MR图像重建的三维模型与术中采集的点云通过相应的面部地标完成初始配准。术中采集的面部点云通常包含由自由角度相机引起的旋转。利用头部姿态与面部地标坐标之间密切的空间几何关系，提出了一种基于头部姿态估计的面部地标定位网络。共享表示头姿估计模块通过增强其对全局面部特征的感知来提高网络性能。所提出的网络有助于在术前和术中点云中定位地标点，实现快速自动配准。从临床CT图像中合成了一个名为3D-FVL的自由视图人脸标志数据集，用于训练。该网络在两个公共数据集和3D-FVL上实现了领先的定位精度和鲁棒性。在临床实验中，使用Artec Eva扫描仪，训练后的神经网络将平均配准时间减少到0.28 s，平均配准误差为2.33 mm。该方法在满足临床手术导航精度要求的同时，显著减少了挂号时间。我们的研究将有助于提高颅底手术的效率和质量。

{"title":"Head pose-assisted localization of facial landmarks for enhanced fast registration in skull base surgery","authors":"Yifei Yang , Jingfan Fan , Tianyu Fu , Deqiang Xiao , Dongsheng Ma , Hong Song , Zhengkai Feng , Youping Liu , Jian Yang","doi":"10.1016/j.compmedimag.2024.102483","DOIUrl":"10.1016/j.compmedimag.2024.102483","url":null,"abstract":"<div><div>In skull base surgery, the method of using a probe to draw or 3D scanners to acquire intraoperative facial point clouds for spatial registration presents several issues. Manual manipulation results in inefficiency and poor consistency. Traditional registration algorithms based on point clouds are highly dependent on the initial pose. The complexity of registration algorithms can also extend the required time. To address these issues, we used an RGB-D camera to capture real-time facial point clouds during surgery. The initial registration of the 3D model reconstructed from preoperative CT/MR images and the point cloud collected during surgery is accomplished through corresponding facial landmarks. The facial point clouds collected intraoperatively often contain rotations caused by the free-angle camera. Benefit from the close spatial geometric relationship between head pose and facial landmarks coordinates, we propose a facial landmarks localization network assisted by estimating head pose. The shared representation head pose estimation module boosts network performance by enhancing its perception of global facial features. The proposed network facilitates the localization of landmark points in both preoperative and intraoperative point clouds, enabling rapid automatic registration. A free-view human facial landmarks dataset called 3D-FVL was synthesized from clinical CT images for training. The proposed network achieves leading localization accuracy and robustness on two public datasets and the 3D-FVL. In clinical experiments, using the Artec Eva scanner, the trained network achieved a concurrent reduction in average registration time to 0.28 s, with an average registration error of 2.33 mm. The proposed method significantly reduced registration time, while meeting clinical accuracy requirements for surgical navigation. Our research will help to improving the efficiency and quality of skull base surgery.</div></div>","PeriodicalId":50631,"journal":{"name":"Computerized Medical Imaging and Graphics","volume":"120 ","pages":"Article 102483"},"PeriodicalIF":5.4,"publicationDate":"2024-12-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142958412","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Attention incorporated network for sharing low-rank, image and k-space information during MR image reconstruction to achieve single breath-hold cardiac Cine imaging 注意在MR图像重建过程中引入网络共享低秩、图像和k空间信息，实现单次屏气心脏电影成像。

IF 5.4 2区医学 Q1 ENGINEERING, BIOMEDICAL

Computerized Medical Imaging and Graphics

Pub Date : 2024-12-28 DOI: 10.1016/j.compmedimag.2024.102475

Siying Xu , Kerstin Hammernik , Andreas Lingg , Jens Kübler , Patrick Krumm , Daniel Rueckert , Sergios Gatidis , Thomas Küstner

Cardiac Cine Magnetic Resonance Imaging (MRI) provides an accurate assessment of heart morphology and function in clinical practice. However, MRI requires long acquisition times, with recent deep learning-based methods showing great promise to accelerate imaging and enhance reconstruction quality. Existing networks exhibit some common limitations that constrain further acceleration possibilities, including single-domain learning, reliance on a single regularization term, and equal feature contribution. To address these limitations, we propose to embed information from multiple domains, including low-rank, image, and k-space, in a novel deep learning network for MRI reconstruction, which we denote as A-LIKNet. A-LIKNet adopts a parallel-branch structure, enabling independent learning in the k-space and image domain. Coupled information sharing layers realize the information exchange between domains. Furthermore, we introduce attention mechanisms into the network to assign greater weights to more critical coils or important temporal frames. Training and testing were conducted on an in-house dataset, including 91 cardiovascular patients and 38 healthy subjects scanned with 2D cardiac Cine using retrospective undersampling. Additionally, we evaluated A-LIKNet on the real-time prospectively undersampled data from the OCMR dataset. The results demonstrate that our proposed A-LIKNet outperforms existing methods and provides high-quality reconstructions. The network can effectively reconstruct highly retrospectively undersampled dynamic MR images up to

24 \times

accelerations, indicating its potential for single breath-hold imaging.

心脏电影磁共振成像（MRI）在临床实践中提供了心脏形态和功能的准确评估。然而，MRI需要较长的采集时间，最近基于深度学习的方法显示出加速成像和提高重建质量的巨大希望。现有的网络表现出一些共同的限制，这些限制了进一步加速的可能性，包括单域学习、依赖单个正则化项和相等的特征贡献。为了解决这些限制，我们建议将来自多个领域的信息（包括低秩、图像和k空间）嵌入到一个用于MRI重建的新型深度学习网络中，我们将其称为a - liknet。a - liknet采用并行分支结构，可以在k空间和图像域进行独立学习。耦合信息共享层实现了域间的信息交换。此外，我们在网络中引入了注意机制，为更关键的线圈或重要的时间框架分配更大的权重。训练和测试是在一个内部数据集上进行的，该数据集包括91名心血管患者和38名健康受试者，使用回顾性欠采样的2D心脏电影扫描。此外，我们在OCMR数据集中的实时前瞻性欠采样数据上评估了A-LIKNet。结果表明，我们提出的A-LIKNet优于现有的方法，并提供了高质量的重建。该网络可以有效地重建高度回顾性采样不足的动态MR图像，高达24倍的加速度，这表明它具有单次屏气成像的潜力。

{"title":"Attention incorporated network for sharing low-rank, image and k-space information during MR image reconstruction to achieve single breath-hold cardiac Cine imaging","authors":"Siying Xu , Kerstin Hammernik , Andreas Lingg , Jens Kübler , Patrick Krumm , Daniel Rueckert , Sergios Gatidis , Thomas Küstner","doi":"10.1016/j.compmedimag.2024.102475","DOIUrl":"10.1016/j.compmedimag.2024.102475","url":null,"abstract":"<div><div>Cardiac Cine Magnetic Resonance Imaging (MRI) provides an accurate assessment of heart morphology and function in clinical practice. However, MRI requires long acquisition times, with recent deep learning-based methods showing great promise to accelerate imaging and enhance reconstruction quality. Existing networks exhibit some common limitations that constrain further acceleration possibilities, including single-domain learning, reliance on a single regularization term, and equal feature contribution. To address these limitations, we propose to embed information from multiple domains, including low-rank, image, and k-space, in a novel deep learning network for MRI reconstruction, which we denote as A-LIKNet. A-LIKNet adopts a parallel-branch structure, enabling independent learning in the k-space and image domain. Coupled information sharing layers realize the information exchange between domains. Furthermore, we introduce attention mechanisms into the network to assign greater weights to more critical coils or important temporal frames. Training and testing were conducted on an in-house dataset, including 91 cardiovascular patients and 38 healthy subjects scanned with 2D cardiac Cine using retrospective undersampling. Additionally, we evaluated A-LIKNet on the real-time prospectively undersampled data from the OCMR dataset. The results demonstrate that our proposed A-LIKNet outperforms existing methods and provides high-quality reconstructions. The network can effectively reconstruct highly retrospectively undersampled dynamic MR images up to <span><math><mrow><mn>24</mn><mo>×</mo></mrow></math></span> accelerations, indicating its potential for single breath-hold imaging.</div></div>","PeriodicalId":50631,"journal":{"name":"Computerized Medical Imaging and Graphics","volume":"120 ","pages":"Article 102475"},"PeriodicalIF":5.4,"publicationDate":"2024-12-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142985476","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

PET-based lesion graphs meet clinical data: An interpretable cross-attention framework for DLBCL treatment response prediction 基于pet的病变图符合临床数据：可解释的DLBCL治疗反应预测的交叉注意框架。

IF 5.4 2区医学 Q1 ENGINEERING, BIOMEDICAL

Computerized Medical Imaging and Graphics

Pub Date : 2024-12-25 DOI: 10.1016/j.compmedimag.2024.102481

Oriane Thiery , Mira Rizkallah , Clément Bailly , Caroline Bodet-Milin , Emmanuel Itti , René-Olivier Casasnovas , Steven Le Gouill , Thomas Carlier , Diana Mateus

Diffuse Large B-cell Lymphoma (DLBCL) is a lymphatic cancer of steadily growing incidence. Its diagnostic and follow-up rely on the analysis of clinical biomarkers and 18F-Fluorodeoxyglucose (FDG)-PET/CT images. In this context, we target the problem of assisting in the early identification of high-risk DLBCL patients from both images and tabular clinical data. We propose a solution based on a graph neural network model, capable of simultaneously modeling the variable number of lesions across patients, and fusing information from both data modalities and over lesions. Given the distributed nature of DLBCL lesions, we represent the PET image of each patient as an attributed lesion graph. Such lesion-graphs keep all relevant image information while offering a compact tradeoff between the characterization of full images and single lesions. We also design a cross-attention module to fuse the image attributes with clinical indicators, which is particularly challenging given the large difference in dimensionality and prognostic strength of each modality. To this end, we propose several cross-attention configurations, discuss the implications of each design, and experimentally compare their performances. The last module fuses the updated attributes across lesions and makes a probabilistic prediction of the patient’s 2-year progression-free survival (PFS). We carry out the experimental validation of our proposed framework on a prospective multicentric dataset of 545 patients. Experimental results show our framework effectively integrates the multi-lesion image information improving over a model relying only on the most prognostic clinical data. The analysis further shows the interpretable properties inherent to our graph-based design, which enables tracing the decision back to the most important lesions and features.

弥漫性大b细胞淋巴瘤（DLBCL）是一种发病率稳步上升的淋巴癌。其诊断和随访依赖于临床生物标志物分析和18f -氟脱氧葡萄糖(FDG)-PET/CT图像。在这种情况下，我们的目标是从图像和表格临床数据中帮助早期识别高危DLBCL患者。我们提出了一种基于图神经网络模型的解决方案，该模型能够同时对不同患者的病变数量进行建模，并融合来自数据模式和病变的信息。考虑到DLBCL病变的分布特性，我们将每个患者的PET图像表示为属性病变图。这样的病变图保留了所有相关的图像信息，同时在完整图像和单个病变的表征之间提供了紧凑的权衡。我们还设计了一个交叉关注模块，将图像属性与临床指标融合在一起，考虑到每种模式在维度和预后强度方面的巨大差异，这尤其具有挑战性。为此，我们提出了几种交叉注意配置，讨论了每种设计的含义，并通过实验比较了它们的性能。最后一个模块融合了更新的病变属性，并对患者的2年无进展生存期（PFS）进行了概率预测。我们在545名患者的前瞻性多中心数据集上对我们提出的框架进行了实验验证。实验结果表明，该框架有效地整合了多病变图像信息，比仅依赖最预后临床数据的模型有所改善。分析进一步显示了我们基于图形的设计固有的可解释属性，这使得决策能够追溯到最重要的病变和特征。

{"title":"PET-based lesion graphs meet clinical data: An interpretable cross-attention framework for DLBCL treatment response prediction","authors":"Oriane Thiery , Mira Rizkallah , Clément Bailly , Caroline Bodet-Milin , Emmanuel Itti , René-Olivier Casasnovas , Steven Le Gouill , Thomas Carlier , Diana Mateus","doi":"10.1016/j.compmedimag.2024.102481","DOIUrl":"10.1016/j.compmedimag.2024.102481","url":null,"abstract":"<div><div>Diffuse Large B-cell Lymphoma (DLBCL) is a lymphatic cancer of steadily growing incidence. Its diagnostic and follow-up rely on the analysis of clinical biomarkers and 18F-Fluorodeoxyglucose (FDG)-PET/CT images. In this context, we target the problem of assisting in the early identification of high-risk DLBCL patients from both images and tabular clinical data. We propose a solution based on a graph neural network model, capable of simultaneously modeling the variable number of lesions across patients, and fusing information from both data modalities and over lesions. Given the distributed nature of DLBCL lesions, we represent the PET image of each patient as an attributed lesion graph. Such lesion-graphs keep all relevant image information while offering a compact tradeoff between the characterization of full images and single lesions. We also design a cross-attention module to fuse the image attributes with clinical indicators, which is particularly challenging given the large difference in dimensionality and prognostic strength of each modality. To this end, we propose several cross-attention configurations, discuss the implications of each design, and experimentally compare their performances. The last module fuses the updated attributes across lesions and makes a probabilistic prediction of the patient’s 2-year progression-free survival (PFS). We carry out the experimental validation of our proposed framework on a prospective multicentric dataset of 545 patients. Experimental results show our framework effectively integrates the multi-lesion image information improving over a model relying only on the most prognostic clinical data. The analysis further shows the interpretable properties inherent to our graph-based design, which enables tracing the decision back to the most important lesions and features.</div></div>","PeriodicalId":50631,"journal":{"name":"Computerized Medical Imaging and Graphics","volume":"120 ","pages":"Article 102481"},"PeriodicalIF":5.4,"publicationDate":"2024-12-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143015673","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

General retinal layer segmentation in OCT images via reinforcement constraint 基于增强约束的普通OCT图像视网膜层分割。

IF 5.4 2区医学 Q1 ENGINEERING, BIOMEDICAL

Computerized Medical Imaging and Graphics

Pub Date : 2024-12-24 DOI: 10.1016/j.compmedimag.2024.102480

Jinbao Hao, Huiqi Li, Shuai Lu, Zeheng Li, Weihang Zhang

The change of layer thickness of retina is closely associated with the development of ocular diseases such as glaucoma and optic disc drusen. Optical coherence tomography (OCT) is a widely used technology to visualize the lamellar structures of retina. Accurate segmentation of retinal lamellar structures is crucial for diagnosis, treatment, and related research of ocular diseases. However, existing studies have focused on improving the segmentation accuracy, they cannot achieve consistent segmentation performance on different types of datasets, such as retinal OCT images with optic disc and interference of diseases. To this end, a general retinal layer segmentation method is presented in this paper. To obtain more continuous and smoother boundaries, feature enhanced decoding module with reinforcement constraint is proposed, fusing boundary prior and distribution prior, and correcting bias in learning process simultaneously. To enhance the model’s perception of the slender retinal structure, position channel attention is introduced, obtaining global dependencies of both space and channel. To handle the imbalanced distribution of retinal OCT images, focal loss is introduced, guiding the model to pay more attention to retinal layers with a smaller proportion. The designed method achieves the state-of-the-art (SOTA) overall performance on five datasets (i.e., MGU, DUKE, NR206, OCTA500 and private dataset).

视网膜层厚度的变化与青光眼、视盘囊肿等眼部疾病的发生密切相关。光学相干断层扫描（OCT）是一种广泛应用于视网膜板层结构可视化的技术。视网膜板层结构的准确分割对于眼科疾病的诊断、治疗和相关研究至关重要。然而，现有的研究主要集中在提高分割精度上，无法在不同类型的数据集上实现一致的分割性能，例如视盘视网膜OCT图像和疾病干扰。为此，本文提出了一种通用的视网膜层分割方法。为了获得更连续平滑的边界，提出了带有强化约束的特征增强解码模块，融合边界先验和分布先验，同时校正学习过程中的偏差。为了增强模型对细长视网膜结构的感知，引入了位置通道注意，获得了空间和通道的全局依赖关系。为了解决视网膜OCT图像分布不平衡的问题，引入焦损失，引导模型更多地关注比例较小的视网膜层。设计的方法在5个数据集（MGU、DUKE、NR206、OCTA500和private dataset）上实现了最先进（SOTA）的整体性能。

{"title":"General retinal layer segmentation in OCT images via reinforcement constraint","authors":"Jinbao Hao, Huiqi Li, Shuai Lu, Zeheng Li, Weihang Zhang","doi":"10.1016/j.compmedimag.2024.102480","DOIUrl":"10.1016/j.compmedimag.2024.102480","url":null,"abstract":"<div><div>The change of layer thickness of retina is closely associated with the development of ocular diseases such as glaucoma and optic disc drusen. Optical coherence tomography (OCT) is a widely used technology to visualize the lamellar structures of retina. Accurate segmentation of retinal lamellar structures is crucial for diagnosis, treatment, and related research of ocular diseases. However, existing studies have focused on improving the segmentation accuracy, they cannot achieve consistent segmentation performance on different types of datasets, such as retinal OCT images with optic disc and interference of diseases. To this end, a general retinal layer segmentation method is presented in this paper. To obtain more continuous and smoother boundaries, feature enhanced decoding module with reinforcement constraint is proposed, fusing boundary prior and distribution prior, and correcting bias in learning process simultaneously. To enhance the model’s perception of the slender retinal structure, position channel attention is introduced, obtaining global dependencies of both space and channel. To handle the imbalanced distribution of retinal OCT images, focal loss is introduced, guiding the model to pay more attention to retinal layers with a smaller proportion. The designed method achieves the state-of-the-art (SOTA) overall performance on five datasets (i.e., MGU, DUKE, NR206, OCTA500 and private dataset).</div></div>","PeriodicalId":50631,"journal":{"name":"Computerized Medical Imaging and Graphics","volume":"120 ","pages":"Article 102480"},"PeriodicalIF":5.4,"publicationDate":"2024-12-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142933309","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Computer-assisted diagnosis for axillary lymph node metastasis of early breast cancer based on transformer with dual-modal adaptive mid-term fusion using ultrasound elastography 超声弹性成像双模态自适应中期融合变压器对早期乳腺癌腋窝淋巴结转移的计算机辅助诊断

IF 5.4 2区医学 Q1 ENGINEERING, BIOMEDICAL

Computerized Medical Imaging and Graphics

Pub Date : 2024-11-26 DOI: 10.1016/j.compmedimag.2024.102472

Chihao Gong , Yinglan Wu , Guangyuan Zhang , Xuan Liu , Xiaoyao Zhu , Nian Cai , Jian Li

Accurate preoperative qualitative assessment of axillary lymph node metastasis (ALNM) in early breast cancer patients is crucial for precise clinical staging and selection of axillary treatment strategies. Although previous studies have introduced artificial intelligence (AI) to enhance the assessment performance of ALNM, they all focus on the prediction performances of their AI models and neglect the clinical assistance to the radiologists, which brings some issues to the clinical practice. To this end, we propose a human–AI collaboration strategy for ALNM diagnosis of early breast cancer, in which a novel deep learning framework, termed DAMF-former, is designed to assist radiologists in evaluating ALNM. Specifically, the DAMF-former focuses on the axillary region rather than the primary tumor area in previous studies. To mimic the radiologists’ alternative integration of the UE images of the target axillary lymph nodes for comprehensive analysis, adaptive mid-term fusion is proposed to alternatively extract and adaptively fuse the high-level features from the dual-modal UE images (i.e., B-mode ultrasound and Shear Wave Elastography). To further improve the diagnostic outcome of the DAMF-former, an adaptive Youden index scheme is proposed to deal with the fully fused dual-modal UE image features at the end of the framework, which can balance the diagnostic performance in terms of sensitivity and specificity. The clinical experiment indicates that the designed DAMF-former can assist and improve the diagnostic abilities of less-experienced radiologists for ALNM. Especially, the junior radiologists can significantly improve the diagnostic outcome from 0.807 AUC [95% CI: 0.781, 0.830] to 0.883 AUC [95% CI: 0.861, 0.902] (

P

-value

<

0.0001). Moreover, there are great agreements among radiologists of different levels when assisted by the DAMF-former (Kappa value ranging from 0.805 to 0.895;

P

-value

<

0.0001), suggesting that less-experienced radiologists can potentially achieve a diagnostic level similar to that of experienced radiologists through human–AI collaboration. This study explores a potential solution to human–AI collaboration for ALNM diagnosis based on UE images.

早期乳腺癌患者腋窝淋巴结转移（ALNM）的术前准确定性评估对于准确的临床分期和选择腋窝治疗策略至关重要。虽然以往的研究引入了人工智能（AI）来提高ALNM的评估性能，但都侧重于人工智能模型的预测性能，忽视了对放射科医生的临床辅助，这给临床实践带来了一些问题。为此，我们提出了一种用于早期乳腺癌ALNM诊断的人类-人工智能协作策略，其中设计了一种称为DAMF-former的新型深度学习框架，以协助放射科医生评估ALNM。具体而言，在以往的研究中，DAMF-former侧重于腋窝区域而不是原发肿瘤区域。为了模仿放射科医生对目标腋窝淋巴结UE图像的替代整合进行综合分析，提出了自适应中期融合，从双模UE图像（即b超和横波弹性成像）中交替提取和自适应融合高级特征。为了进一步提高DAMF-former的诊断效果，提出了一种自适应的约登指数方案来处理框架末端完全融合的双峰UE图像特征，从而在敏感性和特异性方面平衡诊断性能。临床实验表明，所设计的DAMF-former能够辅助和提高经验不足的放射科医师对ALNM的诊断能力。特别是，初级放射科医师可以显著改善诊断结果，从0.807 AUC [95% CI: 0.781, 0.830]提高到0.883 AUC [95% CI: 0.861, 0.902] （p值<；0.0001）。此外，在DAMF-former (Kappa值为0.805 ~ 0.895；p值<；0.0001)，这表明经验不足的放射科医生可以通过人类与人工智能的合作达到与经验丰富的放射科医生相似的诊断水平。本研究探索了一种基于UE图像的人工智能协同诊断ALNM的潜在解决方案。

{"title":"Computer-assisted diagnosis for axillary lymph node metastasis of early breast cancer based on transformer with dual-modal adaptive mid-term fusion using ultrasound elastography","authors":"Chihao Gong , Yinglan Wu , Guangyuan Zhang , Xuan Liu , Xiaoyao Zhu , Nian Cai , Jian Li","doi":"10.1016/j.compmedimag.2024.102472","DOIUrl":"10.1016/j.compmedimag.2024.102472","url":null,"abstract":"<div><div>Accurate preoperative qualitative assessment of axillary lymph node metastasis (ALNM) in early breast cancer patients is crucial for precise clinical staging and selection of axillary treatment strategies. Although previous studies have introduced artificial intelligence (AI) to enhance the assessment performance of ALNM, they all focus on the prediction performances of their AI models and neglect the clinical assistance to the radiologists, which brings some issues to the clinical practice. To this end, we propose a human–AI collaboration strategy for ALNM diagnosis of early breast cancer, in which a novel deep learning framework, termed DAMF-former, is designed to assist radiologists in evaluating ALNM. Specifically, the DAMF-former focuses on the axillary region rather than the primary tumor area in previous studies. To mimic the radiologists’ alternative integration of the UE images of the target axillary lymph nodes for comprehensive analysis, adaptive mid-term fusion is proposed to alternatively extract and adaptively fuse the high-level features from the dual-modal UE images (i.e., B-mode ultrasound and Shear Wave Elastography). To further improve the diagnostic outcome of the DAMF-former, an adaptive Youden index scheme is proposed to deal with the fully fused dual-modal UE image features at the end of the framework, which can balance the diagnostic performance in terms of sensitivity and specificity. The clinical experiment indicates that the designed DAMF-former can assist and improve the diagnostic abilities of less-experienced radiologists for ALNM. Especially, the junior radiologists can significantly improve the diagnostic outcome from 0.807 AUC [95% CI: 0.781, 0.830] to 0.883 AUC [95% CI: 0.861, 0.902] (<span><math><mi>P</mi></math></span>-value <span><math><mo><</mo></math></span>0.0001). Moreover, there are great agreements among radiologists of different levels when assisted by the DAMF-former (Kappa value ranging from 0.805 to 0.895; <span><math><mi>P</mi></math></span>-value <span><math><mo><</mo></math></span>0.0001), suggesting that less-experienced radiologists can potentially achieve a diagnostic level similar to that of experienced radiologists through human–AI collaboration. This study explores a potential solution to human–AI collaboration for ALNM diagnosis based on UE images.</div></div>","PeriodicalId":50631,"journal":{"name":"Computerized Medical Imaging and Graphics","volume":"119 ","pages":"Article 102472"},"PeriodicalIF":5.4,"publicationDate":"2024-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142743192","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Uncertainty-aware regression model to predict post-operative visual acuity in patients with macular holes 不确定性感知回归模型预测黄斑裂孔患者术后视力

IF 5.4 2区医学 Q1 ENGINEERING, BIOMEDICAL

Computerized Medical Imaging and Graphics

Pub Date : 2024-11-26 DOI: 10.1016/j.compmedimag.2024.102461

Burak Kucukgoz , Ke Zou , Declan C. Murphy , David H. Steel , Boguslaw Obara , Huazhu Fu

Full-thickness macular holes are a relatively common and visually disabling condition with a prevalence of approximately 0.5% in the over-40-year-old age group. If left untreated, the hole typically enlarges, reducing visual acuity (VA) below the definition of blindness in the eye affected. They are now routinely treated with surgery, which can close the hole and improve vision in most cases. The extent of improvement, however, is variable and dependent on the size of the hole and other features which can be discerned in spectral-domain optical coherence tomography imaging, which is now routinely available in eye clinics globally. Artificial intelligence (AI) models have been developed to enable surgical decision-making and have achieved relatively high predictive performance. However, their black-box behavior is opaque to users and uncertainty associated with their predictions is not typically stated, leading to a lack of trust among clinicians and patients. In this paper, we describe an uncertainty-aware regression model (U-ARM) for predicting VA for people undergoing macular hole surgery using preoperative spectral-domain optical coherence tomography images, achieving an MAE of 6.07, RMSE of 9.11 and R2 of 0.47 in internal tests, and an MAE of 6.49, RMSE of 9.49, and R2 of 0.42 in external tests. In addition to predicting VA following surgery, U-ARM displays its associated uncertainty, a

p

-value of <0.005 in internal and external tests, showing the predictions are not due to random chance. We then qualitatively evaluated the performance of U-ARM. Lastly, we demonstrate out-of-sample data performance, generalizing well to data outside the training distribution, low-quality images, and unseen instances not encountered during training. The results show that U-ARM outperforms commonly used methods in terms of prediction and reliability. U-ARM is thus a promising approach for clinical settings and can improve the reliability of AI models in predicting VA.

全层黄斑孔是一种相对常见的致盲疾病，在40岁以上人群中患病率约为0.5%。如果不及时治疗，孔洞通常会扩大，使受影响眼睛的视力降低到失明的定义以下。他们现在的常规治疗是手术，在大多数情况下，手术可以关闭这个洞，改善视力。然而，改善的程度是可变的，取决于孔洞的大小和光谱域光学相干断层扫描成像中可以识别的其他特征，这种成像现在在全球眼科诊所常规使用。人工智能（AI）模型已被开发用于外科决策，并取得了相对较高的预测性能。然而，他们的黑箱行为对用户来说是不透明的，与他们的预测相关的不确定性通常不会被陈述，导致临床医生和患者之间缺乏信任。在本文中，我们描述了一个不确定性感知回归模型（U-ARM），用于使用术前光谱域光学相干断层扫描图像预测黄斑孔手术患者的VA，内部测试的MAE为6.07，RMSE为9.11，R2为0.47，外部测试的MAE为6.49，RMSE为9.49，R2为0.42。除了预测手术后VA外，U-ARM还显示了其相关的不确定性，在内部和外部测试中p值为<；0.005，表明预测不是由于随机机会。然后，我们对U-ARM的性能进行了定性评估。最后，我们展示了样本外数据的性能，可以很好地推广到训练分布之外的数据、低质量图像和训练过程中未遇到的未见实例。结果表明，U-ARM在预测和可靠性方面优于常用方法。因此，U-ARM在临床环境中是一种很有前途的方法，可以提高人工智能模型预测VA的可靠性。

{"title":"Uncertainty-aware regression model to predict post-operative visual acuity in patients with macular holes","authors":"Burak Kucukgoz , Ke Zou , Declan C. Murphy , David H. Steel , Boguslaw Obara , Huazhu Fu","doi":"10.1016/j.compmedimag.2024.102461","DOIUrl":"10.1016/j.compmedimag.2024.102461","url":null,"abstract":"<div><div>Full-thickness macular holes are a relatively common and visually disabling condition with a prevalence of approximately 0.5% in the over-40-year-old age group. If left untreated, the hole typically enlarges, reducing visual acuity (VA) below the definition of blindness in the eye affected. They are now routinely treated with surgery, which can close the hole and improve vision in most cases. The extent of improvement, however, is variable and dependent on the size of the hole and other features which can be discerned in spectral-domain optical coherence tomography imaging, which is now routinely available in eye clinics globally. Artificial intelligence (AI) models have been developed to enable surgical decision-making and have achieved relatively high predictive performance. However, their black-box behavior is opaque to users and uncertainty associated with their predictions is not typically stated, leading to a lack of trust among clinicians and patients. In this paper, we describe an uncertainty-aware regression model (U-ARM) for predicting VA for people undergoing macular hole surgery using preoperative spectral-domain optical coherence tomography images, achieving an MAE of 6.07, RMSE of 9.11 and R2 of 0.47 in internal tests, and an MAE of 6.49, RMSE of 9.49, and R2 of 0.42 in external tests. In addition to predicting VA following surgery, U-ARM displays its associated uncertainty, a <span><math><mi>p</mi></math></span>-value of <0.005 in internal and external tests, showing the predictions are not due to random chance. We then qualitatively evaluated the performance of U-ARM. Lastly, we demonstrate out-of-sample data performance, generalizing well to data outside the training distribution, low-quality images, and unseen instances not encountered during training. The results show that U-ARM outperforms commonly used methods in terms of prediction and reliability. U-ARM is thus a promising approach for clinical settings and can improve the reliability of AI models in predicting VA.</div></div>","PeriodicalId":50631,"journal":{"name":"Computerized Medical Imaging and Graphics","volume":"119 ","pages":"Article 102461"},"PeriodicalIF":5.4,"publicationDate":"2024-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142743193","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Self-supervised learning on dual-sequence magnetic resonance imaging for automatic segmentation of nasopharyngeal carcinoma 在双序列磁共振成像上进行自我监督学习以自动分割鼻咽癌

IF 5.4 2区医学 Q1 ENGINEERING, BIOMEDICAL

Computerized Medical Imaging and Graphics

Pub Date : 2024-11-22 DOI: 10.1016/j.compmedimag.2024.102471

Zongyou Cai , Zhangnan Zhong , Haiwei Lin , Bingsheng Huang , Ziyue Xu , Bin Huang , Wei Deng , Qiting Wu , Kaixin Lei , Jiegeng Lyu , Yufeng Ye , Hanwei Chen , Jian Zhang

Automating the segmentation of nasopharyngeal carcinoma (NPC) is crucial for therapeutic procedures but presents challenges given the hurdles in amassing extensively annotated datasets. Although previous studies have applied self-supervised learning to capitalize on unlabeled data to improve segmentation performance, these methods often overlooked the benefits of dual-sequence magnetic resonance imaging (MRI). In the present study, we incorporated self-supervised learning with a saliency transformation module using unlabeled dual-sequence MRI for accurate NPC segmentation. 44 labeled and 72 unlabeled patients were collected to develop and evaluate our network. Impressively, our network achieved a mean Dice similarity coefficient (DSC) of 0.77, which is consistent with a previous study that relied on a training set of 4,100 annotated cases. The results further revealed that our approach required minimal adjustments, primarily

<

20% tweak in the DSC, to meet clinical standards. By enhancing the automatic segmentation of NPC, our method alleviates the annotation burden on oncologists, curbs subjectivity, and ensures reliable NPC delineation.

鼻咽癌（NPC）的自动分割对治疗过程至关重要，但由于在积累大量标注数据集方面存在障碍，因此面临着挑战。尽管之前的研究已经应用了自我监督学习来利用未标记数据提高分割性能，但这些方法往往忽略了双序列磁共振成像（MRI）的优势。在本研究中，我们将自我监督学习与显著性转换模块相结合，利用未标记的双序列核磁共振成像进行准确的鼻咽癌分割。为了开发和评估我们的网络，我们收集了 44 位已标记和 72 位未标记的患者。令人印象深刻的是，我们的网络达到了 0.77 的平均 Dice 相似系数 (DSC)，这与之前一项依赖于 4100 个标注病例的训练集的研究结果一致。研究结果进一步表明，我们的方法只需进行最小限度的调整，主要是对 DSC 进行 20% 的调整，即可达到临床标准。通过加强对鼻咽癌的自动分割，我们的方法减轻了肿瘤学家的注释负担，减少了主观性，并确保了可靠的鼻咽癌划分。

{"title":"Self-supervised learning on dual-sequence magnetic resonance imaging for automatic segmentation of nasopharyngeal carcinoma","authors":"Zongyou Cai , Zhangnan Zhong , Haiwei Lin , Bingsheng Huang , Ziyue Xu , Bin Huang , Wei Deng , Qiting Wu , Kaixin Lei , Jiegeng Lyu , Yufeng Ye , Hanwei Chen , Jian Zhang","doi":"10.1016/j.compmedimag.2024.102471","DOIUrl":"10.1016/j.compmedimag.2024.102471","url":null,"abstract":"<div><div>Automating the segmentation of nasopharyngeal carcinoma (NPC) is crucial for therapeutic procedures but presents challenges given the hurdles in amassing extensively annotated datasets. Although previous studies have applied self-supervised learning to capitalize on unlabeled data to improve segmentation performance, these methods often overlooked the benefits of dual-sequence magnetic resonance imaging (MRI). In the present study, we incorporated self-supervised learning with a saliency transformation module using unlabeled dual-sequence MRI for accurate NPC segmentation. 44 labeled and 72 unlabeled patients were collected to develop and evaluate our network. Impressively, our network achieved a mean Dice similarity coefficient (DSC) of 0.77, which is consistent with a previous study that relied on a training set of 4,100 annotated cases. The results further revealed that our approach required minimal adjustments, primarily <span><math><mo><</mo></math></span> 20% tweak in the DSC, to meet clinical standards. By enhancing the automatic segmentation of NPC, our method alleviates the annotation burden on oncologists, curbs subjectivity, and ensures reliable NPC delineation.</div></div>","PeriodicalId":50631,"journal":{"name":"Computerized Medical Imaging and Graphics","volume":"118 ","pages":"Article 102471"},"PeriodicalIF":5.4,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142722395","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Single color digital H&E staining with In-and-Out Net 单色数字 H&E 染色与进出网。

IF 5.4 2区医学 Q1 ENGINEERING, BIOMEDICAL

Computerized Medical Imaging and Graphics

Pub Date : 2024-11-20 DOI: 10.1016/j.compmedimag.2024.102468

Mengkun Chen , Yen-Tung Liu , Fadeel Sher Khan , Matthew C. Fox , Jason S. Reichenberg , Fabiana C.P.S. Lopes , Katherine R. Sebastian , Mia K. Markey , James W. Tunnell

Digital staining streamlines traditional staining procedures by digitally generating stained images from unstained or differently stained images. While conventional staining methods involve time-consuming chemical processes, digital staining offers an efficient and low-infrastructure alternative. Researchers can expedite tissue analysis without physical sectioning by leveraging microscopy-based techniques, such as confocal microscopy. However, interpreting grayscale or pseudo-color microscopic images remains challenging for pathologists and surgeons accustomed to traditional histologically stained images. To fill this gap, various studies explore digitally simulating staining to mimic targeted histological stains. This paper introduces a novel network, In-and-Out Net, designed explicitly for digital staining tasks. Based on Generative Adversarial Networks (GAN), our model efficiently transforms Reflectance Confocal Microscopy (RCM) images into Hematoxylin and Eosin (H&E) stained images. Using aluminum chloride preprocessing for skin tissue, we enhance nuclei contrast in RCM images. We trained the model with digital H&E labels featuring two fluorescence channels, eliminating the need for image registration and providing pixel-level ground truth. Our contributions include proposing an optimal training strategy, conducting a comparative analysis demonstrating state-of-the-art performance, validating the model through an ablation study, and collecting perfectly matched input and ground truth images without registration. In-and-Out Net showcases promising results, offering a valuable tool for digital staining tasks and advancing the field of histological image analysis.

数字染色法通过从未染色或不同染色的图像中以数字方式生成染色图像，从而简化了传统染色程序。传统染色方法涉及耗时的化学过程，而数字染色提供了一种高效、低基础设施的替代方法。研究人员可以利用共聚焦显微镜等基于显微镜的技术，在不进行物理切片的情况下加快组织分析。然而，对于习惯于传统组织染色图像的病理学家和外科医生来说，解读灰度或伪彩色显微图像仍然是一项挑战。为了填补这一空白，各种研究都在探索以数字方式模拟染色，以模仿有针对性的组织学染色。本文介绍了一种专为数字染色任务设计的新型网络--In-and-Out Net。基于生成对抗网络（GAN），我们的模型能有效地将反射共聚焦显微镜（RCM）图像转换为苏木精和伊红（H&E）染色图像。通过对皮肤组织进行氯化铝预处理，我们增强了 RCM 图像中的细胞核对比度。我们使用具有两个荧光通道的数字 H&E 标签对模型进行了训练，从而消除了图像配准的需要，并提供了像素级的地面实况。我们的贡献包括提出了最佳训练策略，进行了比较分析，展示了最先进的性能，通过消融研究验证了模型，并收集了完全匹配的输入图像和无需配准的地面实况图像。In-and-Out Net展示了很有前景的结果，为数字染色任务提供了有价值的工具，推动了组织学图像分析领域的发展。

{"title":"Single color digital H&E staining with In-and-Out Net","authors":"Mengkun Chen , Yen-Tung Liu , Fadeel Sher Khan , Matthew C. Fox , Jason S. Reichenberg , Fabiana C.P.S. Lopes , Katherine R. Sebastian , Mia K. Markey , James W. Tunnell","doi":"10.1016/j.compmedimag.2024.102468","DOIUrl":"10.1016/j.compmedimag.2024.102468","url":null,"abstract":"<div><div>Digital staining streamlines traditional staining procedures by digitally generating stained images from unstained or differently stained images. While conventional staining methods involve time-consuming chemical processes, digital staining offers an efficient and low-infrastructure alternative. Researchers can expedite tissue analysis without physical sectioning by leveraging microscopy-based techniques, such as confocal microscopy. However, interpreting grayscale or pseudo-color microscopic images remains challenging for pathologists and surgeons accustomed to traditional histologically stained images. To fill this gap, various studies explore digitally simulating staining to mimic targeted histological stains. This paper introduces a novel network, In-and-Out Net, designed explicitly for digital staining tasks. Based on Generative Adversarial Networks (GAN), our model efficiently transforms Reflectance Confocal Microscopy (RCM) images into Hematoxylin and Eosin (H&E) stained images. Using aluminum chloride preprocessing for skin tissue, we enhance nuclei contrast in RCM images. We trained the model with digital H&E labels featuring two fluorescence channels, eliminating the need for image registration and providing pixel-level ground truth. Our contributions include proposing an optimal training strategy, conducting a comparative analysis demonstrating state-of-the-art performance, validating the model through an ablation study, and collecting perfectly matched input and ground truth images without registration. In-and-Out Net showcases promising results, offering a valuable tool for digital staining tasks and advancing the field of histological image analysis.</div></div>","PeriodicalId":50631,"journal":{"name":"Computerized Medical Imaging and Graphics","volume":"118 ","pages":"Article 102468"},"PeriodicalIF":5.4,"publicationDate":"2024-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142695998","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0