Pub Date : 2024-11-26DOI: 10.1016/j.patrec.2024.11.025
Long Wen , Yuxing Ye , Lei Zuo
Surface defect detection (SDD) is the necessary technique to monitor the surface quality of production. However, fine grain defects caused by stress loading, environmental influences, and construction defects is still a challenge to detect. In this research, the convolutional neural network for crack segmentation is developed based on the feature fusion and feedback on the global features and multi-scale feature (GAF-Net). First, a multi-scale feature feedback module (MSFF) is proposed, which uses four different scales to refine local features by fusing high-level and sub-high-level features to perform feedback correction. Secondly, the global feature module (GF) is proposed to generate a fine global information map using local features and adaptive weighted fusion with the correction map for crack detection. Finally, the GAF-Net network with multi-level feature maps is deeply supervised to accelerate GAF-Net and improve the detection accuracy. GAF-Net is trained and experimented on three publicly available pavement crack datasets, and the results show that GAF-Net achieves state-of-the-art results in the IoU segmentation metrics when compared to other deep learning methods (Crackforest: 53.61 %; Crack500: 65.19 %; DeepCrack: 81.63 %).
{"title":"GAF-Net: A new automated segmentation method based on multiscale feature fusion and feedback module","authors":"Long Wen , Yuxing Ye , Lei Zuo","doi":"10.1016/j.patrec.2024.11.025","DOIUrl":"10.1016/j.patrec.2024.11.025","url":null,"abstract":"<div><div>Surface defect detection (SDD) is the necessary technique to monitor the surface quality of production. However, fine grain defects caused by stress loading, environmental influences, and construction defects is still a challenge to detect. In this research, the convolutional neural network for crack segmentation is developed based on the feature fusion and feedback on the global features and multi-scale feature (GAF-Net). First, a multi-scale feature feedback module (MSFF) is proposed, which uses four different scales to refine local features by fusing high-level and sub-high-level features to perform feedback correction. Secondly, the global feature module (GF) is proposed to generate a fine global information map using local features and adaptive weighted fusion with the correction map for crack detection. Finally, the GAF-Net network with multi-level feature maps is deeply supervised to accelerate GAF-Net and improve the detection accuracy. GAF-Net is trained and experimented on three publicly available pavement crack datasets, and the results show that GAF-Net achieves state-of-the-art results in the IoU segmentation metrics when compared to other deep learning methods (Crackforest: 53.61 %; Crack500: 65.19 %; DeepCrack: 81.63 %).</div></div>","PeriodicalId":54638,"journal":{"name":"Pattern Recognition Letters","volume":"187 ","pages":"Pages 86-92"},"PeriodicalIF":3.9,"publicationDate":"2024-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142746066","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-26DOI: 10.1016/j.patrec.2024.11.023
Sanket Wathore, Subrahmanyam Gorthi
Panoramic X-rays are crucial in dental radiology, providing detailed images that are essential for diagnosing and planning treatment for various oral conditions. The advent of automated methods that learn from annotated data promises to significantly aid clinical experts in making accurate diagnoses. However, these methods often require large amounts of annotated data, making the generation of high-quality annotations for panoramic X-rays both challenging and time-consuming. This paper introduces a novel bilateral symmetry-based augmentation method specifically designed to enhance tooth segmentation in panoramic X-rays. By exploiting the inherent bilateral symmetry of these images, our proposed method systematically generates augmented data, leading to substantial improvements in the performance of tooth segmentation models. By increasing the training data size fourfold, our approach proportionately reduces the effort required to manually annotate extensive datasets. These findings highlight the potential of leveraging the symmetrical properties of medical images to enhance model performance and accuracy in dental radiology. The effectiveness of the proposed method is evaluated on three widely adopted deep learning models: U-Net, SE U-Net, and TransUNet. Significant improvements in segmentation accuracy are observed with the proposed augmentation method across all models. For example, the average Dice Similarity Coefficient (DSC) increases by over 8%, reaching 76.7% for TransUNet. Further, comparisons with existing augmentation methods, including rigid transform-based and elastic grid-based techniques, show that the proposed method consistently outperforms them with additional improvements up to 5% in terms of average DSC, with the exact improvement varying depending on the model and training dataset size. We have made the data augmentation codes and tools developed based on our method available at https://github.com/wathoresanket/bilateralsymmetrybasedaugmentation.
{"title":"Bilateral symmetry-based augmentation method for improved tooth segmentation in panoramic X-rays","authors":"Sanket Wathore, Subrahmanyam Gorthi","doi":"10.1016/j.patrec.2024.11.023","DOIUrl":"10.1016/j.patrec.2024.11.023","url":null,"abstract":"<div><div>Panoramic X-rays are crucial in dental radiology, providing detailed images that are essential for diagnosing and planning treatment for various oral conditions. The advent of automated methods that learn from annotated data promises to significantly aid clinical experts in making accurate diagnoses. However, these methods often require large amounts of annotated data, making the generation of high-quality annotations for panoramic X-rays both challenging and time-consuming. This paper introduces a novel bilateral symmetry-based augmentation method specifically designed to enhance tooth segmentation in panoramic X-rays. By exploiting the inherent bilateral symmetry of these images, our proposed method systematically generates augmented data, leading to substantial improvements in the performance of tooth segmentation models. By increasing the training data size fourfold, our approach proportionately reduces the effort required to manually annotate extensive datasets. These findings highlight the potential of leveraging the symmetrical properties of medical images to enhance model performance and accuracy in dental radiology. The effectiveness of the proposed method is evaluated on three widely adopted deep learning models: U-Net, SE U-Net, and TransUNet. Significant improvements in segmentation accuracy are observed with the proposed augmentation method across all models. For example, the average Dice Similarity Coefficient (DSC) increases by over 8%, reaching 76.7% for TransUNet. Further, comparisons with existing augmentation methods, including rigid transform-based and elastic grid-based techniques, show that the proposed method consistently outperforms them with additional improvements up to 5% in terms of average DSC, with the exact improvement varying depending on the model and training dataset size. We have made the data augmentation codes and tools developed based on our method available at <span><span>https://github.com/wathoresanket/bilateralsymmetrybasedaugmentation</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":54638,"journal":{"name":"Pattern Recognition Letters","volume":"188 ","pages":"Pages 1-7"},"PeriodicalIF":3.9,"publicationDate":"2024-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142742880","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-24DOI: 10.1016/j.patrec.2024.11.003
Mudassar Ali , Haoji Hu , Tong Wu , Maryam Mansoor , Qiong Luo , Weizeng Zheng , Neng Jin
Accurate tumor segmentation within MRI images is of great importance for both diagnosis and treatment; however, in many cases, sufficient annotated datasets may not be available. This paper develops a novel approach to the medical image segmentation of tumors in the brain, liver, and pelvic regions within MRI images, by combining an attention-enhanced U-Net model with a cGAN. We introduce three key novelties: a patch discriminator in the cGAN to enhance realism of generated images, attention mechanisms in the U-Net to enhance the accuracy of segmentation, and finally an application to pelvic MRI segmentation, which has seen little exploration. Our method addresses the issue of limited availability of annotated data by generating realistic synthetic images to augment the process of training. Our experimental results on brain, liver, and pelvic MRI datasets show that our approach outperforms the state-of-the-art methods with a Dice Coefficient of 98.61 % for brain MRI, 88.60 % for liver MRI, and 91.93 % for pelvic MRI. We can also observe great increases in the Hausdorff Distance, at especially complex anatomical regions such as tumor boundaries. The proposed combination of synthetic data creation and novel segmentation techniques opens new perspectives for robust medical image segmentation.
{"title":"Segmentation of MRI tumors and pelvic anatomy via cGAN-synthesized data and attention-enhanced U-Net","authors":"Mudassar Ali , Haoji Hu , Tong Wu , Maryam Mansoor , Qiong Luo , Weizeng Zheng , Neng Jin","doi":"10.1016/j.patrec.2024.11.003","DOIUrl":"10.1016/j.patrec.2024.11.003","url":null,"abstract":"<div><div>Accurate tumor segmentation within MRI images is of great importance for both diagnosis and treatment; however, in many cases, sufficient annotated datasets may not be available. This paper develops a novel approach to the medical image segmentation of tumors in the brain, liver, and pelvic regions within MRI images, by combining an attention-enhanced U-Net model with a cGAN. We introduce three key novelties: a patch discriminator in the cGAN to enhance realism of generated images, attention mechanisms in the U-Net to enhance the accuracy of segmentation, and finally an application to pelvic MRI segmentation, which has seen little exploration. Our method addresses the issue of limited availability of annotated data by generating realistic synthetic images to augment the process of training. Our experimental results on brain, liver, and pelvic MRI datasets show that our approach outperforms the state-of-the-art methods with a Dice Coefficient of 98.61 % for brain MRI, 88.60 % for liver MRI, and 91.93 % for pelvic MRI. We can also observe great increases in the Hausdorff Distance, at especially complex anatomical regions such as tumor boundaries. The proposed combination of synthetic data creation and novel segmentation techniques opens new perspectives for robust medical image segmentation.</div></div>","PeriodicalId":54638,"journal":{"name":"Pattern Recognition Letters","volume":"187 ","pages":"Pages 100-106"},"PeriodicalIF":3.9,"publicationDate":"2024-11-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142746068","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-23DOI: 10.1016/j.patrec.2024.11.019
Dennis J. Silva , Jiří Kosinka , Ronaldo F. Hashimoto , Jos B.T.M. Roerdink , Alexandre Morimitsu , Wonder A.L. Alves
A component tree is a graph representation that encodes the connected components of the upper or lower level sets of a grayscale image. Consequently, the nodes of a component tree represent binary images of the encoded connected components. There exist various algorithms that efficiently extract information and attributes of nodes of a component tree by incrementally exploiting the subset relation encoding in the tree. However, to the best of our knowledge, there is no such incremental approach to extract the contours of the nodes. In this paper, we propose an efficient incremental method to compute the contours of the nodes of a component tree by counting the edges (sides) of contour pixels. In addition, we discuss our method’s time complexity. We also experimentally show that our proposed method is faster than the standard approach based on node reconstruction.
{"title":"Incremental component tree contour computation","authors":"Dennis J. Silva , Jiří Kosinka , Ronaldo F. Hashimoto , Jos B.T.M. Roerdink , Alexandre Morimitsu , Wonder A.L. Alves","doi":"10.1016/j.patrec.2024.11.019","DOIUrl":"10.1016/j.patrec.2024.11.019","url":null,"abstract":"<div><div>A component tree is a graph representation that encodes the connected components of the upper or lower level sets of a grayscale image. Consequently, the nodes of a component tree represent binary images of the encoded connected components. There exist various algorithms that efficiently extract information and attributes of nodes of a component tree by incrementally exploiting the subset relation encoding in the tree. However, to the best of our knowledge, there is no such incremental approach to extract the contours of the nodes. In this paper, we propose an efficient incremental method to compute the contours of the nodes of a component tree by counting the edges (sides) of contour pixels. In addition, we discuss our method’s time complexity. We also experimentally show that our proposed method is faster than the standard approach based on node reconstruction.</div></div>","PeriodicalId":54638,"journal":{"name":"Pattern Recognition Letters","volume":"187 ","pages":"Pages 115-121"},"PeriodicalIF":3.9,"publicationDate":"2024-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142746070","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-23DOI: 10.1016/j.patrec.2024.11.015
Wonder A.L. Alves , Wander S. Campos , Charles F. Gobber , Dennis J. Silva , Ronaldo F. Hashimoto
Morphological Attribute Profiles serve as powerful tools for extracting meaningful features from remote sensing data. The construction of Morphological Attribute Profiles relies on two primary parameters: the choice of attribute type and the definition of a numerical threshold sequence. However, selecting an appropriate threshold sequence can be a difficult task, as an inappropriate choice can lead to an uninformative feature space. In this paper, we propose a semi-automatic approach based on the theory of Maximally Stable Extremal Regions to address this challenge. Our approach takes an increasing attribute type and an initial sequence of thresholds as input and locally adjusts threshold values based on region stability within the image. Experimental results demonstrate that our method significantly increases classification accuracy through the refinement of threshold values.
{"title":"Multichannel image classification based on adaptive attribute profiles","authors":"Wonder A.L. Alves , Wander S. Campos , Charles F. Gobber , Dennis J. Silva , Ronaldo F. Hashimoto","doi":"10.1016/j.patrec.2024.11.015","DOIUrl":"10.1016/j.patrec.2024.11.015","url":null,"abstract":"<div><div>Morphological Attribute Profiles serve as powerful tools for extracting meaningful features from remote sensing data. The construction of Morphological Attribute Profiles relies on two primary parameters: the choice of attribute type and the definition of a numerical threshold sequence. However, selecting an appropriate threshold sequence can be a difficult task, as an inappropriate choice can lead to an uninformative feature space. In this paper, we propose a semi-automatic approach based on the theory of Maximally Stable Extremal Regions to address this challenge. Our approach takes an increasing attribute type and an initial sequence of thresholds as input and locally adjusts threshold values based on region stability within the image. Experimental results demonstrate that our method significantly increases classification accuracy through the refinement of threshold values.</div></div>","PeriodicalId":54638,"journal":{"name":"Pattern Recognition Letters","volume":"187 ","pages":"Pages 107-114"},"PeriodicalIF":3.9,"publicationDate":"2024-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142746069","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-22DOI: 10.1016/j.patrec.2024.11.020
Zexin Ji , Beiji Zou , Xiaoyan Kui , Hua Li , Pierre Vera , Su Ruan
Existing deep learning-based super-resolution generation approaches usually depend on the backbone of convolutional neural networks (CNNs) or Transformers. CNN-based approaches are unable to model long-range dependencies, whereas Transformer-based approaches encounter significant computational burdens due to quadratic complexity in calculations. Moreover, high-frequency texture details in images generated by existing approaches still remain indistinct, posing a major challenge in super-resolution tasks. To overcome these problems, we propose a self-prior guided Mamba network with edge-aware constraint (SEMambaSR) for medical image super-resolution. Recently, State Space Models (SSMs), notably Mamba, have gained prominence for the ability to efficiently model long-range dependencies with low complexity. In this paper, we propose to integrate Mamba into the Unet network allowing to extract multi-scale local and global features to generate high-quality super-resolution images. Additionally, we introduce perturbations by randomly adding a brightness window to the input image, enabling the network to mine the self-prior information of the image. We also design an improved 2D-Selective-Scan (ISS2D) module to learn and adaptively fuse multi-directional long-range dependencies in image features to enhance feature representation. An edge-aware constraint is exploited to learn the multi-scale edge information from encoder features for better synthesis of texture boundaries. Our qualitative and quantitative experimental findings indicate superior super-resolution performance over current methods on IXI and BraTS2021 medical datasets. Specifically, our approach achieved a PSNR of 33.44 dB and an SSIM of 0.9371 on IXI, and a PSNR of 41.99 dB and an SSIM of 0.9846 on BraTS2021, both for 2 upsampling. The downstream vision task on brain tumor segmentation, using a U-Net network, also reveals the effectiveness of our approach, with a mean Dice Score of 57.06% on the BraTS2021 dataset.
现有的基于深度学习的超分辨率生成方法通常依赖于卷积神经网络（cnn）或变压器的主干。基于cnn的方法无法对长期依赖关系进行建模，而基于transformer的方法由于计算的二次复杂度而面临巨大的计算负担。此外，现有方法生成的图像中的高频纹理细节仍然不清晰，这对超分辨率任务构成了重大挑战。为了克服这些问题，我们提出了一种具有边缘感知约束的自先验引导Mamba网络（SEMambaSR）用于医学图像超分辨率。最近，状态空间模型（State Space Models, ssm），特别是Mamba，因为能够以低复杂性高效地建模远程依赖关系而获得了突出的地位。在本文中，我们建议将曼巴整合到Unet网络中，允许提取多尺度局部和全局特征，以生成高质量的超分辨率图像。此外，我们通过在输入图像中随机添加亮度窗口来引入扰动，使网络能够挖掘图像的自先验信息。我们还设计了一个改进的二维选择性扫描（ISS2D）模块来学习和自适应融合图像特征中的多向远程依赖关系，以增强特征表征。利用边缘感知约束从编码器特征中学习多尺度边缘信息，从而更好地合成纹理边界。我们的定性和定量实验结果表明，在IXI和BraTS2021医疗数据集上，超分辨率性能优于当前方法。具体来说，我们的方法在IXI上实现了33.44 dB的PSNR和0.9371的SSIM，在BraTS2021上实现了41.99 dB的PSNR和0.9846的SSIM，两者都是2倍上采样。使用U-Net网络的脑肿瘤分割下游视觉任务也显示了我们方法的有效性，在BraTS2021数据集上的平均Dice Score为57.06%。
{"title":"Generation of super-resolution for medical image via a self-prior guided Mamba network with edge-aware constraint","authors":"Zexin Ji , Beiji Zou , Xiaoyan Kui , Hua Li , Pierre Vera , Su Ruan","doi":"10.1016/j.patrec.2024.11.020","DOIUrl":"10.1016/j.patrec.2024.11.020","url":null,"abstract":"<div><div>Existing deep learning-based super-resolution generation approaches usually depend on the backbone of convolutional neural networks (CNNs) or Transformers. CNN-based approaches are unable to model long-range dependencies, whereas Transformer-based approaches encounter significant computational burdens due to quadratic complexity in calculations. Moreover, high-frequency texture details in images generated by existing approaches still remain indistinct, posing a major challenge in super-resolution tasks. To overcome these problems, we propose a self-prior guided Mamba network with edge-aware constraint (SEMambaSR) for medical image super-resolution. Recently, State Space Models (SSMs), notably Mamba, have gained prominence for the ability to efficiently model long-range dependencies with low complexity. In this paper, we propose to integrate Mamba into the Unet network allowing to extract multi-scale local and global features to generate high-quality super-resolution images. Additionally, we introduce perturbations by randomly adding a brightness window to the input image, enabling the network to mine the self-prior information of the image. We also design an improved 2D-Selective-Scan (ISS2D) module to learn and adaptively fuse multi-directional long-range dependencies in image features to enhance feature representation. An edge-aware constraint is exploited to learn the multi-scale edge information from encoder features for better synthesis of texture boundaries. Our qualitative and quantitative experimental findings indicate superior super-resolution performance over current methods on IXI and BraTS2021 medical datasets. Specifically, our approach achieved a PSNR of 33.44 dB and an SSIM of 0.9371 on IXI, and a PSNR of 41.99 dB and an SSIM of 0.9846 on BraTS2021, both for 2<span><math><mo>×</mo></math></span> upsampling. The downstream vision task on brain tumor segmentation, using a U-Net network, also reveals the effectiveness of our approach, with a mean Dice Score of 57.06% on the BraTS2021 dataset.</div></div>","PeriodicalId":54638,"journal":{"name":"Pattern Recognition Letters","volume":"187 ","pages":"Pages 93-99"},"PeriodicalIF":3.9,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142746067","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-22DOI: 10.1016/j.patrec.2024.10.011
Hojoon Lee , Seunghwan Lee , Inyoung Jung , Sungeun Hong
Test-time adaptation (TTA) refines pre-trained models during deployment, enabling them to effectively manage new, previously unseen data. However, existing TTA methods focus mainly on global domain alignment, which reduces domain-level gaps but often leads to suboptimal performance. This is because they fail to explicitly consider class-wise alignment, resulting in errors when reliable pseudo-labels are unavailable and source domain samples are inaccessible. In this study, we propose a prototypical class-wise test-time adaptation method, which consists of class-wise prototype adaptation and reliable pseudo-labeling. A main challenge in this approach is the lack of direct access to source domain samples. We leverage the class-specific knowledge contained in the weights of the pre-trained model. To construct class prototypes from the unlabeled target domain, we further introduce a methodology to enhance the reliability of pseudo labels. Our method is adaptable to various models and has been extensively validated, consistently outperforming baselines across multiple benchmark datasets.
{"title":"Prototypical class-wise test-time adaptation","authors":"Hojoon Lee , Seunghwan Lee , Inyoung Jung , Sungeun Hong","doi":"10.1016/j.patrec.2024.10.011","DOIUrl":"10.1016/j.patrec.2024.10.011","url":null,"abstract":"<div><div>Test-time adaptation (TTA) refines pre-trained models during deployment, enabling them to effectively manage new, previously unseen data. However, existing TTA methods focus mainly on global domain alignment, which reduces domain-level gaps but often leads to suboptimal performance. This is because they fail to explicitly consider class-wise alignment, resulting in errors when reliable pseudo-labels are unavailable and source domain samples are inaccessible. In this study, we propose a prototypical class-wise test-time adaptation method, which consists of class-wise prototype adaptation and reliable pseudo-labeling. A main challenge in this approach is the lack of direct access to source domain samples. We leverage the class-specific knowledge contained in the weights of the pre-trained model. To construct class prototypes from the unlabeled target domain, we further introduce a methodology to enhance the reliability of pseudo labels. Our method is adaptable to various models and has been extensively validated, consistently outperforming baselines across multiple benchmark datasets.</div></div>","PeriodicalId":54638,"journal":{"name":"Pattern Recognition Letters","volume":"187 ","pages":"Pages 49-55"},"PeriodicalIF":3.9,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142699182","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-22DOI: 10.1016/j.patrec.2024.11.017
Anastasia Konik , Patrick Clarysse , Nicolas Duchateau
Personalization of biophysical models to real data is essential to achieve realistic simulations or generate relevant synthetic populations. However, some of these models involve randomness, which poses two challenges: they do not allow the standard personalization to each individual’s data and they lack an analytical formulation required for optimization. In previous work, we introduced a population-based personalization strategy which overcomes these challenges and demonstrated its feasibility on simple 2D geometrical models of myocardial infarct. The method consists in matching the distributions of the synthetic and real populations, quantified through the Kullback–Leibler (KL) divergence. Personalization is achieved with a gradient-free algorithm (CMA-ES), which generates sets of candidate solutions represented by their covariance matrix, whose coefficients evolve until the synthetic and real data are matched. However, the robustness of this strategy regarding settings and more complex data was not challenged. In this work, we specifically address these points, with (i) an improved design, (ii) a thorough evaluation on crucial aspects of the personalization process, including hyperparameters and initialization, and (iii) the application to 3D data. Despite some limits of the simple geometrical models used, our method is able to capture the main characteristics of the real data, as demonstrated both on 2D and 3D segmented late Gadolinium images of 123 subjects with acute myocardial infarction.
{"title":"Detailed evaluation of a population-wise personalization approach to generate synthetic myocardial infarct images","authors":"Anastasia Konik , Patrick Clarysse , Nicolas Duchateau","doi":"10.1016/j.patrec.2024.11.017","DOIUrl":"10.1016/j.patrec.2024.11.017","url":null,"abstract":"<div><div>Personalization of biophysical models to real data is essential to achieve realistic simulations or generate relevant synthetic populations. However, some of these models involve randomness, which poses two challenges: they do not allow the standard personalization to each individual’s data and they lack an analytical formulation required for optimization. In previous work, we introduced a population-based personalization strategy which overcomes these challenges and demonstrated its feasibility on simple 2D geometrical models of myocardial infarct. The method consists in matching the distributions of the synthetic and real populations, quantified through the Kullback–Leibler (KL) divergence. Personalization is achieved with a gradient-free algorithm (CMA-ES), which generates sets of candidate solutions represented by their covariance matrix, whose coefficients evolve until the synthetic and real data are matched. However, the robustness of this strategy regarding settings and more complex data was not challenged. In this work, we specifically address these points, with (i) an improved design, (ii) a thorough evaluation on crucial aspects of the personalization process, including hyperparameters and initialization, and (iii) the application to 3D data. Despite some limits of the simple geometrical models used, our method is able to capture the main characteristics of the real data, as demonstrated both on 2D and 3D segmented late Gadolinium images of 123 subjects with acute myocardial infarction.</div></div>","PeriodicalId":54638,"journal":{"name":"Pattern Recognition Letters","volume":"188 ","pages":"Pages 8-14"},"PeriodicalIF":3.9,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142742881","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-22DOI: 10.1016/j.patrec.2024.11.018
Junyong Kang , Byeongho Heo , Junsuk Choe
Vision Transformers (ViTs) have demonstrated remarkable performances on various computer vision tasks. Attention scores are often used to explain the decision-making process of ViTs, showing which tokens are more important than others. However, the attention scores have several limitations as an explanation for ViT, such as conflicting with other explainable methods or highlighting unrelated tokens. In order to address this limitation, we propose a novel method for generating a visual explanation map from ViTs. Unlike previous approaches that rely on attention scores, our method leverages ViT features and conducts a single forward pass through our Patch-level Mask prediction (PM) module. Our visual explanation map provides class-dependent and probabilistic interpretation that can identify crucial regions of model decisions. Experimental results demonstrate that our approach outperforms previous techniques in both classification and interpretability aspects. Additionally, it can be applied to the weakly-supervised object localization (WSOL) tasks using pseudo mask labels. Our method requires no extra parameters and necessitates minimal locality supervision, utilizing less than 1% of the ImageNet-1k training dataset.
视觉转换器（ViTs）在各种计算机视觉任务中表现出了卓越的性能。注意力分数通常被用来解释 ViTs 的决策过程，显示哪些标记比其他标记更重要。然而，注意力分数作为对 ViT 的解释有一些局限性，例如与其他可解释方法相冲突或突出不相关的标记。为了解决这一局限性，我们提出了一种从 ViT 生成视觉解释图的新方法。与以往依赖注意力分数的方法不同，我们的方法利用了 ViT 的特征，并通过我们的补丁级掩码预测（PM）模块进行一次前向传递。我们的视觉解释图提供了与类别相关的概率解释，可以识别出模型决策的关键区域。实验结果表明，我们的方法在分类和可解释性方面都优于之前的技术。此外，它还可用于使用伪掩码标签的弱监督对象定位（WSOL）任务。我们的方法不需要额外的参数，只需最小的定位监督，使用不到 ImageNet-1k 训练数据集的 1%。
{"title":"Improving ViT interpretability with patch-level mask prediction","authors":"Junyong Kang , Byeongho Heo , Junsuk Choe","doi":"10.1016/j.patrec.2024.11.018","DOIUrl":"10.1016/j.patrec.2024.11.018","url":null,"abstract":"<div><div>Vision Transformers (ViTs) have demonstrated remarkable performances on various computer vision tasks. Attention scores are often used to explain the decision-making process of ViTs, showing which tokens are more important than others. However, the attention scores have several limitations as an explanation for ViT, such as conflicting with other explainable methods or highlighting unrelated tokens. In order to address this limitation, we propose a novel method for generating a visual explanation map from ViTs. Unlike previous approaches that rely on attention scores, our method leverages ViT features and conducts a single forward pass through our Patch-level Mask prediction (PM) module. Our visual explanation map provides class-dependent and probabilistic interpretation that can identify crucial regions of model decisions. Experimental results demonstrate that our approach outperforms previous techniques in both classification and interpretability aspects. Additionally, it can be applied to the weakly-supervised object localization (WSOL) tasks using pseudo mask labels. Our method requires no extra parameters and necessitates minimal locality supervision, utilizing less than 1% of the ImageNet-1k training dataset.</div></div>","PeriodicalId":54638,"journal":{"name":"Pattern Recognition Letters","volume":"187 ","pages":"Pages 73-79"},"PeriodicalIF":3.9,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142721864","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-19DOI: 10.1016/j.patrec.2024.11.010
Davide Talon , Alessio Del Bue , Stuart James
Jigsaw puzzles are a popular and enjoyable pastime that humans can easily solve, even with many pieces. However, solving a jigsaw is a combinatorial problem, and the space of possible solutions is exponential in the number of pieces, intractable for pairwise solutions. In contrast to the classical pairwise local matching of pieces based on edge heuristics, we estimate an approximate solution image, i.e., a mental image, of the puzzle and exploit it to guide the placement of pieces as a piece-to-global assignment problem. Therefore, from unordered pieces, we consider conditioned generation approaches, including Generative Adversarial Networks (GAN) models, Slot Attention (SA) and Vision Transformers (ViT), to recover the solution image. Given the generated solution representation, we cast the jigsaw solving as a 1-to-1 assignment matching problem using Hungarian attention, which places pieces in corresponding positions in the global solution estimate. Results show that the newly proposed GANzzle-SA and GANzzle-VIT benefit from the early fusion strategy where pieces are jointly compressed and gathered for global structure recovery. A single deep learning model generalizes to puzzles of different sizes and improves the performances by a large margin. Evaluated on PuzzleCelebA and PuzzleWikiArts, our approaches bridge the gap of deep learning strategies with respect to optimization-based classic puzzle solvers.
{"title":"GANzzle++: Generative approaches for jigsaw puzzle solving as local to global assignment in latent spatial representations","authors":"Davide Talon , Alessio Del Bue , Stuart James","doi":"10.1016/j.patrec.2024.11.010","DOIUrl":"10.1016/j.patrec.2024.11.010","url":null,"abstract":"<div><div>Jigsaw puzzles are a popular and enjoyable pastime that humans can easily solve, even with many pieces. However, solving a jigsaw is a combinatorial problem, and the space of possible solutions is exponential in the number of pieces, intractable for pairwise solutions. In contrast to the classical pairwise local matching of pieces based on edge heuristics, we estimate an approximate solution image, i.e., a <em>mental image</em>, of the puzzle and exploit it to guide the placement of pieces as a piece-to-global assignment problem. Therefore, from unordered pieces, we consider conditioned generation approaches, including Generative Adversarial Networks (GAN) models, Slot Attention (SA) and Vision Transformers (ViT), to recover the solution image. Given the generated solution representation, we cast the jigsaw solving as a 1-to-1 assignment matching problem using Hungarian attention, which places pieces in corresponding positions in the global solution estimate. Results show that the newly proposed GANzzle-SA and GANzzle-VIT benefit from the early fusion strategy where pieces are jointly compressed and gathered for global structure recovery. A single deep learning model generalizes to puzzles of different sizes and improves the performances by a large margin. Evaluated on PuzzleCelebA and PuzzleWikiArts, our approaches bridge the gap of deep learning strategies with respect to optimization-based classic puzzle solvers.</div></div>","PeriodicalId":54638,"journal":{"name":"Pattern Recognition Letters","volume":"187 ","pages":"Pages 35-41"},"PeriodicalIF":3.9,"publicationDate":"2024-11-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142699180","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号