Image and Vision Computing最新文献

英文中文

SAMNet: Adapting segment anything model for accurate light field salient object detection

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2025-02-01 DOI: 10.1016/j.imavis.2024.105403

Xingzheng Wang, Jianbin Wu, Shaoyong Wu, Jiahui Li

Light field salient object detection (LF SOD) is an important task that aims to segment visually salient objects from the surroundings. However, existing methods still struggle to achieve accurate detection, especially in complex scenes. Recently, segment anything model (SAM) excels in various vision tasks with its strong object segmentation ability and generalization capability, which is suitable for solving the LF SOD challenge. In this paper, we aim to adapt the SAM for accurate LF SOD. Specifically, we propose a network named SAMNet with two adaptation designs. Firstly, to enhance the perception of salient objects, we design a task-oriented multi-scale convolution adapter (MSCA) integrated into SAM’s image encoder. Parameters in the image encoder except MSCA are frozen to balance detection accuracy and computational requirements. Furthermore, to effectively utilize the rich scene information of LF data, we design a data-oriented cross-modal fusion module (CMFM) to fuse SAM features of different modalities. Comprehensive experiments on four benchmark datasets demonstrate the effectiveness of SAMNet over current state-of-the-art methods. In particular, SAMNet achieves the highest F-measures of 0.945, 0.819, 0.868, and 0.898, respectively. To the best of our knowledge, this is the first work that adapts a vision foundation model to LF SOD.

{"title":"SAMNet: Adapting segment anything model for accurate light field salient object detection","authors":"Xingzheng Wang, Jianbin Wu, Shaoyong Wu, Jiahui Li","doi":"10.1016/j.imavis.2024.105403","DOIUrl":"10.1016/j.imavis.2024.105403","url":null,"abstract":"<div><div>Light field salient object detection (LF SOD) is an important task that aims to segment visually salient objects from the surroundings. However, existing methods still struggle to achieve accurate detection, especially in complex scenes. Recently, segment anything model (SAM) excels in various vision tasks with its strong object segmentation ability and generalization capability, which is suitable for solving the LF SOD challenge. In this paper, we aim to adapt the SAM for accurate LF SOD. Specifically, we propose a network named SAMNet with two adaptation designs. Firstly, to enhance the perception of salient objects, we design a task-oriented multi-scale convolution adapter (MSCA) integrated into SAM’s image encoder. Parameters in the image encoder except MSCA are frozen to balance detection accuracy and computational requirements. Furthermore, to effectively utilize the rich scene information of LF data, we design a data-oriented cross-modal fusion module (CMFM) to fuse SAM features of different modalities. Comprehensive experiments on four benchmark datasets demonstrate the effectiveness of SAMNet over current state-of-the-art methods. In particular, SAMNet achieves the highest F-measures of 0.945, 0.819, 0.868, and 0.898, respectively. To the best of our knowledge, this is the first work that adapts a vision foundation model to LF SOD.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"154 ","pages":"Article 105403"},"PeriodicalIF":4.2,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143138488","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Privacy-preserving explainable AI enable federated learning-based denoising fingerprint recognition model

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2025-02-01 DOI: 10.1016/j.imavis.2025.105420

Haewon Byeon , Mohammed E. Seno , Divya Nimma , Janjhyam Venkata Naga Ramesh , Abdelhamid Zaidi , Azzah AlGhamdi , Ismail Keshta , Mukesh Soni , Mohammad Shabaz

Most existing fingerprint recognition methods are based on machine learning and often overlook the privacy and heterogeneity of data when training on large datasets, leading to user information leakage and decreased recognition accuracy. To collaboratively optimize model accuracy under privacy protection, a novel fingerprint recognition algorithm based on artificial intelligence enable federated learning-based Fingerprint Recognition, (AI-Fed-FR) is proposed. First, federated learning is used to iteratively aggregate parameters from various clients, thereby improving the performance of the global model. Second, Explainable AI is applied for denoising low-quality fingerprint images to enhance fingerprint texture structure. Third, to address the fairness issue caused by client heterogeneity, a client scheduling strategy based on reservoir sampling is proposed. Finally, simulation experiments are conducted on three real-world datasets to analyze the effectiveness of AI-Fed-FR. Experimental results show that AI-Fed-FR improves accuracy by 5.32% compared to local learning and by 8.56% compared to the federated averaging algorithm, achieving accuracy close to centralized learning. This study is the first to demonstrate the feasibility of combining federated learning with fingerprint recognition, enhancing the security and scalability of fingerprint recognition algorithms and providing a reference for the application of federated learning in biometric technologies.

{"title":"Privacy-preserving explainable AI enable federated learning-based denoising fingerprint recognition model","authors":"Haewon Byeon , Mohammed E. Seno , Divya Nimma , Janjhyam Venkata Naga Ramesh , Abdelhamid Zaidi , Azzah AlGhamdi , Ismail Keshta , Mukesh Soni , Mohammad Shabaz","doi":"10.1016/j.imavis.2025.105420","DOIUrl":"10.1016/j.imavis.2025.105420","url":null,"abstract":"<div><div>Most existing fingerprint recognition methods are based on machine learning and often overlook the privacy and heterogeneity of data when training on large datasets, leading to user information leakage and decreased recognition accuracy. To collaboratively optimize model accuracy under privacy protection, a novel fingerprint recognition algorithm based on artificial intelligence enable federated learning-based Fingerprint Recognition, (AI-Fed-FR) is proposed. First, federated learning is used to iteratively aggregate parameters from various clients, thereby improving the performance of the global model. Second, Explainable AI is applied for denoising low-quality fingerprint images to enhance fingerprint texture structure. Third, to address the fairness issue caused by client heterogeneity, a client scheduling strategy based on reservoir sampling is proposed. Finally, simulation experiments are conducted on three real-world datasets to analyze the effectiveness of AI-Fed-FR. Experimental results show that AI-Fed-FR improves accuracy by 5.32% compared to local learning and by 8.56% compared to the federated averaging algorithm, achieving accuracy close to centralized learning. This study is the first to demonstrate the feasibility of combining federated learning with fingerprint recognition, enhancing the security and scalability of fingerprint recognition algorithms and providing a reference for the application of federated learning in biometric technologies.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"154 ","pages":"Article 105420"},"PeriodicalIF":4.2,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143138667","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Image re-identification: Where self-supervision meets vision-language learning

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2025-02-01 DOI: 10.1016/j.imavis.2025.105415

Bin Wang , Yuying Liang , Lei Cai , Huakun Huang , Huanqiang Zeng

Recently, large-scale vision-language pre-trained models like CLIP have shown impressive performance in image re-identification (ReID). In this work, we explore whether self-supervision can aid in the use of CLIP for image ReID tasks. Specifically, we propose SVLL-ReID, the first attempt to integrate self-supervision and pre-trained CLIP via two training stages to facilitate the image ReID. We observe that: (1) incorporating language self-supervision in the first training stage can make the learnable text prompts more identity-specific, and (2) incorporating vision self-supervision in the second training stage can make the image features learned by the image encoder more discriminative. These observations imply that: (1) the text prompt learning in the first stage can benefit from the language self-supervision, and (2) the image feature learning in the second stage can benefit from the vision self-supervision. These benefits jointly facilitate the performance gain of the proposed SVLL-ReID. By conducting experiments on six image ReID benchmark datasets without any concrete text labels, we find that the proposed SVLL-ReID achieves the overall best performances compared with state-of-the-arts. Codes will be publicly available at https://github.com/BinWangGzhu/SVLL-ReID.

{"title":"Image re-identification: Where self-supervision meets vision-language learning","authors":"Bin Wang , Yuying Liang , Lei Cai , Huakun Huang , Huanqiang Zeng","doi":"10.1016/j.imavis.2025.105415","DOIUrl":"10.1016/j.imavis.2025.105415","url":null,"abstract":"<div><div>Recently, large-scale vision-language pre-trained models like CLIP have shown impressive performance in image re-identification (ReID). In this work, we explore whether self-supervision can aid in the use of CLIP for image ReID tasks. Specifically, we propose SVLL-ReID, the first attempt to integrate self-supervision and pre-trained CLIP via two training stages to facilitate the image ReID. We observe that: (1) incorporating <em>language self-supervision</em> in the first training stage can make the learnable text prompts more identity-specific, and (2) incorporating <em>vision self-supervision</em> in the second training stage can make the image features learned by the image encoder more discriminative. These observations imply that: (1) the text prompt learning in the first stage can benefit from the language self-supervision, and (2) the image feature learning in the second stage can benefit from the vision self-supervision. These benefits jointly facilitate the performance gain of the proposed SVLL-ReID. By conducting experiments on six image ReID benchmark datasets without any concrete text labels, we find that the proposed SVLL-ReID achieves the overall best performances compared with state-of-the-arts. Codes will be publicly available at <span><span>https://github.com/BinWangGzhu/SVLL-ReID</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"154 ","pages":"Article 105415"},"PeriodicalIF":4.2,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143138670","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

GANSD: A generative adversarial network based on saliency detection for infrared and visible image fusion

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2025-02-01 DOI: 10.1016/j.imavis.2024.105410

Yinghua Fu , Zhaofeng Liu , Jiansheng Peng , Rohit Gupta , Dawei Zhang

Image fusion technology, which integrates infrared images providing valuable contrast information with visible light images rich in texture details, represents an effective and rational approach for object detection and tracking. Previous methods have often neglected crucial information due to a lack of saliency detection and have failed to fully utilize complementary information by separately processing different features from the two original images. To address these limitations and enhance fusion techniques, we propose a generative adversarial network with saliency detection (GANSD) for image fusion through an adversarial process. This approach simplifies the design of fusion rules and improves the quality of fused images. By incorporating saliency detection, GANSD effectively preserves both foreground and background information from the input images. The architecture also integrates complementary information to prevent data loss from the input images. Simultaneously, an attention mechanism within the generator emphasizes the importance of different feature channels. Extensive experiments on two public datasets, TNO and Roadscene, demonstrate that GANSD provides both qualitative and quantitative advantages over nine state-of-the-art (SOTA) methods.

{"title":"GANSD: A generative adversarial network based on saliency detection for infrared and visible image fusion","authors":"Yinghua Fu , Zhaofeng Liu , Jiansheng Peng , Rohit Gupta , Dawei Zhang","doi":"10.1016/j.imavis.2024.105410","DOIUrl":"10.1016/j.imavis.2024.105410","url":null,"abstract":"<div><div>Image fusion technology, which integrates infrared images providing valuable contrast information with visible light images rich in texture details, represents an effective and rational approach for object detection and tracking. Previous methods have often neglected crucial information due to a lack of saliency detection and have failed to fully utilize complementary information by separately processing different features from the two original images. To address these limitations and enhance fusion techniques, we propose a generative adversarial network with saliency detection (GANSD) for image fusion through an adversarial process. This approach simplifies the design of fusion rules and improves the quality of fused images. By incorporating saliency detection, GANSD effectively preserves both foreground and background information from the input images. The architecture also integrates complementary information to prevent data loss from the input images. Simultaneously, an attention mechanism within the generator emphasizes the importance of different feature channels. Extensive experiments on two public datasets, TNO and Roadscene, demonstrate that GANSD provides both qualitative and quantitative advantages over nine state-of-the-art (SOTA) methods.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"154 ","pages":"Article 105410"},"PeriodicalIF":4.2,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143138673","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

AI-based intelligent hybrid framework (BO-DenseXGB) for multi- classification of brain tumor using MRI

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2025-02-01 DOI: 10.1016/j.imavis.2025.105417

Chandni , Monika Sachdeva , Alok Kumar Singh Kushwaha

A brain tumor is one of the most deadly tumors in the world and can affect both adults and children. According to its shape, severity, or region affected, it comes in different types or grades. The precise treatment strategy necessitates the early detection and classification of the correct type and grade of the tumor. Magnetic Resonance imaging (MRI) is the most extensively used medical imaging technique for examining tumors. The manual examination in clinical practices is constrained by the huge amount of data generated by MRI, which makes tumor classification challenging and time-consuming. Hence, automated methods are the need of the hour for precise and timely diagnosis. This paper proposes Artificial Intelligence (AI) based automated framework to classify tumors into meningioma, glioma, and pituitary classes. The proposed framework exploits the hierarchical feature learning capabilities of the Convolutional Neural Network (CNN) in combination with an optimized boosting classifier. Hyper-parameters of the boosting classifier are tuned with Bayesian Optimization. An overall accuracy of 99.02% is obtained during the experimental evaluation of the proposed model using the benchmark Figshare dataset, which comprises 3064 MRI images. The experimental outcomes confirm that the proposed deep learning strategy outperforms the existing approaches in a convincing manner.

{"title":"AI-based intelligent hybrid framework (BO-DenseXGB) for multi- classification of brain tumor using MRI","authors":"Chandni , Monika Sachdeva , Alok Kumar Singh Kushwaha","doi":"10.1016/j.imavis.2025.105417","DOIUrl":"10.1016/j.imavis.2025.105417","url":null,"abstract":"<div><div>A brain tumor is one of the most deadly tumors in the world and can affect both adults and children. According to its shape, severity, or region affected, it comes in different types or grades. The precise treatment strategy necessitates the early detection and classification of the correct type and grade of the tumor. Magnetic Resonance imaging (MRI) is the most extensively used medical imaging technique for examining tumors. The manual examination in clinical practices is constrained by the huge amount of data generated by MRI, which makes tumor classification challenging and time-consuming. Hence, automated methods are the need of the hour for precise and timely diagnosis. This paper proposes Artificial Intelligence (AI) based automated framework to classify tumors into meningioma, glioma, and pituitary classes. The proposed framework exploits the hierarchical feature learning capabilities of the Convolutional Neural Network (CNN) in combination with an optimized boosting classifier. Hyper-parameters of the boosting classifier are tuned with Bayesian Optimization. An overall accuracy of 99.02% is obtained during the experimental evaluation of the proposed model using the benchmark Figshare dataset, which comprises 3064 MRI images. The experimental outcomes confirm that the proposed deep learning strategy outperforms the existing approaches in a convincing manner.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"154 ","pages":"Article 105417"},"PeriodicalIF":4.2,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143139142","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

EPFDNet: Camouflaged object detection with edge perception in frequency domain

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2025-02-01 DOI: 10.1016/j.imavis.2024.105358

Xian Fang , Jiatong Chen , Yaming Wang , Mingfeng Jiang , Jianhua Ma , Xin Wang

Camouflaged object detection (COD) is a relatively new field of computer vision research. The challenge of this task lies in accurately segmenting camouflaged objects from backgrounds that are similar in appearance. In fact, the generation of reliable edges is an effective mean of distinguishing between the foreground and background of the image, which is beneficial for assisting in determining the location of camouflaged objects. Inspired by this, we design an Edge Encoder that decomposes features into different frequency bands adopting learnable wavelets and focuses on high-frequency components with sufficient edge details to extract accurate edges. Subsequently, the Feature Aggregation Module is proposed to integrate contextual features, which generates rough edge details by sensing the difference between two branch features and use this information to further refine our edge features. Furthermore, the Stage Enhancement Module is developed to enhance the features through reverse attention guidance and dilate convolution, which mines the detailed structural information of the camouflaged objects area by eliminating foreground. The superiority of our proposed method (EPFDNet) over the existing 17 state-of-the-art methods is demonstrated through extensive experiments on three widely used COD benchmark datasets. The code has been released at https://github.com/LitterMa-820/EPFDNet.

{"title":"EPFDNet: Camouflaged object detection with edge perception in frequency domain","authors":"Xian Fang , Jiatong Chen , Yaming Wang , Mingfeng Jiang , Jianhua Ma , Xin Wang","doi":"10.1016/j.imavis.2024.105358","DOIUrl":"10.1016/j.imavis.2024.105358","url":null,"abstract":"<div><div>Camouflaged object detection (COD) is a relatively new field of computer vision research. The challenge of this task lies in accurately segmenting camouflaged objects from backgrounds that are similar in appearance. In fact, the generation of reliable edges is an effective mean of distinguishing between the foreground and background of the image, which is beneficial for assisting in determining the location of camouflaged objects. Inspired by this, we design an Edge Encoder that decomposes features into different frequency bands adopting learnable wavelets and focuses on high-frequency components with sufficient edge details to extract accurate edges. Subsequently, the Feature Aggregation Module is proposed to integrate contextual features, which generates rough edge details by sensing the difference between two branch features and use this information to further refine our edge features. Furthermore, the Stage Enhancement Module is developed to enhance the features through reverse attention guidance and dilate convolution, which mines the detailed structural information of the camouflaged objects area by eliminating foreground. The superiority of our proposed method (EPFDNet) over the existing 17 state-of-the-art methods is demonstrated through extensive experiments on three widely used COD benchmark datasets. The code has been released at <span><span>https://github.com/LitterMa-820/EPFDNet</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"154 ","pages":"Article 105358"},"PeriodicalIF":4.2,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143138204","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Exploring underwater image quality: A review of current methodologies and emerging trends

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2025-02-01 DOI: 10.1016/j.imavis.2024.105389

Xiaoyi Xu , Hui Cai , Mingjie Wang , Weiling Chen , Rongxin Zhang , Tiesong Zhao

The complex underwater environment often leads to issues such as light scattering, color distortion, structural blurring, and noise interference in underwater images, hindering accurate scene representation. Numerous algorithms have been devised for underwater image recovery and enhancement, yet their outcomes exhibit significant variability. Thus, evaluating the quality of underwater images effectively is crucial for assessing these algorithms. This paper provides an overview of research on Underwater Image Quality Assessment (UIQA) by examining its methodologies, challenges, and future trends. Initially, the imaging principle of underwater images is introduced to summarize the primary factors affecting their quality. Subsequently, publicly available underwater image databases and UIQA methods are systematically classified and analyzed. Furthermore, extensive experimental comparisons are conducted to evaluate the performance of published quality assessment algorithms and discuss the relationship between perceived quality and utility in underwater images. Lastly, future trends in UIQA research are anticipated.

{"title":"Exploring underwater image quality: A review of current methodologies and emerging trends","authors":"Xiaoyi Xu , Hui Cai , Mingjie Wang , Weiling Chen , Rongxin Zhang , Tiesong Zhao","doi":"10.1016/j.imavis.2024.105389","DOIUrl":"10.1016/j.imavis.2024.105389","url":null,"abstract":"<div><div>The complex underwater environment often leads to issues such as light scattering, color distortion, structural blurring, and noise interference in underwater images, hindering accurate scene representation. Numerous algorithms have been devised for underwater image recovery and enhancement, yet their outcomes exhibit significant variability. Thus, evaluating the quality of underwater images effectively is crucial for assessing these algorithms. This paper provides an overview of research on Underwater Image Quality Assessment (UIQA) by examining its methodologies, challenges, and future trends. Initially, the imaging principle of underwater images is introduced to summarize the primary factors affecting their quality. Subsequently, publicly available underwater image databases and UIQA methods are systematically classified and analyzed. Furthermore, extensive experimental comparisons are conducted to evaluate the performance of published quality assessment algorithms and discuss the relationship between perceived quality and utility in underwater images. Lastly, future trends in UIQA research are anticipated.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"154 ","pages":"Article 105389"},"PeriodicalIF":4.2,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143138235","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Generative AI in the context of assistive technologies: Trends, limitations and future directions

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2025-02-01 DOI: 10.1016/j.imavis.2024.105347

Biying Fu , Abdenour Hadid , Naser Damer

With the tremendous successes of Large Language Models (LLMs) like ChatGPT for text generation and Dall-E for high-quality image generation, generative Artificial Intelligence (AI) models have shown a hype in our society. Generative AI seamlessly delved into different aspects of society ranging from economy, education, legislation, computer science, finance, and even healthcare. This article provides a comprehensive survey on the increased and promising use of generative AI in assistive technologies benefiting different parties, ranging from the assistive system developers, medical practitioners, care workforce, to the people who need the care and the comfort. Ethical concerns, biases, lack of transparency, insufficient explainability, and limited trustworthiness are major challenges when using generative AI in assistive technologies, particularly in systems that impact people directly. Key future research directions to address these issues include creating standardized rules, establishing commonly accepted evaluation metrics and benchmarks for explainability and reasoning processes, and making further advancements in understanding and reducing bias and its potential harms. Beyond showing the current trends of applying generative AI in the scope of assistive technologies in four identified key domains, which include care sectors, medical sectors, helping people in need, and co-working, the survey also discusses the current limitations and provides promising future research directions to foster better integration of generative AI in assistive technologies.

{"title":"Generative AI in the context of assistive technologies: Trends, limitations and future directions","authors":"Biying Fu , Abdenour Hadid , Naser Damer","doi":"10.1016/j.imavis.2024.105347","DOIUrl":"10.1016/j.imavis.2024.105347","url":null,"abstract":"<div><div>With the tremendous successes of Large Language Models (LLMs) like ChatGPT for text generation and Dall-E for high-quality image generation, generative Artificial Intelligence (AI) models have shown a hype in our society. Generative AI seamlessly delved into different aspects of society ranging from economy, education, legislation, computer science, finance, and even healthcare. This article provides a comprehensive survey on the increased and promising use of generative AI in assistive technologies benefiting different parties, ranging from the assistive system developers, medical practitioners, care workforce, to the people who need the care and the comfort. Ethical concerns, biases, lack of transparency, insufficient explainability, and limited trustworthiness are major challenges when using generative AI in assistive technologies, particularly in systems that impact people directly. Key future research directions to address these issues include creating standardized rules, establishing commonly accepted evaluation metrics and benchmarks for explainability and reasoning processes, and making further advancements in understanding and reducing bias and its potential harms. Beyond showing the current trends of applying generative AI in the scope of assistive technologies in four identified key domains, which include care sectors, medical sectors, helping people in need, and co-working, the survey also discusses the current limitations and provides promising future research directions to foster better integration of generative AI in assistive technologies.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"154 ","pages":"Article 105347"},"PeriodicalIF":4.2,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143138237","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Dual multi scale networks for medical image segmentation using contrastive learning

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2025-02-01 DOI: 10.1016/j.imavis.2024.105371

Akshat Dhamale , Ratnavel Rajalakshmi , Ananthakrishnan Balasundaram

DMSNet, a novel model for medical image segmentation is proposed in this research work. DMSNet employs a dual multi-scale architecture, combining the computational efficiency of EfficientNet B5 with the contextual understanding of the Pyramid Vision Transformer (PVT). Integration of a multi-scale module in both encoders enhances the model's capacity to capture intricate details across various resolutions, enabling precise delineation of complex foreground boundaries. Notably, DMSNet incorporates contrastive learning with a novel pixel-wise contrastive loss function during training, contributing to heightened segmentation accuracy and improved generalization capabilities. The model's performance is demonstrated through experimental evaluation on the four diverse datasets including Brain tumor segmentation (BraTS 2020), Diabetic Foot ulcer segmentation (DFU), Polyps (KVASIR-SEG) and Breast cancer segmentation (BCSS). We have employed recently introduced metrics to evaluate and compare our model with other state-of-the-art architectures. By advancing segmentation accuracy through innovative architectural design, multi-scale modules, and contrastive learning techniques, DMSNet represents a significant stride in the field, with potential implications for improved patient care and outcomes.

{"title":"Dual multi scale networks for medical image segmentation using contrastive learning","authors":"Akshat Dhamale , Ratnavel Rajalakshmi , Ananthakrishnan Balasundaram","doi":"10.1016/j.imavis.2024.105371","DOIUrl":"10.1016/j.imavis.2024.105371","url":null,"abstract":"<div><div>DMSNet, a novel model for medical image segmentation is proposed in this research work. DMSNet employs a dual multi-scale architecture, combining the computational efficiency of EfficientNet B5 with the contextual understanding of the Pyramid Vision Transformer (PVT). Integration of a multi-scale module in both encoders enhances the model's capacity to capture intricate details across various resolutions, enabling precise delineation of complex foreground boundaries. Notably, DMSNet incorporates contrastive learning with a novel pixel-wise contrastive loss function during training, contributing to heightened segmentation accuracy and improved generalization capabilities. The model's performance is demonstrated through experimental evaluation on the four diverse datasets including Brain tumor segmentation (BraTS 2020), Diabetic Foot ulcer segmentation (DFU), Polyps (KVASIR-SEG) and Breast cancer segmentation (BCSS). We have employed recently introduced metrics to evaluate and compare our model with other state-of-the-art architectures. By advancing segmentation accuracy through innovative architectural design, multi-scale modules, and contrastive learning techniques, DMSNet represents a significant stride in the field, with potential implications for improved patient care and outcomes.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"154 ","pages":"Article 105371"},"PeriodicalIF":4.2,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143138248","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Enhancing weakly supervised semantic segmentation with efficient and robust neighbor-attentive superpixel aggregation

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2025-02-01 DOI: 10.1016/j.imavis.2024.105391

Chen Wang , Huifang Ma , Di Zhang , Xiaolong Li , Zhixin Li

Image-level Weakly-Supervised Semantic Segmentation (WSSS) has become prominent as a technique that utilizes readily available image-level supervisory information. However, traditional methods that rely on pseudo-segmentation labels derived from Class Activation Maps (CAMs) are limited in terms of segmentation accuracy, primarily due to the incomplete nature of CAMs. Despite recent advancements in improving the comprehensiveness of CAM-derived pseudo-labels, challenges persist in handling ambiguity at object boundaries, and these methods also tend to be computationally intensive. To address these challenges, we propose a novel framework called Neighbor-Attentive Superpixel Aggregation (NASA). Inspired by the effectiveness of superpixel segmentation in homogenizing images through color and texture analysis, NASA enables the transformation from superpixel-wise to pixel-wise pseudo-labels. This approach significantly reduces semantic uncertainty at object boundaries and alleviates the computational overhead associated with direct pixel-wise label generation from CAMs. Besides, we introduce a superpixel augmentation strategy to enhance the model’s discrimination capabilities across different superpixels. Empirical studies demonstrate the superiority of NASA over existing WSSS methodologies. On the PASCAL VOC 2012 and MS COCO 2014 datasets, NASA achieves impressive mIoU scores of 73.5% and 46.4%, respectively.

{"title":"Enhancing weakly supervised semantic segmentation with efficient and robust neighbor-attentive superpixel aggregation","authors":"Chen Wang , Huifang Ma , Di Zhang , Xiaolong Li , Zhixin Li","doi":"10.1016/j.imavis.2024.105391","DOIUrl":"10.1016/j.imavis.2024.105391","url":null,"abstract":"<div><div>Image-level Weakly-Supervised Semantic Segmentation (WSSS) has become prominent as a technique that utilizes readily available image-level supervisory information. However, traditional methods that rely on pseudo-segmentation labels derived from Class Activation Maps (CAMs) are limited in terms of segmentation accuracy, primarily due to the incomplete nature of CAMs. Despite recent advancements in improving the comprehensiveness of CAM-derived pseudo-labels, challenges persist in handling ambiguity at object boundaries, and these methods also tend to be computationally intensive. To address these challenges, we propose a novel framework called Neighbor-Attentive Superpixel Aggregation (NASA). Inspired by the effectiveness of superpixel segmentation in homogenizing images through color and texture analysis, NASA enables the transformation from superpixel-wise to pixel-wise pseudo-labels. This approach significantly reduces semantic uncertainty at object boundaries and alleviates the computational overhead associated with direct pixel-wise label generation from CAMs. Besides, we introduce a superpixel augmentation strategy to enhance the model’s discrimination capabilities across different superpixels. Empirical studies demonstrate the superiority of NASA over existing WSSS methodologies. On the PASCAL VOC 2012 and MS COCO 2014 datasets, NASA achieves impressive mIoU scores of 73.5% and 46.4%, respectively.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"154 ","pages":"Article 105391"},"PeriodicalIF":4.2,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143138384","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

首页上一页

下一页尾页

类型

全部化学•材料生命科学医学物理工程技术环境•农林材料科学地球科学法学管理学化学环境科学与生态学计算机科学教育学经济学农林科学人文科学生物学数学物理与天体物理心理学综合性期刊其他工业工程理学历史学农学文学信息工程

数据库

全部 ACS Publications Elsevier ieeexplore Springer The Royal Society of Chemistry Wiley

期刊

Image and Vision Computing

全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.

﹀