Image and Vision Computing最新文献_第7页

A robust and secure video recovery scheme with deep compressive sensing 基于深度压缩感知的鲁棒安全视频恢复方案

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2026-02-01 Epub Date: 2025-12-01 DOI: 10.1016/j.imavis.2025.105853

Jagannath Sethi , Jaydeb Bhaumik , Ananda S. Chowdhury

In this paper, we propose a secure high quality video recovery scheme which can be useful for diverse applications like telemedicine and cloud-based surveillance. Our solution consists of deep learning-based video Compressive Sensing (CS) followed by a strategy for encrypting the compressed video. We split a video into a number of Groups Of Pictures (GOPs), where, each GOP consists of both keyframes and non-keyframes. The proposed video CS method uses a convolutional neural network (CNN) with a Structural Similarity Index Measure (SSIM) based loss function. Our recovery process has two stages. In the initial recovery stage, CNN is employed to make efficient use of spatial redundancy. In the deep recovery stage, non-keyframes are compensated by utilizing both keyframes and neighboring non-keyframes. Keyframes use multilevel feature compensation, and neighboring non-keyframes use single-level feature compensation. Additionally, we propose an unpredictable and complex chaotic map, with a broader chaotic range, termed as Sine Symbolic Chaotic Map (SSCM). For encrypting compressed features, we suggest a secure encryption scheme consisting of four operations: Forward Diffusion, Substitution, Backward Diffusion, and XORing with SSCM based chaotic sequence. Through extensive experimentation, we establish the efficacy of our combined solution over i) several state-of-the-art image and video CS methods, and ii) a number of video encryption techniques.

在本文中，我们提出了一种安全的高质量视频恢复方案，该方案可用于远程医疗和基于云的监控等多种应用。我们的解决方案包括基于深度学习的视频压缩感知（CS），然后是加密压缩视频的策略。我们将视频分成若干组图片（GOPs），其中每个GOP由关键帧和非关键帧组成。所提出的视频CS方法使用了卷积神经网络（CNN）和基于结构相似指数度量（SSIM）的损失函数。我们的恢复过程有两个阶段。在初始恢复阶段，采用CNN来有效利用空间冗余。在深度恢复阶段，利用关键帧和相邻的非关键帧对非关键帧进行补偿。关键帧使用多级特征补偿，相邻的非关键帧使用单级特征补偿。此外，我们提出了一种不可预测的复杂混沌映射，具有更广泛的混沌范围，称为正弦符号混沌映射（SSCM）。对于压缩特征的加密，我们提出了一种安全的加密方案，包括四种操作：前向扩散、替换、后向扩散和基于SSCM的混沌序列的XORing。通过广泛的实验，我们确定了我们的组合解决方案优于i)几种最先进的图像和视频CS方法，以及ii)许多视频加密技术的有效性。

{"title":"A robust and secure video recovery scheme with deep compressive sensing","authors":"Jagannath Sethi , Jaydeb Bhaumik , Ananda S. Chowdhury","doi":"10.1016/j.imavis.2025.105853","DOIUrl":"10.1016/j.imavis.2025.105853","url":null,"abstract":"<div><div>In this paper, we propose a secure high quality video recovery scheme which can be useful for diverse applications like telemedicine and cloud-based surveillance. Our solution consists of deep learning-based video Compressive Sensing (CS) followed by a strategy for encrypting the compressed video. We split a video into a number of Groups Of Pictures (GOPs), where, each GOP consists of both keyframes and non-keyframes. The proposed video CS method uses a convolutional neural network (CNN) with a Structural Similarity Index Measure (SSIM) based loss function. Our recovery process has two stages. In the initial recovery stage, CNN is employed to make efficient use of spatial redundancy. In the deep recovery stage, non-keyframes are compensated by utilizing both keyframes and neighboring non-keyframes. Keyframes use multilevel feature compensation, and neighboring non-keyframes use single-level feature compensation. Additionally, we propose an unpredictable and complex chaotic map, with a broader chaotic range, termed as Sine Symbolic Chaotic Map (SSCM). For encrypting compressed features, we suggest a secure encryption scheme consisting of four operations: Forward Diffusion, Substitution, Backward Diffusion, and XORing with SSCM based chaotic sequence. Through extensive experimentation, we establish the efficacy of our combined solution over i) several state-of-the-art image and video CS methods, and ii) a number of video encryption techniques.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"166 ","pages":"Article 105853"},"PeriodicalIF":4.2,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145685309","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Preserving instance-level characteristics for multi-instance generation 为多实例生成保留实例级特征

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2026-02-01 Epub Date: 2025-11-28 DOI: 10.1016/j.imavis.2025.105851

Jaehak Ryu , Sungwon Moon , Donghyeon Cho

Recently, there have been efforts to explore instance-level control in diffusion models, where multiple instances are generated independently and then integrated into a single scene. However, several issues arise when instances are closely positioned or overlapping. First, independently generated instances frequently differ in style and lack coherence, leading to changes in their attributes as they influence each other when merged. Second, instances often merge with one another or become absorbed into others. To tackle these challenges, we propose a local latent refinement (LLR) that enforces each local latent to meet its conditions and remain distinct from others. We also propose a local latent injection (LLI) method that gradually integrates local latents during global latent generation for smoother fusion. Also, we find that the variance of latents changes significantly after instance fusion, which greatly impacts the quality of the generated images. To remedy this, we apply an instance normalization layer to regulate the variance of the fused latents, thereby producing high-quality images. Extensive experiments demonstrate that our approach achieves both high fidelity in instance layout and superior image quality, even in cases of high overlap among instances.

最近，人们一直在努力探索扩散模型中的实例级控制，其中多个实例独立生成，然后集成到单个场景中。但是，当实例位置很近或重叠时，会出现几个问题。首先，独立生成的实例往往风格不同，缺乏连贯性，导致它们的属性发生变化，因为它们在合并时相互影响。第二，实例经常相互合并或被其他实例吸收。为了应对这些挑战，我们提出了一种局部潜在改进（LLR），强制每个局部潜在满足其条件并与其他潜在不同。我们还提出了一种局部潜注入（LLI）方法，该方法在全局潜生成过程中逐步整合局部潜，使融合更平滑。同时，我们发现在实例融合后，潜势的方差会发生很大的变化，这极大地影响了生成图像的质量。为了解决这个问题，我们应用实例归一化层来调节融合电位的方差，从而产生高质量的图像。大量的实验表明，即使在实例之间高度重叠的情况下，我们的方法也可以实现高保真的实例布局和优越的图像质量。

{"title":"Preserving instance-level characteristics for multi-instance generation","authors":"Jaehak Ryu , Sungwon Moon , Donghyeon Cho","doi":"10.1016/j.imavis.2025.105851","DOIUrl":"10.1016/j.imavis.2025.105851","url":null,"abstract":"<div><div>Recently, there have been efforts to explore instance-level control in diffusion models, where multiple instances are generated independently and then integrated into a single scene. However, several issues arise when instances are closely positioned or overlapping. First, independently generated instances frequently differ in style and lack coherence, leading to changes in their attributes as they influence each other when merged. Second, instances often merge with one another or become absorbed into others. To tackle these challenges, we propose a local latent refinement (LLR) that enforces each local latent to meet its conditions and remain distinct from others. We also propose a local latent injection (LLI) method that gradually integrates local latents during global latent generation for smoother fusion. Also, we find that the variance of latents changes significantly after instance fusion, which greatly impacts the quality of the generated images. To remedy this, we apply an instance normalization layer to regulate the variance of the fused latents, thereby producing high-quality images. Extensive experiments demonstrate that our approach achieves both high fidelity in instance layout and superior image quality, even in cases of high overlap among instances.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"166 ","pages":"Article 105851"},"PeriodicalIF":4.2,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145618474","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Enhancing skin cancer classification with Soft Attention and genetic algorithm-optimized ensemble learning 基于软注意和遗传算法优化的集成学习增强皮肤癌分类

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2026-02-01 Epub Date: 2025-12-03 DOI: 10.1016/j.imavis.2025.105848

Vibhav Ranjan, Kuldeep Chaurasia, Jagendra Singh

Skin cancer detection is a critical task in dermatology, where early diagnosis can significantly improve patient outcomes. In this work, we propose a novel approach for skin cancer classification that combines three deep learning models—InceptionResNetV2 with Soft Attention (SA), ResNet50V2 with SA, and DenseNet201—optimized using a Genetic Algorithm (GA) to find the best ensemble weights. The approach integrates several key innovations: Sigmoid Focal Cross-entropy Loss to address class imbalance, Mish activation for improved gradient flow, and Cosine Annealing learning rate scheduling for enhanced convergence. The GA-based optimization fine-tunes the ensemble weights to maximize classification performance, especially for challenging skin cancer types like melanoma. Experimental results on the HAM10000 dataset demonstrate the effectiveness of the proposed ensemble model, achieving superior accuracy and precision compared to individual models. This work offers a robust framework for skin cancer detection, combining state-of-the-art deep learning techniques with an optimization strategy.

皮肤癌检测是皮肤科的一项关键任务，早期诊断可以显著改善患者的预后。在这项工作中，我们提出了一种新的皮肤癌分类方法，该方法结合了三个深度学习模型——带有软注意（SA）的inception resnetv2、带有SA的ResNet50V2和使用遗传算法（GA）优化的densenet201，以找到最佳的集合权重。该方法集成了几个关键的创新：Sigmoid焦点交叉熵损失（Sigmoid Focal Cross-entropy Loss）来解决类别不平衡问题，Mish激活来改善梯度流，余弦退火（Cosine退火）学习率调度来增强收敛性。基于遗传算法的优化对集合权重进行微调，以最大限度地提高分类性能，特别是对于黑色素瘤等具有挑战性的皮肤癌类型。在HAM10000数据集上的实验结果证明了该集成模型的有效性，与单个模型相比具有更高的准确度和精度。这项工作为皮肤癌检测提供了一个强大的框架，结合了最先进的深度学习技术和优化策略。

{"title":"Enhancing skin cancer classification with Soft Attention and genetic algorithm-optimized ensemble learning","authors":"Vibhav Ranjan, Kuldeep Chaurasia, Jagendra Singh","doi":"10.1016/j.imavis.2025.105848","DOIUrl":"10.1016/j.imavis.2025.105848","url":null,"abstract":"<div><div>Skin cancer detection is a critical task in dermatology, where early diagnosis can significantly improve patient outcomes. In this work, we propose a novel approach for skin cancer classification that combines three deep learning models—InceptionResNetV2 with Soft Attention (SA), ResNet50V2 with SA, and DenseNet201—optimized using a Genetic Algorithm (GA) to find the best ensemble weights. The approach integrates several key innovations: Sigmoid Focal Cross-entropy Loss to address class imbalance, Mish activation for improved gradient flow, and Cosine Annealing learning rate scheduling for enhanced convergence. The GA-based optimization fine-tunes the ensemble weights to maximize classification performance, especially for challenging skin cancer types like melanoma. Experimental results on the HAM10000 dataset demonstrate the effectiveness of the proposed ensemble model, achieving superior accuracy and precision compared to individual models. This work offers a robust framework for skin cancer detection, combining state-of-the-art deep learning techniques with an optimization strategy.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"166 ","pages":"Article 105848"},"PeriodicalIF":4.2,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145737138","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

On the relevance of patch-based extraction methods for monocular depth estimation 基于斑块的提取方法在单目深度估计中的相关性研究

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2026-02-01 Epub Date: 2025-12-03 DOI: 10.1016/j.imavis.2025.105857

Pasquale Coscia, Antonio Fusillo, Angelo Genovese, Vincenzo Piuri, Fabio Scotti

Scene geometry estimation from images plays a key role in robotics, augmented reality, and autonomous systems. In particular, Monocular Depth Estimation (MDE) focuses on predicting depth using a single RGB image, avoiding the need for expensive sensors. State-of-the-art approaches use deep learning models for MDE while processing images as a whole, sub-optimally exploiting their spatial information. A recent research direction focuses on smaller image patches, as depth information varies across different regions of an image. This approach reduces model complexity and improves performance by capturing finer spatial details. From this perspective, we propose a novel warp patch-based extraction method which corrects perspective camera distortions, and employ it in tailored training and inference pipelines. Our experimental results show that our patch-based approach outperforms its full-image-trained counterpart and the classical crop patch-based extraction. With our technique, we obtain a general performance enhancements over recent state-of-the-art models. Code is available at https://github.com/AntonioFusillo/PatchMDE.

基于图像的场景几何估计在机器人、增强现实和自主系统中起着关键作用。特别是，单目深度估计（MDE）侧重于使用单个RGB图像预测深度，避免了对昂贵传感器的需求。最先进的方法使用深度学习模型进行MDE，同时整体处理图像，次优地利用其空间信息。最近的研究方向集中在较小的图像补丁上，因为图像的不同区域的深度信息不同。这种方法通过捕获更精细的空间细节来降低模型复杂性并提高性能。从这个角度来看，我们提出了一种新的基于翘曲补丁的提取方法，该方法可以纠正透视相机的畸变，并将其应用于定制的训练和推理管道中。实验结果表明，基于斑块的方法优于基于全图像训练的方法和经典的基于作物斑块的提取方法。使用我们的技术，我们获得了比最新的最先进的模型的一般性能增强。代码可从https://github.com/AntonioFusillo/PatchMDE获得。

{"title":"On the relevance of patch-based extraction methods for monocular depth estimation","authors":"Pasquale Coscia, Antonio Fusillo, Angelo Genovese, Vincenzo Piuri, Fabio Scotti","doi":"10.1016/j.imavis.2025.105857","DOIUrl":"10.1016/j.imavis.2025.105857","url":null,"abstract":"<div><div>Scene geometry estimation from images plays a key role in robotics, augmented reality, and autonomous systems. In particular, Monocular Depth Estimation (MDE) focuses on predicting depth using a single RGB image, avoiding the need for expensive sensors. State-of-the-art approaches use deep learning models for MDE while processing images as a whole, sub-optimally exploiting their spatial information. A recent research direction focuses on smaller image patches, as depth information varies across different regions of an image. This approach reduces model complexity and improves performance by capturing finer spatial details. From this perspective, we propose a novel warp patch-based extraction method which corrects perspective camera distortions, and employ it in tailored training and inference pipelines. Our experimental results show that our patch-based approach outperforms its full-image-trained counterpart and the classical crop patch-based extraction. With our technique, we obtain a general performance enhancements over recent state-of-the-art models. Code is available at <span><span>https://github.com/AntonioFusillo/PatchMDE</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"166 ","pages":"Article 105857"},"PeriodicalIF":4.2,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145737139","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

SEAGNet: Spatial–Epipolar–Angular–Global feature learning for light field super-resolution 面向光场超分辨率的空间-极-角-全局特征学习

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2026-02-01 Epub Date: 2025-12-13 DOI: 10.1016/j.imavis.2025.105866

Xingzheng Wang, Haotian Zhang, Yuhang Lin, Yuanbo Huang, Jiahao Lin

In light field (LF) image super-resolution (SR), comprehensive learning of LF information is crucial for accurately recovering image details. Because 4D LF structures are so complex, current methods usually use special convolutions and modules to separately extract different LF characteristics (like spatial, angular, and EPI features) before combining them. But these methods focus too much on local LF information and not enough on global 4D LF features. This makes it hard for them to get better. To overcome this issue, we suggest a straightforward yet effective Global Feature Extraction Module (GFEM). This module extracts the global information from the entire 4D light field. Our method does this by using all of these features together. We also introduce a tool called the Progressive Angular Feature Extractor (PAFE), which gradually expands the area that extracts features to make sure it can extract features at different angles. We also designed a Spatial Gated Feed-forward Network (SGFN) to replace the standard feed-forward network in Transformer. This has resulted in our new Intra-Transformer architecture, which optimizes feature flow and enhances local detail extraction. We did a lot of experiments on different public datasets, and these showed that our method is better than other methods that are currently available.

在光场图像超分辨率（SR）中，全面学习光场信息是准确恢复图像细节的关键。由于4D LF结构非常复杂，目前的方法通常使用特殊的卷积和模块分别提取不同的LF特征（如空间特征、角度特征和EPI特征），然后再组合。但这些方法过于关注局部LF信息，而对全局4D LF特征关注不够。这使得他们很难变得更好。为了克服这个问题，我们提出了一个简单而有效的全局特征提取模块（GFEM）。该模块从整个4D光场中提取全局信息。我们的方法通过将所有这些特征结合在一起来实现这一点。我们还引入了渐进式角度特征提取器（Progressive Angular Feature Extractor， PAFE）工具，它逐步扩大提取特征的区域，以确保能够提取不同角度的特征。我们还设计了一个空间门控前馈网络（SGFN）来取代变压器中的标准前馈网络。这导致了我们新的Intra-Transformer架构，它优化了特征流并增强了局部细节提取。我们在不同的公共数据集上做了大量的实验，这些实验表明我们的方法比目前可用的其他方法更好。

{"title":"SEAGNet: Spatial–Epipolar–Angular–Global feature learning for light field super-resolution","authors":"Xingzheng Wang, Haotian Zhang, Yuhang Lin, Yuanbo Huang, Jiahao Lin","doi":"10.1016/j.imavis.2025.105866","DOIUrl":"10.1016/j.imavis.2025.105866","url":null,"abstract":"<div><div>In light field (LF) image super-resolution (SR), comprehensive learning of LF information is crucial for accurately recovering image details. Because 4D LF structures are so complex, current methods usually use special convolutions and modules to separately extract different LF characteristics (like spatial, angular, and EPI features) before combining them. But these methods focus too much on local LF information and not enough on global 4D LF features. This makes it hard for them to get better. To overcome this issue, we suggest a straightforward yet effective Global Feature Extraction Module (GFEM). This module extracts the global information from the entire 4D light field. Our method does this by using all of these features together. We also introduce a tool called the Progressive Angular Feature Extractor (PAFE), which gradually expands the area that extracts features to make sure it can extract features at different angles. We also designed a Spatial Gated Feed-forward Network (SGFN) to replace the standard feed-forward network in Transformer. This has resulted in our new Intra-Transformer architecture, which optimizes feature flow and enhances local detail extraction. We did a lot of experiments on different public datasets, and these showed that our method is better than other methods that are currently available.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"166 ","pages":"Article 105866"},"PeriodicalIF":4.2,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145790255","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multi-modal cooperative fusion network for dual-stream RGB-D salient object detection 双流RGB-D显著目标检测的多模态协同融合网络

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2026-02-01 Epub Date: 2025-11-19 DOI: 10.1016/j.imavis.2025.105835

Jingyu Wu , Fuming Sun , Haojie Li , Mingyu Lu

Most existing RGB-D salient object detection tasks use convolution operations to design complex fusion modules for cross-modal information fusion. How to correctly integrate RGB and depth features into multi-modal features is important to salient object detection (SOD). Due to the differences between different modal features, the salient object detection model is seriously hindered in achieving better performance. To address the issues mentioned above, we design a multi-modal cooperative fusion network (MCFNet) to achieve RGB-D SOD. Firstly, we propose an edge feature refinement module to remove interference information in shallow features and improve the edge accuracy of SOD. Secondly, a depth optimization module is designed to optimize erroneous estimates in the depth maps, which effectively reduces the impact of noise and improves the performance of the model. Finally, we construct a progressive fusion module that gradually integrates RGB and depth features in a layered manner to achieve an efficient fusion of cross-modal features. Experimental results on six datasets show that our MCFNet performs better than other state-of-the-art (SOTA) methods, which provide new ideas for salient object detection tasks.

现有的RGB-D显著目标检测任务大多采用卷积运算来设计复杂的融合模块，实现跨模态信息融合。如何正确地将RGB和深度特征整合到多模态特征中，对于显著目标检测（SOD）具有重要意义。由于不同模态特征之间存在差异，严重阻碍了显著目标检测模型获得更好的性能。为了解决上述问题，我们设计了一个多模态协同融合网络（MCFNet）来实现RGB-D SOD。首先，提出边缘特征细化模块，去除浅层特征中的干扰信息，提高SOD边缘精度；其次，设计深度优化模块，对深度图中的错误估计进行优化，有效降低了噪声的影响，提高了模型的性能；最后，我们构建了一个递进融合模块，以分层的方式逐步融合RGB和深度特征，以实现高效的跨模态特征融合。在六个数据集上的实验结果表明，我们的MCFNet比其他最先进的方法（SOTA）性能更好，为显著目标检测任务提供了新的思路。

{"title":"Multi-modal cooperative fusion network for dual-stream RGB-D salient object detection","authors":"Jingyu Wu , Fuming Sun , Haojie Li , Mingyu Lu","doi":"10.1016/j.imavis.2025.105835","DOIUrl":"10.1016/j.imavis.2025.105835","url":null,"abstract":"<div><div>Most existing RGB-D salient object detection tasks use convolution operations to design complex fusion modules for cross-modal information fusion. How to correctly integrate RGB and depth features into multi-modal features is important to salient object detection (SOD). Due to the differences between different modal features, the salient object detection model is seriously hindered in achieving better performance. To address the issues mentioned above, we design a multi-modal cooperative fusion network (MCFNet) to achieve RGB-D SOD. Firstly, we propose an edge feature refinement module to remove interference information in shallow features and improve the edge accuracy of SOD. Secondly, a depth optimization module is designed to optimize erroneous estimates in the depth maps, which effectively reduces the impact of noise and improves the performance of the model. Finally, we construct a progressive fusion module that gradually integrates RGB and depth features in a layered manner to achieve an efficient fusion of cross-modal features. Experimental results on six datasets show that our MCFNet performs better than other state-of-the-art (SOTA) methods, which provide new ideas for salient object detection tasks.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"166 ","pages":"Article 105835"},"PeriodicalIF":4.2,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145685264","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Attention and mask-guided context fusion network for camouflaged object detection 用于伪装目标检测的注意力和面具引导上下文融合网络

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2026-02-01 Epub Date: 2025-12-22 DOI: 10.1016/j.imavis.2025.105887

Qiuying Han , Shaohui Zhang , Peng Wang

Camouflaged Object Detection (COD) aims to accurately identify objects visually embedded in their surroundings, which is considerably more challenging than conventional object detection due to low contrast, complex backgrounds, and varying object scales. Although recent deep learning approaches have shown promising results, they often suffer from incomplete or inaccurate detections, primarily due to the inadequate exploitation of multi-scale contextual features and cross-level information. To address these limitations, we propose a novel architecture, termed Attention and Mask-guided Context Fusion Network (AMCFNet). The framework comprises two core modules: Attentional Multi-scale Context Aggregation (AMCA) and Mask-guided Cross-level Fusion (MCF). The AMCA module improves the semantic representation of features at different levels by merging both global and local context information through a bidirectional attention mechanism, which includes wavelet-based modulation of channel and spatial data. The MCF module leverages high-level mask priors to guide the fusion of semantic and spatial features, applying an attention-weighted mechanism to highlight object-related regions while minimizing background interference. Comprehensive tests on four well-known COD benchmark datasets show that AMCFNet outperforms existing methods, providing more accurate camouflaged object detection under various challenging conditions.

伪装目标检测（COD）旨在准确识别视觉上嵌入周围环境的目标，由于低对比度、复杂背景和不同的目标尺度，这比传统的目标检测更具挑战性。尽管最近的深度学习方法已经显示出有希望的结果，但它们经常遭受不完整或不准确的检测，主要是由于对多尺度上下文特征和跨层信息的利用不足。为了解决这些限制，我们提出了一种新的架构，称为注意力和面具引导的上下文融合网络（AMCFNet）。该框架包括两个核心模块：注意多尺度上下文聚合（AMCA）和掩码引导跨层融合（MCF）。AMCA模块通过双向注意机制（包括基于小波的信道和空间数据调制）合并全局和局部上下文信息，提高了不同层次特征的语义表示。MCF模块利用高级掩模先验来指导语义和空间特征的融合，应用注意力加权机制来突出目标相关区域，同时最大限度地减少背景干扰。在四个知名COD基准数据集上的综合测试表明，AMCFNet优于现有方法，在各种具有挑战性的条件下提供更准确的伪装目标检测。

{"title":"Attention and mask-guided context fusion network for camouflaged object detection","authors":"Qiuying Han , Shaohui Zhang , Peng Wang","doi":"10.1016/j.imavis.2025.105887","DOIUrl":"10.1016/j.imavis.2025.105887","url":null,"abstract":"<div><div>Camouflaged Object Detection (COD) aims to accurately identify objects visually embedded in their surroundings, which is considerably more challenging than conventional object detection due to low contrast, complex backgrounds, and varying object scales. Although recent deep learning approaches have shown promising results, they often suffer from incomplete or inaccurate detections, primarily due to the inadequate exploitation of multi-scale contextual features and cross-level information. To address these limitations, we propose a novel architecture, termed Attention and Mask-guided Context Fusion Network (AMCFNet). The framework comprises two core modules: Attentional Multi-scale Context Aggregation (AMCA) and Mask-guided Cross-level Fusion (MCF). The AMCA module improves the semantic representation of features at different levels by merging both global and local context information through a bidirectional attention mechanism, which includes wavelet-based modulation of channel and spatial data. The MCF module leverages high-level mask priors to guide the fusion of semantic and spatial features, applying an attention-weighted mechanism to highlight object-related regions while minimizing background interference. Comprehensive tests on four well-known COD benchmark datasets show that AMCFNet outperforms existing methods, providing more accurate camouflaged object detection under various challenging conditions.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"166 ","pages":"Article 105887"},"PeriodicalIF":4.2,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145840003","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

MOT-STM: Maritime Object Tracking: A Spatial-Temporal and Metadata-based approach 海事目标跟踪：一种基于时空和元数据的方法

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2026-01-01 Epub Date: 2025-11-12 DOI: 10.1016/j.imavis.2025.105826

Vinayak S. Nageli , Arshad Jamal , Puneet Goyal , Rama Krishna Sai S Gorthi

Object Tracking and Re-Identification (Re-ID) in maritime environments using drone video streams presents significant challenges, especially in search and rescue operations. These challenges mainly arise from the small size of objects from high drone altitudes, sudden movements of the drone’s gimbal and limited appearance diversity of objects. The frequent occlusion in these challenging conditions makes Re-ID difficult in long-term tracking.

In this work, we present a novel framework, Maritime Object Tracking with Spatial–Temporal and Metadata-based modeling (MOT-STM), designed for robust tracking and re-identification of maritime objects in challenging environments. The proposed framework adapts multi-resolution spatial feature extraction using Cross-Stage Partial with Full-Stage (C2FDark) backbone combined with temporal modeling via Video Swin Transformer (VST), enabling effective spatio-temporal representation. This design enhances detection and significantly improves tracking performance in the maritime domain.

We also propose a metadata-driven Re-ID module named Metadata-Assisted Re-ID (MARe-ID), which leverages drone’s metadata such as Global Positioning System (GPS) coordinates, altitude and camera orientation to enhance long-term tracking. Unlike traditional appearance-based Re-ID, MARe-ID remains effective even in scenarios with limited visual diversity among the tracked objects and is generic enough to be integrated into any State-of-the-Art (SotA) multi-object tracking framework as a Re-ID module.

Through extensive experiments on the challenging SeaDronesSee dataset, we demonstrate that MOT-STM significantly outperforms existing methods in maritime object tracking. Our approach achieves a state-of-the-art performance attaining a HOTA score of 70.14% and an IDF1 score of 88.70%, showing the effectiveness and robustness of the proposed MOT-STM framework.

在海上环境中使用无人机视频流进行目标跟踪和重新识别（Re-ID）提出了重大挑战，特别是在搜索和救援行动中。这些挑战主要来自无人机高度的小尺寸物体，无人机框架的突然运动以及物体的有限外观多样性。在这些具有挑战性的条件下，频繁的遮挡使得Re-ID难以长期跟踪。在这项工作中，我们提出了一个新的框架，即基于时空和元数据建模的海事目标跟踪（MOT-STM），旨在在具有挑战性的环境中对海事目标进行鲁棒跟踪和重新识别。该框架采用跨阶段部分与全阶段（C2FDark）主干网进行多分辨率空间特征提取，并结合视频Swin变压器（VST）进行时间建模，实现了有效的时空表征。该设计增强了探测能力，显著提高了海事领域的跟踪性能。我们还提出了一种元数据驱动的Re-ID模块，称为元数据辅助Re-ID (MARe-ID)，它利用无人机的元数据，如全球定位系统（GPS）坐标、高度和相机方向来增强长期跟踪。与传统的基于外观的Re-ID不同，MARe-ID即使在被跟踪对象之间的视觉多样性有限的情况下仍然有效，并且足够通用，可以作为Re-ID模块集成到任何最先进的（SotA）多目标跟踪框架中。通过在具有挑战性的SeaDronesSee数据集上进行大量实验，我们证明了MOT-STM在海上目标跟踪方面明显优于现有方法。我们的方法达到了最先进的性能，HOTA得分为70.14%，IDF1得分为88.70%，显示了所提出的MOT-STM框架的有效性和鲁棒性。

{"title":"MOT-STM: Maritime Object Tracking: A Spatial-Temporal and Metadata-based approach","authors":"Vinayak S. Nageli , Arshad Jamal , Puneet Goyal , Rama Krishna Sai S Gorthi","doi":"10.1016/j.imavis.2025.105826","DOIUrl":"10.1016/j.imavis.2025.105826","url":null,"abstract":"<div><div>Object Tracking and Re-Identification (Re-ID) in maritime environments using drone video streams presents significant challenges, especially in search and rescue operations. These challenges mainly arise from the small size of objects from high drone altitudes, sudden movements of the drone’s gimbal and limited appearance diversity of objects. The frequent occlusion in these challenging conditions makes Re-ID difficult in long-term tracking.</div><div>In this work, we present a novel framework, Maritime Object Tracking with Spatial–Temporal and Metadata-based modeling (MOT-STM), designed for robust tracking and re-identification of maritime objects in challenging environments. The proposed framework adapts multi-resolution spatial feature extraction using Cross-Stage Partial with Full-Stage (C2FDark) backbone combined with temporal modeling via Video Swin Transformer (VST), enabling effective spatio-temporal representation. This design enhances detection and significantly improves tracking performance in the maritime domain.</div><div>We also propose a metadata-driven Re-ID module named Metadata-Assisted Re-ID (MARe-ID), which leverages drone’s metadata such as Global Positioning System (GPS) coordinates, altitude and camera orientation to enhance long-term tracking. Unlike traditional appearance-based Re-ID, MARe-ID remains effective even in scenarios with limited visual diversity among the tracked objects and is generic enough to be integrated into any State-of-the-Art (SotA) multi-object tracking framework as a Re-ID module.</div><div>Through extensive experiments on the challenging SeaDronesSee dataset, we demonstrate that MOT-STM significantly outperforms existing methods in maritime object tracking. Our approach achieves a state-of-the-art performance attaining a HOTA score of 70.14% and an IDF1 score of 88.70%, showing the effectiveness and robustness of the proposed MOT-STM framework.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"165 ","pages":"Article 105826"},"PeriodicalIF":4.2,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145520880","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

FaceMINT: A library for gaining insights into biometric face recognition via mechanistic interpretability FaceMINT：一个通过机制可解释性获得生物特征面部识别见解的库

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2026-01-01 Epub Date: 2025-11-07 DOI: 10.1016/j.imavis.2025.105804

Peter Rot , Robert Jutreša , Peter Peer , Vitomir Štruc , Walter Scheirer , Klemen Grm

Deep-learning models, including those used in biometric recognition, have achieved remarkable performance on benchmark datasets as well as real-world recognition tasks. However, a major drawback of these models is their lack of transparency in decision-making. Mechanistic interpretability has emerged as a promising research field intended to help us gain insights into such models, but its application to biometric data remains limited. In this work, we bridge this gap by introducing the FaceMINT library, a publicly available Python library (build on top of Pytorch) that enables biometric researchers to inspect their models through mechanistic interpretability. It provides a plug-and-play solution that allows researchers to seamlessly switch between the analyzed biometric models, evaluate state-of-the-art sparse autoencoders, select from various image parametrizations, and fine-tune hyperparameters. Using a large scale Glint360K dataset, we demonstrate the usability of FaceMINT by applying its functionality to two state-of-the-art (deep-learning) face recognition models: AdaFace, based on Convolutional Neural Networks (CNN), and SwinFace, based on transformers. The proposed library implements various sparse auto-encoders (SAEs), including vanilla SAE, Gated SAE, JumpReLU SAE, and TopK SAE, which have achieved state-of-the-art results in the mechanistic interpretability of large language models. Our study highlights the promise of mechanistic interpretability in the biometric field, providing new avenues for researchers to explore model transparency and refine biometric recognition systems. The library is publicly available at www.gitlab.com/peterrot/facemint.

深度学习模型，包括那些用于生物识别的模型，已经在基准数据集和现实世界的识别任务上取得了显着的性能。然而，这些模型的一个主要缺点是决策缺乏透明度。机械可解释性已成为一个有前途的研究领域，旨在帮助我们深入了解这些模型，但其在生物识别数据中的应用仍然有限。在这项工作中，我们通过引入FaceMINT库来弥合这一差距，FaceMINT库是一个公开可用的Python库（建立在Pytorch之上），它使生物识别研究人员能够通过机械可解释性来检查他们的模型。它提供了一个即插即用的解决方案，允许研究人员在分析的生物识别模型之间无缝切换，评估最先进的稀疏自编码器，从各种图像参数化中进行选择，并微调超参数。使用大规模的Glint360K数据集，我们通过将其功能应用于两种最先进的（深度学习）面部识别模型来展示FaceMINT的可用性：基于卷积神经网络（CNN）的adface和基于变压器的SwinFace。提出的库实现了各种稀疏自编码器（SAE），包括vanilla SAE、Gated SAE、JumpReLU SAE和TopK SAE，它们在大型语言模型的机制可解释性方面取得了最先进的成果。我们的研究强调了生物识别领域机制可解释性的前景，为研究人员探索模型透明度和改进生物识别系统提供了新的途径。该图书馆可在www.gitlab.com/peterrot/facemint上公开访问。

{"title":"FaceMINT: A library for gaining insights into biometric face recognition via mechanistic interpretability","authors":"Peter Rot , Robert Jutreša , Peter Peer , Vitomir Štruc , Walter Scheirer , Klemen Grm","doi":"10.1016/j.imavis.2025.105804","DOIUrl":"10.1016/j.imavis.2025.105804","url":null,"abstract":"<div><div>Deep-learning models, including those used in biometric recognition, have achieved remarkable performance on benchmark datasets as well as real-world recognition tasks. However, a major drawback of these models is their lack of transparency in decision-making. Mechanistic interpretability has emerged as a promising research field intended to help us gain insights into such models, but its application to biometric data remains limited. In this work, we bridge this gap by introducing the FaceMINT library, a publicly available Python library (build on top of Pytorch) that enables biometric researchers to inspect their models through mechanistic interpretability. It provides a plug-and-play solution that allows researchers to seamlessly switch between the analyzed biometric models, evaluate state-of-the-art sparse autoencoders, select from various image parametrizations, and fine-tune hyperparameters. Using a large scale Glint360K dataset, we demonstrate the usability of FaceMINT by applying its functionality to two state-of-the-art (deep-learning) face recognition models: AdaFace, based on Convolutional Neural Networks (CNN), and SwinFace, based on transformers. The proposed library implements various sparse auto-encoders (SAEs), including vanilla SAE, Gated SAE, JumpReLU SAE, and TopK SAE, which have achieved state-of-the-art results in the mechanistic interpretability of large language models. Our study highlights the promise of mechanistic interpretability in the biometric field, providing new avenues for researchers to explore model transparency and refine biometric recognition systems. The library is publicly available at <span><span>www.gitlab.com/peterrot/facemint</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"165 ","pages":"Article 105804"},"PeriodicalIF":4.2,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145521234","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Enhanced skeleton-based Group Activity Recognition through spatio-temporal graph convolution with cross-dimensional attention 基于跨维注意的时空图卷积增强基于骨骼的群体活动识别

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2026-01-01 Epub Date: 2025-10-30 DOI: 10.1016/j.imavis.2025.105784

Dongli Wang , Yongcan Weng , Xiaolin Zhu , Yan Zhou , Zixin Zhang , Richard Irampaye

Group Activity Recognition is a pivotal task in video understanding, with broad applications ranging from surveillance to human–computer interaction. Traditional RGB-based methods face challenges such as privacy concerns, environmental sensitivity, and fragmented scene-level semantic understanding. Skeleton-based approaches offer a promising alternative but often suffer from limited exploration of heterogeneous features and the absence of explicit modeling for human-object interactions. In this paper, we introduce a lightweight framework for skeleton-based GAR, leveraging an attention-enhanced spatio-temporal graph convolutional network. Specially, we first decouple joint and bone features along with their motion patterns, constructing a global human-object relational graph using an attention graph convolution module (AGCM). Additionally, we incorporate a Multi-Scale Temporal Convolution Module (MTC) and a Cross-Dimensional Attention Module (CDAM) to dynamically focus on key spatio-temporal nodes and feature channels. Our method achieves significant improvements in accuracy while maintaining high computational efficiency, making it suitable for real-time applications in privacy-sensitive scenarios. Experiments on the Volleyball and NBA datasets demonstrate that our method achieves competitive performance using only skeleton input, significantly reducing parameters and computational cost compared to mainstream approaches. Here, our method show an improvement in Multi-Class Per-Class Accuracy (MPCA) to 96.1% on the Volleyball dataset and 71.6% on the NBA dataset, offering a lightweight and efficient solution for GAR in privacy-sensitive scenarios.

群体活动识别是视频理解中的一项关键任务，具有广泛的应用范围，从监控到人机交互。传统的基于rgb的方法面临着隐私问题、环境敏感性和碎片化场景级语义理解等挑战。基于骨架的方法提供了一个很有前途的选择，但往往受到对异构特征的有限探索和缺乏对人-物交互的显式建模的影响。在本文中，我们为基于骨架的GAR引入了一个轻量级框架，利用注意力增强的时空图卷积网络。特别地，我们首先解耦关节和骨骼特征及其运动模式，使用注意图卷积模块（AGCM）构建全局人-物关系图。此外，我们还结合了一个多尺度时间卷积模块（MTC）和一个跨维度关注模块（CDAM）来动态关注关键的时空节点和特征通道。我们的方法在保持高计算效率的同时，在精度上有了显著的提高，适合于隐私敏感场景下的实时应用。在排球和NBA数据集上的实验表明，与主流方法相比，我们的方法仅使用骨架输入就可以获得具有竞争力的性能，显著减少了参数和计算成本。在这里，我们的方法显示，在排球数据集上，多类每类准确率（MPCA）提高到96.1%，在NBA数据集上提高到71.6%，为隐私敏感场景中的GAR提供了轻量级和高效的解决方案。

{"title":"Enhanced skeleton-based Group Activity Recognition through spatio-temporal graph convolution with cross-dimensional attention","authors":"Dongli Wang , Yongcan Weng , Xiaolin Zhu , Yan Zhou , Zixin Zhang , Richard Irampaye","doi":"10.1016/j.imavis.2025.105784","DOIUrl":"10.1016/j.imavis.2025.105784","url":null,"abstract":"<div><div>Group Activity Recognition is a pivotal task in video understanding, with broad applications ranging from surveillance to human–computer interaction. Traditional RGB-based methods face challenges such as privacy concerns, environmental sensitivity, and fragmented scene-level semantic understanding. Skeleton-based approaches offer a promising alternative but often suffer from limited exploration of heterogeneous features and the absence of explicit modeling for human-object interactions. In this paper, we introduce a lightweight framework for skeleton-based GAR, leveraging an attention-enhanced spatio-temporal graph convolutional network. Specially, we first decouple joint and bone features along with their motion patterns, constructing a global human-object relational graph using an attention graph convolution module (AGCM). Additionally, we incorporate a Multi-Scale Temporal Convolution Module (MTC) and a Cross-Dimensional Attention Module (CDAM) to dynamically focus on key spatio-temporal nodes and feature channels. Our method achieves significant improvements in accuracy while maintaining high computational efficiency, making it suitable for real-time applications in privacy-sensitive scenarios. Experiments on the Volleyball and NBA datasets demonstrate that our method achieves competitive performance using only skeleton input, significantly reducing parameters and computational cost compared to mainstream approaches. Here, our method show an improvement in Multi-Class Per-Class Accuracy (MPCA) to 96.1% on the Volleyball dataset and 71.6% on the NBA dataset, offering a lightweight and efficient solution for GAR in privacy-sensitive scenarios.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"165 ","pages":"Article 105784"},"PeriodicalIF":4.2,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145521219","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0