Pub Date : 2026-01-16DOI: 10.1016/j.cviu.2026.104658
Toshinori Yamauchi
Visual explanation for object detectors is crucial for ensuring model reliability and promoting their application across a variety of domains. However, existing visual explanation methods often produce low-quality heat maps that highlight only parts of important regions for detected instances. This limitation reduces the interpretability of object detectors and hinders effective model analysis. To address this issue, we propose Spatial Sensitive Grad-CAM++, a visual explanation method for object detectors designed to enhance the quality of heat maps. The proposed method introduces a weighted combination of gradient maps into the heat map computation, taking into account the architecture of object detectors. This approach enables a more accurate representation of each CNN channel’s contribution to the final heat map, resulting in high-quality heat maps that better identify important regions. In quantitative evaluations using the deletion and insertion metrics, we confirm that the proposed method outperforms existing methods by approximately 30% and 8%, respectively. In qualitative evaluations, we further demonstrate the superiority of the proposed method. These results suggest that the proposed method provides more faithful explanations, allowing for more accurate and reliable model analysis.
{"title":"Spatial Sensitive Grad-CAM++: Towards High-Quality Visual Explanations for Object Detectors via Weighted Combination of Gradient Maps","authors":"Toshinori Yamauchi","doi":"10.1016/j.cviu.2026.104658","DOIUrl":"10.1016/j.cviu.2026.104658","url":null,"abstract":"<div><div>Visual explanation for object detectors is crucial for ensuring model reliability and promoting their application across a variety of domains. However, existing visual explanation methods often produce low-quality heat maps that highlight only parts of important regions for detected instances. This limitation reduces the interpretability of object detectors and hinders effective model analysis. To address this issue, we propose Spatial Sensitive Grad-CAM++, a visual explanation method for object detectors designed to enhance the quality of heat maps. The proposed method introduces a weighted combination of gradient maps into the heat map computation, taking into account the architecture of object detectors. This approach enables a more accurate representation of each CNN channel’s contribution to the final heat map, resulting in high-quality heat maps that better identify important regions. In quantitative evaluations using the deletion and insertion metrics, we confirm that the proposed method outperforms existing methods by approximately 30% and 8%, respectively. In qualitative evaluations, we further demonstrate the superiority of the proposed method. These results suggest that the proposed method provides more faithful explanations, allowing for more accurate and reliable model analysis.</div></div>","PeriodicalId":50633,"journal":{"name":"Computer Vision and Image Understanding","volume":"264 ","pages":"Article 104658"},"PeriodicalIF":3.5,"publicationDate":"2026-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146038533","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-01-14DOI: 10.1016/j.cviu.2026.104655
Fei Wan, Ruicong Zhi
Investigating the associations between facial action units (AUs) and emotions (EMOs) helps to eliminate the constraints imposed by predefined emotion patterns. However, accurately modeling the soft, probabilistic AU–EMO relationships and inferring emotional states from AU sequences remains a challenging task. To address this, we propose a Bi-intelligence Probabilistic Graph model (BiPG-FER), which flexibly learns interpretable AU–AU and AU–EMO associations and enables automatic facial expression inference from AU sequences. In the input phase, a small portion of external prior knowledge is incorporated to mitigate the high-entropy fluctuations often caused by random initialization. We construct a two-layer fully connected AU–EMO association graph and develop an end-to-end architecture with a masking mechanism that dynamically updates the AU–AU and AU–EMO relationships by computing joint probabilities. An oversampling strategy, combined with a adaptive thresholding and a data-contribution-aware reweighting scheme, is introduced to address the skewed post-distribution of emotion labels. Finally, we design a strategy that preserves previous model weights and generates pseudo-samples based on the top-k conditional AU–EMO probabilities, allowing the model to evolve smoothly in a continuously changing and heterogeneous data stream. Experimental results demonstrate that the proposed BiPG-FER effectively produces interpretable probabilistic associations while improving recognition performance on both micro-expression and macro-expression datasets.
{"title":"BiPG-FER: Bi-intelligence probabilistic graph for facial expression inference drived by action units","authors":"Fei Wan, Ruicong Zhi","doi":"10.1016/j.cviu.2026.104655","DOIUrl":"10.1016/j.cviu.2026.104655","url":null,"abstract":"<div><div>Investigating the associations between facial action units (AUs) and emotions (EMOs) helps to eliminate the constraints imposed by predefined emotion patterns. However, accurately modeling the soft, probabilistic AU–EMO relationships and inferring emotional states from AU sequences remains a challenging task. To address this, we propose a Bi-intelligence Probabilistic Graph model (BiPG-FER), which flexibly learns interpretable AU–AU and AU–EMO associations and enables automatic facial expression inference from AU sequences. In the input phase, a small portion of external prior knowledge is incorporated to mitigate the high-entropy fluctuations often caused by random initialization. We construct a two-layer fully connected AU–EMO association graph and develop an end-to-end architecture with a masking mechanism that dynamically updates the AU–AU and AU–EMO relationships by computing joint probabilities. An oversampling strategy, combined with a adaptive thresholding and a data-contribution-aware reweighting scheme, is introduced to address the skewed post-distribution of emotion labels. Finally, we design a strategy that preserves previous model weights and generates pseudo-samples based on the top-k conditional AU–EMO probabilities, allowing the model to evolve smoothly in a continuously changing and heterogeneous data stream. Experimental results demonstrate that the proposed BiPG-FER effectively produces interpretable probabilistic associations while improving recognition performance on both micro-expression and macro-expression datasets.</div></div>","PeriodicalId":50633,"journal":{"name":"Computer Vision and Image Understanding","volume":"264 ","pages":"Article 104655"},"PeriodicalIF":3.5,"publicationDate":"2026-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145978916","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-01-12DOI: 10.1016/j.cviu.2026.104648
Hang Su , Hong-Bo Zhang , Jia-Yin Luo , Jing-Hua Liu , Zhen-Zhen Sun , Ji-Xiang Du
Transformer-based methods have shown strong potential in human-object interaction (HOI) detection, yet challenges remain due to task interference and unstable query initialization. To overcome these limitations, the paper adopts a cascaded-parallel decoders architecture that balances the efficiency of one-stage models with the task decoupling advantages of two-stage designs. A key component is an anchor-guided query generator, which explicitly incorporates human-object spatial information into query initialization. This provides queries with strong spatial awareness, stabilizes training, and significantly improves human and object localization-a crucial prerequisite for accurate HOI detection. In addition, interaction features are further refined by modeling multi-relational cues within each triplet, facilitating more reliable verb classification. Extensive experiments on the HICO-DET and V-COCO datasets demonstrate that the proposed method achieves superior performance compared to state-of-the-art approaches.
{"title":"Cascaded-parallel decoders and anchor-guided query generator for human-object interaction","authors":"Hang Su , Hong-Bo Zhang , Jia-Yin Luo , Jing-Hua Liu , Zhen-Zhen Sun , Ji-Xiang Du","doi":"10.1016/j.cviu.2026.104648","DOIUrl":"10.1016/j.cviu.2026.104648","url":null,"abstract":"<div><div>Transformer-based methods have shown strong potential in human-object interaction (HOI) detection, yet challenges remain due to task interference and unstable query initialization. To overcome these limitations, the paper adopts a cascaded-parallel decoders architecture that balances the efficiency of one-stage models with the task decoupling advantages of two-stage designs. A key component is an anchor-guided query generator, which explicitly incorporates human-object spatial information into query initialization. This provides queries with strong spatial awareness, stabilizes training, and significantly improves human and object localization-a crucial prerequisite for accurate HOI detection. In addition, interaction features are further refined by modeling multi-relational cues within each triplet, facilitating more reliable verb classification. Extensive experiments on the HICO-DET and V-COCO datasets demonstrate that the proposed method achieves superior performance compared to state-of-the-art approaches.</div></div>","PeriodicalId":50633,"journal":{"name":"Computer Vision and Image Understanding","volume":"264 ","pages":"Article 104648"},"PeriodicalIF":3.5,"publicationDate":"2026-01-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145978918","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-01-12DOI: 10.1016/j.cviu.2026.104657
Hao Xu , Arbind Agrahari Baniya , Sam Wells , Mohamed Reda Bouadjenek , Richard Dazeley , Sunil Aryal
Ball tracking is a fundamental problem in computer vision, particularly in sports analytics, where it underpins tasks such as analyzing ball movement in soccer and basketball or detecting bounce locations in tennis and table tennis. Most existing methods are developed and evaluated on resource-rich, commercial sports footage with ideal camera angles, high-resolution imagery, and multiple viewpoints. In contrast, many other sports contexts, including semi-professional leagues, local amateur competitions, and Paralympic sports, lack these resources. Footage in these settings often comes from single, fixed, and suboptimal viewpoints, where occlusion becomes a dominant challenge for automated tracking. Existing methods frequently fall short in such conditions because their architectures and training strategies do not explicitly account for prolonged or full occlusion. To address this gap, we present the Table Tennis Australia (TTA) dataset, the first professionally annotated Paralympic table tennis benchmark with dense visibility labels, captured under realistic single-view conditions. With 2,396 occluded instances (including 998 fully occluded), TTA is the most occlusion-rich publicly available dataset to date. Alongside the dataset, we propose the Temporal Occlusion Tracking Network (TOTNet), a novel tracking system designed to maintain localization accuracy even under extended occlusion. Through comprehensive experiments on four sports tracking datasets, TOTNet achieves state-of-the-art performance, with substantial gains in full-occlusion scenarios. We release the dataset, code, and evaluation scripts to foster reproducibility and future research in occlusion robust tracking for low resource sports; all materials are available at https://github.com/AugustRushG/TOTNet.
{"title":"TOTNet: Occlusion-aware temporal tracking for robust ball detection in sports videos","authors":"Hao Xu , Arbind Agrahari Baniya , Sam Wells , Mohamed Reda Bouadjenek , Richard Dazeley , Sunil Aryal","doi":"10.1016/j.cviu.2026.104657","DOIUrl":"10.1016/j.cviu.2026.104657","url":null,"abstract":"<div><div>Ball tracking is a fundamental problem in computer vision, particularly in sports analytics, where it underpins tasks such as analyzing ball movement in soccer and basketball or detecting bounce locations in tennis and table tennis. Most existing methods are developed and evaluated on resource-rich, commercial sports footage with ideal camera angles, high-resolution imagery, and multiple viewpoints. In contrast, many other sports contexts, including semi-professional leagues, local amateur competitions, and Paralympic sports, lack these resources. Footage in these settings often comes from single, fixed, and suboptimal viewpoints, where occlusion becomes a dominant challenge for automated tracking. Existing methods frequently fall short in such conditions because their architectures and training strategies do not explicitly account for prolonged or full occlusion. To address this gap, we present the <strong>Table Tennis Australia (TTA) dataset</strong>, the first professionally annotated Paralympic table tennis benchmark with dense visibility labels, captured under realistic single-view conditions. With <strong>2,396</strong> occluded instances (including 998 fully occluded), TTA is the most occlusion-rich publicly available dataset to date. Alongside the dataset, we propose the <strong>Temporal Occlusion Tracking Network (TOTNet)</strong>, a novel tracking system designed to maintain localization accuracy even under extended occlusion. Through comprehensive experiments on four sports tracking datasets, TOTNet achieves state-of-the-art performance, with substantial gains in full-occlusion scenarios. We release the dataset, code, and evaluation scripts to foster reproducibility and future research in occlusion robust tracking for low resource sports; all materials are available at <span><span>https://github.com/AugustRushG/TOTNet</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50633,"journal":{"name":"Computer Vision and Image Understanding","volume":"264 ","pages":"Article 104657"},"PeriodicalIF":3.5,"publicationDate":"2026-01-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145978915","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-01-12DOI: 10.1016/j.cviu.2026.104656
Junhao Sun, Lanfei Zhao
Action Quality Assessment (AQA) aims to quantitatively evaluate the execution quality of complex human actions, which poses significant challenges due to the need for jointly modeling spatio-temporal dynamics and semantic structures. Existing approaches typically rely on static single-branch architectures, limiting their capacity to balance local fine-grained details and global rhythmic dependencies, especially in high-complexity scenarios. To address these limitations, we propose a novel Spatio-Temporal Adaptive Recalibration (STAR) Block, which enables highly discriminative representation learning via a multi-dimensional modeling strategy. Specifically, we first design a Multi-Scale Context Encoder to capture subtle local cues by leveraging parallel convolutions across spatial, temporal, and joint domains, enhancing the perception of motion details and short-term dynamics. Second, we introduce an Axial Attention-Based Global Dependency Modeling Module, which efficiently captures long-range temporal relationships while preserving the original spatio-temporal structure, thus reinforcing the understanding of phase coherence and motion rhythm. Third, a Dynamic Attention-Guided Adaptive Feature Fusion mechanism is proposed to integrate multi-path temporal semantics by assigning adaptive weights to local and global representations, enabling dynamic equilibrium in temporal modeling. Across multiple metrics, our STAR Block delivers remarkably superior performance with significant margins over state-of-the-art methods, achieving an average Spearman’s improvement of 1.56% on AQA-7, 0.57% on MTL-AQA with DD supervision, and near-perfect 99.52% accuracy on FR-FS, as proven by extensive evaluations.
{"title":"STAR Block: Adaptive spatio-temporal recalibration for action quality assessment","authors":"Junhao Sun, Lanfei Zhao","doi":"10.1016/j.cviu.2026.104656","DOIUrl":"10.1016/j.cviu.2026.104656","url":null,"abstract":"<div><div>Action Quality Assessment (AQA) aims to quantitatively evaluate the execution quality of complex human actions, which poses significant challenges due to the need for jointly modeling spatio-temporal dynamics and semantic structures. Existing approaches typically rely on static single-branch architectures, limiting their capacity to balance local fine-grained details and global rhythmic dependencies, especially in high-complexity scenarios. To address these limitations, we propose a novel Spatio-Temporal Adaptive Recalibration (STAR) Block, which enables highly discriminative representation learning via a multi-dimensional modeling strategy. Specifically, we first design a Multi-Scale Context Encoder to capture subtle local cues by leveraging parallel convolutions across spatial, temporal, and joint domains, enhancing the perception of motion details and short-term dynamics. Second, we introduce an Axial Attention-Based Global Dependency Modeling Module, which efficiently captures long-range temporal relationships while preserving the original spatio-temporal structure, thus reinforcing the understanding of phase coherence and motion rhythm. Third, a Dynamic Attention-Guided Adaptive Feature Fusion mechanism is proposed to integrate multi-path temporal semantics by assigning adaptive weights to local and global representations, enabling dynamic equilibrium in temporal modeling. Across multiple metrics, our STAR Block delivers remarkably superior performance with significant margins over state-of-the-art methods, achieving an average Spearman’s <span><math><mi>ρ</mi></math></span> improvement of 1.56% on AQA-7, 0.57% on MTL-AQA with DD supervision, and near-perfect 99.52% accuracy on FR-FS, as proven by extensive evaluations.</div></div>","PeriodicalId":50633,"journal":{"name":"Computer Vision and Image Understanding","volume":"264 ","pages":"Article 104656"},"PeriodicalIF":3.5,"publicationDate":"2026-01-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145978913","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-01-10DOI: 10.1016/j.cviu.2026.104650
David Tejero-Ruiz , David Solís-Martín , Francisco J. Pérez-Grau , Joaquín Borrego-Díaz
Indoor UAV navigation faces significant challenges due to GPS signal absence and limitations of conventional visual-inertial systems under challenging lighting and motion conditions. This paper presents an event-based visual-inertial odometry system that addresses these limitations through intermediate frame reconstruction from event streams combined with established odometry algorithms. The approach leverages event cameras’ unique characteristics — microsecond temporal resolution, high dynamic range (120 dB), and motion blur immunity — to maintain stable navigation performance under conditions that cause conventional systems to fail. The system achieves real-time operation at 30 Hz frame reconstruction and 20 Hz pose estimation on embedded hardware, consuming 15 W power while adding only 50 g to the UAV platform. Experimental validation in controlled indoor environments demonstrates mean absolute pose errors of 26–42 cm across different operational conditions, comparable to conventional visual-inertial systems. Critically, the system maintains stable performance during rapid lighting transitions, showing only 59% performance degradation compared to baseline conditions, while conventional cameras typically experience complete tracking failure. The results establish event-based visual-inertial odometry as a viable alternative for indoor UAV navigation, particularly in applications requiring environmental robustness over marginal accuracy improvements under optimal conditions.
{"title":"Indoor UAV navigation using event cameras and intermediate frame reconstruction","authors":"David Tejero-Ruiz , David Solís-Martín , Francisco J. Pérez-Grau , Joaquín Borrego-Díaz","doi":"10.1016/j.cviu.2026.104650","DOIUrl":"10.1016/j.cviu.2026.104650","url":null,"abstract":"<div><div>Indoor UAV navigation faces significant challenges due to GPS signal absence and limitations of conventional visual-inertial systems under challenging lighting and motion conditions. This paper presents an event-based visual-inertial odometry system that addresses these limitations through intermediate frame reconstruction from event streams combined with established odometry algorithms. The approach leverages event cameras’ unique characteristics — microsecond temporal resolution, high dynamic range (120 dB), and motion blur immunity — to maintain stable navigation performance under conditions that cause conventional systems to fail. The system achieves real-time operation at 30 Hz frame reconstruction and 20 Hz pose estimation on embedded hardware, consuming 15 W power while adding only 50 g to the UAV platform. Experimental validation in controlled indoor environments demonstrates mean absolute pose errors of 26–42 cm across different operational conditions, comparable to conventional visual-inertial systems. Critically, the system maintains stable performance during rapid lighting transitions, showing only 59% performance degradation compared to baseline conditions, while conventional cameras typically experience complete tracking failure. The results establish event-based visual-inertial odometry as a viable alternative for indoor UAV navigation, particularly in applications requiring environmental robustness over marginal accuracy improvements under optimal conditions.</div></div>","PeriodicalId":50633,"journal":{"name":"Computer Vision and Image Understanding","volume":"264 ","pages":"Article 104650"},"PeriodicalIF":3.5,"publicationDate":"2026-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145978912","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-01-07DOI: 10.1016/j.cviu.2026.104637
Yiming Yang, Feng Guo, Pei Niu
Real-time object detection is pivotal in traffic-related Unmanned Aerial Vehicles (UAV) applications. However, UAV imagery presents significant challenges due to the predominance of small objects and complex backgrounds. Traditional backbones generally perform aggressive early-stage downsampling, causing the loss of fine-grained features. To address these issues, we propose UAVDet, a real-time detection model that combines Convolutional Neural Network (CNN) and Mamba architectures. First, we revisit the conventional backbone design by reconfiguring its depth and width, with a focus on preserving fine-grained details crucial for small object detection. Second, we propose the Cross Stage Partial Mamba (CSPMB) module, which integrates the Mamba structure into the CNN framework to enhance global feature representation and improve robustness against complex background interference. Third, we design Tiny-focused Feature Pyramid Network (TFPN) by rebalancing the feature fusion flow and replacing the large-object detection head with a tiny-object detection head, which significantly improves the perception of small objects. Comprehensive experiments on the VisDrone dataset show that our method improves AP and AP by 4.5% and 5.0%, respectively, while reducing parameters by 84.9% compared to the baseline. It also reaches 53 FPS on an RTX 4090, exceeding the 30 FPS real-time threshold. Additional evaluations on UAVDT and DroneVehicle further verify the method’s robust generalization. These results indicate the effectiveness of the developed method in UAV image detection.
{"title":"UAVDet: A CNN–Mamba hybrid network for efficient small object detection in UAV imagery","authors":"Yiming Yang, Feng Guo, Pei Niu","doi":"10.1016/j.cviu.2026.104637","DOIUrl":"10.1016/j.cviu.2026.104637","url":null,"abstract":"<div><div>Real-time object detection is pivotal in traffic-related Unmanned Aerial Vehicles (UAV) applications. However, UAV imagery presents significant challenges due to the predominance of small objects and complex backgrounds. Traditional backbones generally perform aggressive early-stage downsampling, causing the loss of fine-grained features. To address these issues, we propose UAVDet, a real-time detection model that combines Convolutional Neural Network (CNN) and Mamba architectures. First, we revisit the conventional backbone design by reconfiguring its depth and width, with a focus on preserving fine-grained details crucial for small object detection. Second, we propose the Cross Stage Partial Mamba (CSPMB) module, which integrates the Mamba structure into the CNN framework to enhance global feature representation and improve robustness against complex background interference. Third, we design Tiny-focused Feature Pyramid Network (TFPN) by rebalancing the feature fusion flow and replacing the large-object detection head with a tiny-object detection head, which significantly improves the perception of small objects. Comprehensive experiments on the VisDrone dataset show that our method improves AP and AP<span><math><msub><mrow></mrow><mrow><mi>S</mi></mrow></msub></math></span> by 4.5% and 5.0%, respectively, while reducing parameters by 84.9% compared to the baseline. It also reaches 53 FPS on an RTX 4090, exceeding the 30 FPS real-time threshold. Additional evaluations on UAVDT and DroneVehicle further verify the method’s robust generalization. These results indicate the effectiveness of the developed method in UAV image detection.</div></div>","PeriodicalId":50633,"journal":{"name":"Computer Vision and Image Understanding","volume":"264 ","pages":"Article 104637"},"PeriodicalIF":3.5,"publicationDate":"2026-01-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145928068","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-01-07DOI: 10.1016/j.cviu.2026.104636
Weiwei Duan, Luping Ji, Shengjia Chen, Jianghong Huang
Due to the low signal-to-noise ratio and weak visual contrast, infrared small targets are often submerged in the background. Therefore, it is crucial to preserve target information while extracting distinctive features that distinguish them from the background. However, existing methods generally rely on convolutions and transformers in isolation, which limits their ability to capture robust target features in complex scenes. To address this issue, we propose a new local–global feature collaborative learning (LGFC) framework. It could adequately integrate the local spatial features with the global context of targets in a unified manner. Specifically, we develop an enhanced Gaussian-mask Vision Transformer group with Global Gaussian Attention and Local Window Attention to extract refined global features. The local coarse features obtained from the convolution encoder are then coordinated with the refined global features through Local–Global Collaborating. Moreover, to avoid feature loss during decoding, we propose a level-wise decoding strategy with Cross-layer Feature Interaction to to mitigate information loss in deep networks. Additionally, we introduce a Coarse-to-Fine Refinement post-processing mechanism to improve the precision of target contours. The extensive experiments on three public datasets (NUAA-SIRST, IRSTD-1K and SIRST-AUG) demonstrate the superiority and generalization ability of our proposed LGFC framework for infrared small target detection, outperforming state-of-the-art methods by approximately 2.3% in F1-score on each dataset.
{"title":"Local–global collaborative feature learning with level-wise decoding for infrared small target detection","authors":"Weiwei Duan, Luping Ji, Shengjia Chen, Jianghong Huang","doi":"10.1016/j.cviu.2026.104636","DOIUrl":"10.1016/j.cviu.2026.104636","url":null,"abstract":"<div><div>Due to the low signal-to-noise ratio and weak visual contrast, infrared small targets are often submerged in the background. Therefore, it is crucial to preserve target information while extracting distinctive features that distinguish them from the background. However, existing methods generally rely on convolutions and transformers in isolation, which limits their ability to capture robust target features in complex scenes. To address this issue, we propose a new local–global feature collaborative learning (LGFC) framework. It could adequately integrate the local spatial features with the global context of targets in a unified manner. Specifically, we develop an enhanced <em>Gaussian-mask Vision Transformer</em> group with <em>Global Gaussian Attention</em> and <em>Local Window Attention</em> to extract refined global features. The local coarse features obtained from the convolution encoder are then coordinated with the refined global features through <em>Local–Global Collaborating</em>. Moreover, to avoid feature loss during decoding, we propose a level-wise decoding strategy with <em>Cross-layer Feature Interaction</em> to to mitigate information loss in deep networks. Additionally, we introduce a <em>Coarse-to-Fine Refinement</em> post-processing mechanism to improve the precision of target contours. The extensive experiments on three public datasets (NUAA-SIRST, IRSTD-1K and SIRST-AUG) demonstrate the superiority and generalization ability of our proposed LGFC framework for infrared small target detection, outperforming state-of-the-art methods by approximately 2.3% in F1-score on each dataset.</div></div>","PeriodicalId":50633,"journal":{"name":"Computer Vision and Image Understanding","volume":"264 ","pages":"Article 104636"},"PeriodicalIF":3.5,"publicationDate":"2026-01-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145928062","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-01-07DOI: 10.1016/j.cviu.2026.104638
Zhuo-Ming Du , Hong-An Li , Qian Yu , Wen-He Chen , Fei-long Han
Accurate estimation and correction of global illuminant color, known as color constancy, is crucial for computational photography and computer vision but remains challenging under complex lighting conditions. We propose CSNet, an end-to-end framework that improves color constancy through a novel content-guided feature fusion approach. The input image is first decomposed into three precomputed components: mean intensity, variation magnitude, and variation direction. These components are dynamically reweighted by the Content-Weighting Network (CWN), which generates spatially varying weight maps by leveraging both local and global image features. The reweighted components are fused via the Adaptive Fusion Module (AFM) to produce an HDR-like intermediate representation. This representation is then processed by the Illumination Prediction Network (IPN), which applies semantic-aware weighting to estimate the global illuminant color as an RGB triplet. Extensive experiments on standard benchmarks demonstrate that CSNet achieves state-of-the-art performance, offering robust and visually consistent results under diverse lighting conditions. These advantages make CSNet a powerful tool for applications such as automatic photo correction and augmented reality.
{"title":"CSNet: A content and structure-aware approach for color constancy","authors":"Zhuo-Ming Du , Hong-An Li , Qian Yu , Wen-He Chen , Fei-long Han","doi":"10.1016/j.cviu.2026.104638","DOIUrl":"10.1016/j.cviu.2026.104638","url":null,"abstract":"<div><div>Accurate estimation and correction of global illuminant color, known as color constancy, is crucial for computational photography and computer vision but remains challenging under complex lighting conditions. We propose CSNet, an end-to-end framework that improves color constancy through a novel content-guided feature fusion approach. The input image is first decomposed into three precomputed components: mean intensity, variation magnitude, and variation direction. These components are dynamically reweighted by the Content-Weighting Network (CWN), which generates spatially varying weight maps by leveraging both local and global image features. The reweighted components are fused via the Adaptive Fusion Module (AFM) to produce an HDR-like intermediate representation. This representation is then processed by the Illumination Prediction Network (IPN), which applies semantic-aware weighting to estimate the global illuminant color as an RGB triplet. Extensive experiments on standard benchmarks demonstrate that CSNet achieves state-of-the-art performance, offering robust and visually consistent results under diverse lighting conditions. These advantages make CSNet a powerful tool for applications such as automatic photo correction and augmented reality.</div></div>","PeriodicalId":50633,"journal":{"name":"Computer Vision and Image Understanding","volume":"264 ","pages":"Article 104638"},"PeriodicalIF":3.5,"publicationDate":"2026-01-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145928063","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-01-07DOI: 10.1016/j.cviu.2026.104635
Bowen Xu , Yaru Sui , Longxin Liu, Zhenlong Ma, Yunlong Shi, Wentong Li, Xiaoqiang Ji
Face liveness detection algorithms are widely used in anti-spoofing applications, which guarantee the accuracy and security of face recognition systems. However, with the continuous development of technologies such as 3D printing and artificial intelligence, traditional face-liveness detection algorithms struggle to withstand spoofing attacks effectively. In this paper, we propose a multi-feature fusion algorithm using only facial video for face liveness detection. Initially, we design a dual-channel network named DC-Net. It can extract robust remote photoplethysmography signals directly from 5-second facial videos, as well as fine global texture features from the keyframes of the image sequence. Subsequently, a fusion module based on the attention mechanism is used to carry out feature-level fusion. Ultimately, we use the fully connected layer for binary classification. Our methodology was validated using the REPLAY-ATTACK dataset and the 3DMAD dataset, demonstrating that for printing attacks, screen replay attacks, and 3D mask attacks, our approach attained an accuracy of 99.79% and 100% on both datasets, respectively. Meanwhile, cross-dataset testing was conducted on the CASIA-FASD and HKBU-MARs V1+ datasets, achieving HTER of 25.56% and 0.00%, respectively. This indicates that the algorithm has good accuracy and robustness in dealing with spoofing attacks in many different scenarios, which provides important ideas and technical support for the design and implementation of reliable face recognition systems.
{"title":"A dual-channel model based on multi-feature fusion for face liveness detection","authors":"Bowen Xu , Yaru Sui , Longxin Liu, Zhenlong Ma, Yunlong Shi, Wentong Li, Xiaoqiang Ji","doi":"10.1016/j.cviu.2026.104635","DOIUrl":"10.1016/j.cviu.2026.104635","url":null,"abstract":"<div><div>Face liveness detection algorithms are widely used in anti-spoofing applications, which guarantee the accuracy and security of face recognition systems. However, with the continuous development of technologies such as 3D printing and artificial intelligence, traditional face-liveness detection algorithms struggle to withstand spoofing attacks effectively. In this paper, we propose a multi-feature fusion algorithm using only facial video for face liveness detection. Initially, we design a dual-channel network named DC-Net. It can extract robust remote photoplethysmography signals directly from 5-second facial videos, as well as fine global texture features from the keyframes of the image sequence. Subsequently, a fusion module based on the attention mechanism is used to carry out feature-level fusion. Ultimately, we use the fully connected layer for binary classification. Our methodology was validated using the REPLAY-ATTACK dataset and the 3DMAD dataset, demonstrating that for printing attacks, screen replay attacks, and 3D mask attacks, our approach attained an accuracy of 99.79% and 100% on both datasets, respectively. Meanwhile, cross-dataset testing was conducted on the CASIA-FASD and HKBU-MARs V1+ datasets, achieving HTER of 25.56% and 0.00%, respectively. This indicates that the algorithm has good accuracy and robustness in dealing with spoofing attacks in many different scenarios, which provides important ideas and technical support for the design and implementation of reliable face recognition systems.</div></div>","PeriodicalId":50633,"journal":{"name":"Computer Vision and Image Understanding","volume":"264 ","pages":"Article 104635"},"PeriodicalIF":3.5,"publicationDate":"2026-01-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145928064","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号