Image and Vision Computing最新文献_第4页

Statistic temporal checking and spatial consistency based 3D size reconstruction of multiple objects from indoor monocular videos 基于统计时间检验和空间一致性的室内单目视频多目标三维尺寸重建

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2026-03-01 Epub Date: 2025-12-27 DOI: 10.1016/j.imavis.2025.105890

Ziyue Wang , Xina Cheng , Takeshi Ikenaga

Reconstructing accurate 3D sizes of multiple objects from indoor monocular videos has gradually become a significant topic for robotics, smart homes, and wireless signal analysis. However, existing monocular reconstruction pipelines often focus on the surface or 3D bounding box reconstruction of objects, making unreliable size estimation due to occlusion, missing depth, and incomplete visibility. To accurately reconstruct the real size of objects in different shapes under complex indoor conditions, this work proposes statistic checking module with depth layering and spatial consistency checking for accurate object size reconstruction. First, by checking the frequency of feature points from the semantic information, statistic temporal checking is used to remove outliers around object region by checking the probability of foreground and background region. Secondly, depth layering provides depth prior, which helps to enhance the boundary of objects and increases the 3D reconstruction accuracy. Then, a semantic-guided spatial consistency checking module infers the hidden or occluded parts of objects by exploiting category-specific priors and spatial consistency. The inferred complete object boundaries are enclosed using surface fitting and volumetric filling, resulting in final volumetric occupancy estimates for each individual object. Extensive experiments demonstrate that the proposed method achieves 0.3137 error rate, which is approximately 0.5641 lower than the average.

从室内单目视频中重建多个物体的精确3D尺寸已逐渐成为机器人、智能家居和无线信号分析领域的重要课题。然而，现有的单目重建管道往往侧重于物体的表面或三维边界盒重建，由于遮挡、缺失深度和不完全可见性，导致尺寸估计不可靠。为了在复杂室内条件下准确重建不同形状物体的真实尺寸，本文提出了具有深度分层和空间一致性检查的统计检查模块，用于精确重建物体尺寸。首先，从语义信息中检测特征点的频率，通过检测前景和背景区域的概率，采用统计时态检测去除目标区域周围的离群点；其次，深度分层提供了深度先验，有助于增强物体的边界，提高三维重建精度。然后，一个语义引导的空间一致性检查模块通过利用特定类别的先验和空间一致性推断出对象的隐藏或遮挡部分。推断出的完整物体边界使用表面拟合和体积填充来封闭，从而得到每个单独物体的最终体积占用估计。大量实验表明，该方法的错误率为0.3137，比平均值低约0.5641。

{"title":"Statistic temporal checking and spatial consistency based 3D size reconstruction of multiple objects from indoor monocular videos","authors":"Ziyue Wang , Xina Cheng , Takeshi Ikenaga","doi":"10.1016/j.imavis.2025.105890","DOIUrl":"10.1016/j.imavis.2025.105890","url":null,"abstract":"<div><div>Reconstructing accurate 3D sizes of multiple objects from indoor monocular videos has gradually become a significant topic for robotics, smart homes, and wireless signal analysis. However, existing monocular reconstruction pipelines often focus on the surface or 3D bounding box reconstruction of objects, making unreliable size estimation due to occlusion, missing depth, and incomplete visibility. To accurately reconstruct the real size of objects in different shapes under complex indoor conditions, this work proposes statistic checking module with depth layering and spatial consistency checking for accurate object size reconstruction. First, by checking the frequency of feature points from the semantic information, statistic temporal checking is used to remove outliers around object region by checking the probability of foreground and background region. Secondly, depth layering provides depth prior, which helps to enhance the boundary of objects and increases the 3D reconstruction accuracy. Then, a semantic-guided spatial consistency checking module infers the hidden or occluded parts of objects by exploiting category-specific priors and spatial consistency. The inferred complete object boundaries are enclosed using surface fitting and volumetric filling, resulting in final volumetric occupancy estimates for each individual object. Extensive experiments demonstrate that the proposed method achieves 0.3137 error rate, which is approximately 0.5641 lower than the average.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"167 ","pages":"Article 105890"},"PeriodicalIF":4.2,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145885582","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

MoMIL: Multi-order enhanced multiple instance learning for computational pathology MoMIL：计算病理学的多阶增强多实例学习

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2026-03-01 Epub Date: 2026-01-28 DOI: 10.1016/j.imavis.2026.105918

Yuqi Zhang , Xiaoqian Zhang , Jiakai Wang , Baoyu Liang , Yuancheng Yang , Chao Tong

Computational pathology (CPath) has significantly advanced the clinical practice of pathology. Despite the progress made, Multiple Instance Learning (MIL), a promising paradigm within CPath, continues to face challenges, especially those related to structural fixation and incomplete information utilization. To address these limitations, we propose a novel MIL framework named Multi-order MIL (MoMIL). Our framework utilizes the SSD model to perform long-sequence modeling on multi-order WSI patches and combines lightweight feature fusion to achieve more comprehensive feature information utilization. This framework supports the fusion of a broader range of features and is highly flexible, allowing for expansion based on specific usage requirements. Additionally, we introduce a sequence transformation method specifically designed for WSIs. This method is not only adaptable to different WSI sizes but also captures additional feature expression, resulting in a more effective exploitation of sequential cues. Extensive experiments demonstrate that MoMIL surpasses state-of-the-art MIL methods, up to 0.027 AUC improvements for cancer sub-typing. We conducted extensive experiments on three downstream tasks with a total of five datasets, achieving improvements in all performance metrics. The code is available at https://github.com/YuqiZhang-Buaa/MoMIL.

计算病理学（CPath）显著地促进了病理学的临床实践。尽管取得了一些进展，但多实例学习（MIL）作为CPath中一个有前途的范式，仍然面临着挑战，特别是与结构固定和不完全信息利用有关的挑战。为了解决这些限制，我们提出了一种新的MIL框架，称为多阶MIL （MoMIL）。我们的框架利用SSD模型对多阶WSI补丁进行长序列建模，并结合轻量级特征融合实现更全面的特征信息利用。该框架支持更广泛的功能融合，并且高度灵活，允许根据特定的使用需求进行扩展。此外，我们还介绍了一种专门为wsi设计的序列转换方法。该方法不仅适用于不同的WSI大小，而且可以捕获额外的特征表达式，从而更有效地利用序列线索。大量的实验表明，MoMIL超过了最先进的MIL方法，在癌症分型方面提高了0.027 AUC。我们对总共五个数据集的三个下游任务进行了广泛的实验，在所有性能指标上都取得了改进。代码可在https://github.com/YuqiZhang-Buaa/MoMIL上获得。

{"title":"MoMIL: Multi-order enhanced multiple instance learning for computational pathology","authors":"Yuqi Zhang , Xiaoqian Zhang , Jiakai Wang , Baoyu Liang , Yuancheng Yang , Chao Tong","doi":"10.1016/j.imavis.2026.105918","DOIUrl":"10.1016/j.imavis.2026.105918","url":null,"abstract":"<div><div>Computational pathology (CPath) has significantly advanced the clinical practice of pathology. Despite the progress made, Multiple Instance Learning (MIL), a promising paradigm within CPath, continues to face challenges, especially those related to structural fixation and incomplete information utilization. To address these limitations, we propose a novel MIL framework named Multi-order MIL (MoMIL). Our framework utilizes the SSD model to perform long-sequence modeling on multi-order WSI patches and combines lightweight feature fusion to achieve more comprehensive feature information utilization. This framework supports the fusion of a broader range of features and is highly flexible, allowing for expansion based on specific usage requirements. Additionally, we introduce a sequence transformation method specifically designed for WSIs. This method is not only adaptable to different WSI sizes but also captures additional feature expression, resulting in a more effective exploitation of sequential cues. Extensive experiments demonstrate that MoMIL surpasses state-of-the-art MIL methods, up to 0.027 AUC improvements for cancer sub-typing. We conducted extensive experiments on three downstream tasks with a total of five datasets, achieving improvements in all performance metrics. The code is available at <span><span>https://github.com/YuqiZhang-Buaa/MoMIL</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"167 ","pages":"Article 105918"},"PeriodicalIF":4.2,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146078416","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

MSTPFormer: Mamba-driven spatiotemporal bidirectional dual-stream parallel transformer for 3D human pose estimation MSTPFormer：用于三维人体姿态估计的mamba驱动的时空双向双流并联变压器

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2026-03-01 Epub Date: 2026-01-14 DOI: 10.1016/j.imavis.2026.105912

Tiandi Peng , Yanmin Luo , Jiancong Liang , Gonggeng Lin

Monocular 3D human pose estimation from video sequences requires effectively capturing both spatial and temporal information. However, ensuring long-term temporal consistency while maintaining accurate local motion remains a major challenge. In this paper, we present MSTPFormer, a dual-branch framework that separately models global temporal dynamics and local spatial representations for robust spatio-temporal learning. To model global motion, We design two modules based on the state space mechanism (SSM) of Mamba. The Spatial Scan Block (S-Scan) applies a bidirectional spatial scanning strategy to form closed-loop joint interactions, enhancing local motion chain representation. The Temporal Scan Block (T-Scan) constructs joint-specific temporal channels along the sequence, enabling individualized motion trajectory modeling for each of the 17 joints. For local modeling, we design a Transformer branch to refine spatial features within each frame, thereby enhancing the expressiveness of joint-level details. This dual-branch design enables effective decoupling and fusion of global–local and spatial–temporal cues. Experiments on Human3.6M and MPI-INF-3DHP demonstrate that MSTPFormer achieves state-of-the-art performance, with P1 errors of 37.6 mm on Human3.6M and 13.6 mm on MPI-INF-3DHP.

从视频序列中进行单目三维人体姿态估计需要有效地捕获空间和时间信息。然而，确保长期的时间一致性，同时保持准确的局部运动仍然是主要的挑战。在本文中，我们提出了MSTPFormer，这是一个双分支框架，它分别对全局时间动态和局部空间表示进行建模，以实现健壮的时空学习。为了模拟全局运动，我们基于曼巴的状态空间机制（SSM）设计了两个模块。空间扫描块（S-Scan）采用双向空间扫描策略形成闭环关节相互作用，增强局部运动链表征。时间扫描块（T-Scan）沿着序列构建关节特定的时间通道，为17个关节中的每个关节实现个性化的运动轨迹建模。对于局部建模，我们设计了一个Transformer分支来细化每帧内的空间特征，从而增强了联合级细节的表现力。这种双分支设计能够有效地解耦和融合全局-局部和时空线索。在Human3.6M和MPI-INF-3DHP上的实验表明，MSTPFormer达到了最先进的性能，在Human3.6M和MPI-INF-3DHP上的P1误差分别为37.6 mm和13.6 mm。

{"title":"MSTPFormer: Mamba-driven spatiotemporal bidirectional dual-stream parallel transformer for 3D human pose estimation","authors":"Tiandi Peng , Yanmin Luo , Jiancong Liang , Gonggeng Lin","doi":"10.1016/j.imavis.2026.105912","DOIUrl":"10.1016/j.imavis.2026.105912","url":null,"abstract":"<div><div>Monocular 3D human pose estimation from video sequences requires effectively capturing both spatial and temporal information. However, ensuring long-term temporal consistency while maintaining accurate local motion remains a major challenge. In this paper, we present MSTPFormer, a dual-branch framework that separately models global temporal dynamics and local spatial representations for robust spatio-temporal learning. To model global motion, We design two modules based on the state space mechanism (SSM) of Mamba. The Spatial Scan Block (S-Scan) applies a bidirectional spatial scanning strategy to form closed-loop joint interactions, enhancing local motion chain representation. The Temporal Scan Block (T-Scan) constructs joint-specific temporal channels along the sequence, enabling individualized motion trajectory modeling for each of the 17 joints. For local modeling, we design a Transformer branch to refine spatial features within each frame, thereby enhancing the expressiveness of joint-level details. This dual-branch design enables effective decoupling and fusion of global–local and spatial–temporal cues. Experiments on Human3.6M and MPI-INF-3DHP demonstrate that MSTPFormer achieves state-of-the-art performance, with P1 errors of 37.6 mm on Human3.6M and 13.6 mm on MPI-INF-3DHP.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"167 ","pages":"Article 105912"},"PeriodicalIF":4.2,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146023143","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Long-FAS: Cross-domain face anti-spoofing with long text guidance long - fas：长文本引导的跨域人脸防欺骗

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2026-03-01 Epub Date: 2026-01-06 DOI: 10.1016/j.imavis.2026.105901

Jianwen Zhang , Jianfeng Zhang , Dedong Yang, Rongtao Li, Ziyang Li

Recent studies have demonstrated that utilizing natural language as a supervisory signal can enhance face anti-spoofing (FAS) performance; however, these methods still fall short in fully addressing long-text inputs and fine-grained information. To mitigate these limitations, we leverage MiniGPT-4 to generate detailed long-form textual descriptions of facial features for input images, and propose a novel framework, Long-FAS, which extracts textual and visual information through a dual-branch architecture. Specifically, we incorporate positional encoding for knowledge retention to enable the learning of effective feature representations from long texts, and employ principal component analysis (PCA) matching to capture essential attribute information while prioritizing critical attributes. Furthermore, matching visual and textual features at both coarse and fine granularities enhances the model’s ability to effectively handle both long and short texts, thereby empowering it to learn robust discriminative cues from facial images. Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art counterparts.

最近的研究表明，利用自然语言作为监督信号可以提高人脸抗欺骗（FAS）性能；然而，这些方法仍然不能完全处理长文本输入和细粒度信息。为了减轻这些限制，我们利用MiniGPT-4为输入图像生成详细的长篇面部特征文本描述，并提出了一个新的框架Long-FAS，它通过双分支架构提取文本和视觉信息。具体来说，我们将位置编码用于知识保留，以便从长文本中学习有效的特征表示，并使用主成分分析（PCA）匹配来捕获基本属性信息，同时对关键属性进行优先级排序。此外，在粗粒度和细粒度上匹配视觉和文本特征增强了模型有效处理长文本和短文本的能力，从而使其能够从面部图像中学习稳健的判别线索。大量的实验表明，我们的方法明显优于最先进的同行。

{"title":"Long-FAS: Cross-domain face anti-spoofing with long text guidance","authors":"Jianwen Zhang , Jianfeng Zhang , Dedong Yang, Rongtao Li, Ziyang Li","doi":"10.1016/j.imavis.2026.105901","DOIUrl":"10.1016/j.imavis.2026.105901","url":null,"abstract":"<div><div>Recent studies have demonstrated that utilizing natural language as a supervisory signal can enhance face anti-spoofing (FAS) performance; however, these methods still fall short in fully addressing long-text inputs and fine-grained information. To mitigate these limitations, we leverage MiniGPT-4 to generate detailed long-form textual descriptions of facial features for input images, and propose a novel framework, Long-FAS, which extracts textual and visual information through a dual-branch architecture. Specifically, we incorporate positional encoding for knowledge retention to enable the learning of effective feature representations from long texts, and employ principal component analysis (PCA) matching to capture essential attribute information while prioritizing critical attributes. Furthermore, matching visual and textual features at both coarse and fine granularities enhances the model’s ability to effectively handle both long and short texts, thereby empowering it to learn robust discriminative cues from facial images. Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art counterparts.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"167 ","pages":"Article 105901"},"PeriodicalIF":4.2,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145927625","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Distributed quantum model learning for traffic density estimation 交通密度估计的分布式量子模型学习

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2026-03-01 Epub Date: 2026-01-06 DOI: 10.1016/j.imavis.2026.105900

Kewen Wang, Bin Wang, Wenzhe Zhai, Jing-an Cheng

In Intelligent Autonomous Transport Systems (IATS), the integration of lightweight machine learning techniques enables the deployment of real-time and efficient AI models on edge devices. A fundamental aspect is to estimate traffic density, which is crucial for efficient intelligent traffic control. The rapid progress in deep neural networks (DNNs) has led to a notable improvement in the accuracy of traffic density estimation. However, two main issues remain unsolved. Firstly, current DNN models involve numerous parameters and consume large computing resources, and thus their performance degrades when detecting multi-scale vehicle targets. Secondly, growing privacy concerns have made individuals increasingly unwilling to share their data for model training, which leads to data isolation challenges. To address the problems above, we introduce the Distributed Quantum Model Learning (DQML) model for traffic density estimation. It combines an Efficient Quantum-driven Adaptive (EQA) module to capture multi-scale information using quantum states. In addition, we propose a distributed learning strategy that trains multiple client models with local data and aggregates them via a global parameter server. This strategy ensures privacy protection while offering a significant improvement in estimation performance compared to models trained on limited and isolated data. We evaluated the proposed model on six key benchmarks for vehicle and crowd density analysis, and comprehensive experiments demonstrated that it surpasses other state-of-the-art models in both accuracy and efficiency.

在智能自主运输系统（IATS）中，轻量级机器学习技术的集成可以在边缘设备上部署实时高效的人工智能模型。交通密度估计是实现高效智能交通控制的一个重要方面。深度神经网络（dnn）的快速发展使得交通密度估计的准确性得到了显著提高。然而，两个主要问题仍未解决。首先，目前的深度神经网络模型涉及的参数多，计算资源消耗大，在检测多尺度车辆目标时性能下降。其次，越来越多的隐私问题使得个人越来越不愿意分享他们的数据用于模型训练，这导致了数据隔离的挑战。为了解决上述问题，我们引入分布式量子模型学习（DQML）模型用于交通密度估计。它结合了一个高效量子驱动自适应（EQA）模块，利用量子态捕获多尺度信息。此外，我们提出了一种分布式学习策略，该策略使用本地数据训练多个客户端模型，并通过全局参数服务器聚合它们。与在有限和孤立数据上训练的模型相比，该策略确保了隐私保护，同时显著提高了估计性能。我们在车辆和人群密度分析的六个关键基准上对所提出的模型进行了评估，综合实验表明，它在准确性和效率方面都优于其他最先进的模型。

{"title":"Distributed quantum model learning for traffic density estimation","authors":"Kewen Wang, Bin Wang, Wenzhe Zhai, Jing-an Cheng","doi":"10.1016/j.imavis.2026.105900","DOIUrl":"10.1016/j.imavis.2026.105900","url":null,"abstract":"<div><div>In Intelligent Autonomous Transport Systems (IATS), the integration of lightweight machine learning techniques enables the deployment of real-time and efficient AI models on edge devices. A fundamental aspect is to estimate traffic density, which is crucial for efficient intelligent traffic control. The rapid progress in deep neural networks (DNNs) has led to a notable improvement in the accuracy of traffic density estimation. However, two main issues remain unsolved. Firstly, current DNN models involve numerous parameters and consume large computing resources, and thus their performance degrades when detecting multi-scale vehicle targets. Secondly, growing privacy concerns have made individuals increasingly unwilling to share their data for model training, which leads to data isolation challenges. To address the problems above, we introduce the Distributed Quantum Model Learning (DQML) model for traffic density estimation. It combines an Efficient Quantum-driven Adaptive (EQA) module to capture multi-scale information using quantum states. In addition, we propose a distributed learning strategy that trains multiple client models with local data and aggregates them via a global parameter server. This strategy ensures privacy protection while offering a significant improvement in estimation performance compared to models trained on limited and isolated data. We evaluated the proposed model on six key benchmarks for vehicle and crowd density analysis, and comprehensive experiments demonstrated that it surpasses other state-of-the-art models in both accuracy and efficiency.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"167 ","pages":"Article 105900"},"PeriodicalIF":4.2,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145927626","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Compositional Gamba for 3D human pose estimation 合成甘巴三维人体姿态估计

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2026-03-01 Epub Date: 2026-01-19 DOI: 10.1016/j.imavis.2026.105913

Lu Zhou , Yingying Chen , Jinqiao Wang

GCNs (graph convolutional networks) based 2D to 3D human pose estimation has sparked a wave of research and garnered widespread attention profited from its strong competence in joint relation modeling. Yet, the performance still lags behind on account of the scarcity of universal and sophisticated human knowledge. Advancements in state space models, notably Mamba, which demonstrates extraordinary sequential modeling talents has proved its effectiveness on long sequence modeling and macro knowledge acquisition. To alleviate the modeling bias in existing techniques, we advance an innovative hybrid architecture where GCNs are married with the Mamba to learn the multi-level human knowledge in a collaborative manner which is an effective manner to conquer the dilemma caused by the ill-posed issue. Concretely, we design a compositional Gamba (GCNs-Mamba) block where GCNs and Mamba enforce the local–global modeling upon different feature segments alternatively. Additionally, a compositional pattern is skillfully formulated in which multi-level human topological relation is learned and explicit human prior is embedded. The proposed approach outperforms the preceding published works on both the Human3.6M and MPI-INF-3DHP benchmarks, attesting to the efficacy of the hybrid architecture.

基于图形卷积网络（GCNs）的二维到三维人体姿态估计因其在关节关系建模方面的强大能力而引起了广泛的研究和关注。然而，由于缺乏普遍而复杂的人类知识，这种表现仍然落后。状态空间模型的发展，尤其是表现出非凡序列建模天赋的Mamba，已经证明了它在长序列建模和宏观知识获取方面的有效性。为了减轻现有技术中的建模偏差，我们提出了一种创新的混合架构，将GCNs与曼巴结合起来，以协作的方式学习多层次的人类知识，这是克服病态问题所带来的困境的有效方法。具体来说，我们设计了一个组合Gamba （GCNs-Mamba）块，其中GCNs和Mamba在不同的特征段上交替执行局部全局建模。此外，巧妙地制定了一个组合模式，其中学习了多层次的人类拓扑关系，并嵌入了明确的人类先验。所提出的方法在Human3.6M和MPI-INF-3DHP基准测试上都优于之前发表的作品，证明了混合架构的有效性。

{"title":"Compositional Gamba for 3D human pose estimation","authors":"Lu Zhou , Yingying Chen , Jinqiao Wang","doi":"10.1016/j.imavis.2026.105913","DOIUrl":"10.1016/j.imavis.2026.105913","url":null,"abstract":"<div><div>GCNs (graph convolutional networks) based 2D to 3D human pose estimation has sparked a wave of research and garnered widespread attention profited from its strong competence in joint relation modeling. Yet, the performance still lags behind on account of the scarcity of universal and sophisticated human knowledge. Advancements in state space models, notably Mamba, which demonstrates extraordinary sequential modeling talents has proved its effectiveness on long sequence modeling and macro knowledge acquisition. To alleviate the modeling bias in existing techniques, we advance an innovative hybrid architecture where GCNs are married with the Mamba to learn the multi-level human knowledge in a collaborative manner which is an effective manner to conquer the dilemma caused by the ill-posed issue. Concretely, we design a compositional Gamba (GCNs-Mamba) block where GCNs and Mamba enforce the local–global modeling upon different feature segments alternatively. Additionally, a compositional pattern is skillfully formulated in which multi-level human topological relation is learned and explicit human prior is embedded. The proposed approach outperforms the preceding published works on both the Human3.6M and MPI-INF-3DHP benchmarks, attesting to the efficacy of the hybrid architecture.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"167 ","pages":"Article 105913"},"PeriodicalIF":4.2,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146023142","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Cross-level fusion network for two-stage polyp segmentation via integrity learning 基于完整性学习的两阶段息肉分割交叉融合网络

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2026-03-01 Epub Date: 2025-12-19 DOI: 10.1016/j.imavis.2025.105883

Junzhuo Liu , Dorit Merhof , Zhixiang Wang

Colorectal cancer is one of the most prevalent and lethal forms of cancer. The automated detection, segmentation and classification of early polyp tissues from endoscopy images of the colorectum has demonstrated impressive potential in improving clinical diagnostic accuracy, avoiding missed detections and reducing the incidence of colorectal cancer in the population. However, most existing studies fail to consider the potential of information fusion between different deep neural network layers and optimization with respect to model complexity, resulting in poor clinical utility. To address the above limitations, the concept of integrity learning is introduced, which divides polyp segmentation into two stages for progressive completion, and a cross-level fusion lightweight network, IC-FusionNet, is proposed to accurately segment polyps from endoscopy images. First, the Context Fusion Module (CFM) of the network aggregates the encoder neighboring branches and current level information to achieve macro-integrity learning. In the second stage, polyp detail information from shallower layers and deeper high-dimensional semantic information are aggregated to achieve enhancement between different layers of complementary information. IC-FusionNet is evaluated on five polyp segmentation benchmark datasets across eight evaluation metrics to assess its performance. IC-FusionNet achieves

mDice

of 0.908 and 0.925 on the Kvasir and CVC-ClinicDB datasets, respectively, along with

mIou

of 0.851 and 0.973. On three external polyp segmentation test datasets, the model obtains an average

mDice

of 0.788 and an average

mIou

of 0.712. Compared to existing methods, IC-FusionNet achieves superior or near-optimal performance across most evaluation metrics. Moreover, IC-FusionNet contains only 3.84 M parameters and 0.76G MACs, representing a reduction of 9.22% in parameter count and 74.15% in computational complexity compared to recent lightweight segmentation networks.

结直肠癌是最常见和最致命的癌症之一。从结直肠内镜图像中对早期息肉组织进行自动检测、分割和分类，在提高临床诊断准确性、避免漏诊和降低人群中结直肠癌的发病率方面显示出令人印象深刻的潜力。然而，现有的研究大多没有考虑不同深度神经网络层之间信息融合的潜力和模型复杂性优化，导致临床实用性较差。针对上述局限性，本文引入了完整性学习的概念，将息肉分割分为两个阶段逐步完成，并提出了一个跨层次融合的轻量级网络IC-FusionNet，从内镜图像中准确分割息肉。首先，网络的上下文融合模块（Context Fusion Module， CFM）对编码器相邻分支和当前级别信息进行聚合，实现宏观完整性学习；第二阶段，将较浅层的息肉细节信息与较深层的高维语义信息进行聚合，实现不同层间互补信息的增强。IC-FusionNet在5个息肉分割基准数据集上进行了8个评估指标的评估，以评估其性能。IC-FusionNet在Kvasir和CVC-ClinicDB数据集上的mDice分别为0.908和0.925，mIou分别为0.851和0.973。在三个外部息肉分割测试数据集上，该模型的平均mdevice为0.788，平均mIou为0.712。与现有方法相比，IC-FusionNet在大多数评估指标上都达到了卓越或接近最佳的性能。此外，IC-FusionNet仅包含3.84 M个参数和0.76G个mac，与最近的轻量级分段网络相比，参数数量减少了9.22%，计算复杂度减少了74.15%。

{"title":"Cross-level fusion network for two-stage polyp segmentation via integrity learning","authors":"Junzhuo Liu , Dorit Merhof , Zhixiang Wang","doi":"10.1016/j.imavis.2025.105883","DOIUrl":"10.1016/j.imavis.2025.105883","url":null,"abstract":"<div><div>Colorectal cancer is one of the most prevalent and lethal forms of cancer. The automated detection, segmentation and classification of early polyp tissues from endoscopy images of the colorectum has demonstrated impressive potential in improving clinical diagnostic accuracy, avoiding missed detections and reducing the incidence of colorectal cancer in the population. However, most existing studies fail to consider the potential of information fusion between different deep neural network layers and optimization with respect to model complexity, resulting in poor clinical utility. To address the above limitations, the concept of integrity learning is introduced, which divides polyp segmentation into two stages for progressive completion, and a cross-level fusion lightweight network, IC-FusionNet, is proposed to accurately segment polyps from endoscopy images. First, the Context Fusion Module (CFM) of the network aggregates the encoder neighboring branches and current level information to achieve macro-integrity learning. In the second stage, polyp detail information from shallower layers and deeper high-dimensional semantic information are aggregated to achieve enhancement between different layers of complementary information. IC-FusionNet is evaluated on five polyp segmentation benchmark datasets across eight evaluation metrics to assess its performance. IC-FusionNet achieves <span><math><mi>mDice</mi></math></span> of 0.908 and 0.925 on the Kvasir and CVC-ClinicDB datasets, respectively, along with <span><math><mi>mIou</mi></math></span> of 0.851 and 0.973. On three external polyp segmentation test datasets, the model obtains an average <span><math><mi>mDice</mi></math></span> of 0.788 and an average <span><math><mi>mIou</mi></math></span> of 0.712. Compared to existing methods, IC-FusionNet achieves superior or near-optimal performance across most evaluation metrics. Moreover, IC-FusionNet contains only 3.84 M parameters and 0.76G MACs, representing a reduction of 9.22% in parameter count and 74.15% in computational complexity compared to recent lightweight segmentation networks.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"167 ","pages":"Article 105883"},"PeriodicalIF":4.2,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145842587","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Enhanced medical image segmentation via synergistic feature guidance and multi-scale refinement 基于协同特征引导和多尺度细化的医学图像分割

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2026-03-01 Epub Date: 2026-01-20 DOI: 10.1016/j.imavis.2026.105914

Shaoqiang Wang , Guiling Shi , Xiaofeng Xu , Tiyao Liu , Yawu Zhao , Xiaochun Cheng , Yuchen Wang

Medical image segmentation is pivotal for clinical diagnosis but remains challenged by the inherent trade-offs between global context modeling and local detail preservation, as well as the susceptibility of deep networks to acquisition noise and scale variations. While hybrid CNN-Transformer architectures have emerged to address receptive field limitations, they often incur prohibitive computational costs and lack the inductive bias required for small-sample medical datasets. To resolve these systemic bottlenecks efficiently, we propose SFRNet V2. By integrating parallel local-regional perception, active noise filtration in skip connections, and elastic multi-scale aggregation at the bottleneck, our approach systematically overcomes the limitations of fixed receptive fields and feature ambiguity. Extensive experiments on four diverse public datasets (CVC-ClinicDB, ISIC 2017, TN3K, and MICCAI Tooth) demonstrate that SFRNet V2 consistently outperforms recent competitors. Notably, our model achieves the highest accuracy with only 19.85 M parameters and a rapid inference speed of 2.7 ms, offering a superior balance between precision and clinical deployability.

医学图像分割对临床诊断至关重要，但仍然受到全局上下文建模和局部细节保存之间固有权衡的挑战，以及深度网络对采集噪声和尺度变化的敏感性。虽然混合CNN-Transformer架构已经出现，以解决接受场限制，但它们通常会产生过高的计算成本，并且缺乏小样本医疗数据集所需的诱导偏差。为了有效地解决这些系统瓶颈，我们提出了SFRNet V2。该方法通过在瓶颈处集成并行局部区域感知、跳跃连接中的主动噪声过滤和弹性多尺度聚合，系统地克服了固定接受域和特征模糊的局限性。在四个不同的公共数据集（CVC-ClinicDB、ISIC 2017、TN3K和MICCAI Tooth）上进行的大量实验表明，SFRNet V2的性能始终优于最近的竞争对手。值得注意的是，我们的模型达到了最高的精度，只有19.85 M个参数和2.7 ms的快速推理速度，在精度和临床可部署性之间提供了卓越的平衡。

{"title":"Enhanced medical image segmentation via synergistic feature guidance and multi-scale refinement","authors":"Shaoqiang Wang , Guiling Shi , Xiaofeng Xu , Tiyao Liu , Yawu Zhao , Xiaochun Cheng , Yuchen Wang","doi":"10.1016/j.imavis.2026.105914","DOIUrl":"10.1016/j.imavis.2026.105914","url":null,"abstract":"<div><div>Medical image segmentation is pivotal for clinical diagnosis but remains challenged by the inherent trade-offs between global context modeling and local detail preservation, as well as the susceptibility of deep networks to acquisition noise and scale variations. While hybrid CNN-Transformer architectures have emerged to address receptive field limitations, they often incur prohibitive computational costs and lack the inductive bias required for small-sample medical datasets. To resolve these systemic bottlenecks efficiently, we propose SFRNet V2. By integrating parallel local-regional perception, active noise filtration in skip connections, and elastic multi-scale aggregation at the bottleneck, our approach systematically overcomes the limitations of fixed receptive fields and feature ambiguity. Extensive experiments on four diverse public datasets (CVC-ClinicDB, ISIC 2017, TN3K, and MICCAI Tooth) demonstrate that SFRNet V2 consistently outperforms recent competitors. Notably, our model achieves the highest accuracy with only 19.85 M parameters and a rapid inference speed of 2.7 ms, offering a superior balance between precision and clinical deployability.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"167 ","pages":"Article 105914"},"PeriodicalIF":4.2,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146023141","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

CMALDD-PTAF: Cross-modal adversarial learning for deepfake detection by leveraging pre-trained models and cross-attention fusion CMALDD-PTAF：利用预训练模型和交叉注意融合进行深度假检测的跨模态对抗学习

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2026-02-01 Epub Date: 2025-12-20 DOI: 10.1016/j.imavis.2025.105885

Yuanfan Jin, Yongfang Wang

The emergence of novel deepfake algorithms capable of generating highly realistic manipulated audio–visual content has sparked significant public concern regarding the authenticity and trustworthiness of digital media. This concern has driven the development of multimodal deepfake detection methods. In this paper, we present a novel two-stage multimodal detection framework that harnesses pre-trained audio–visual speech recognition models and cross-attention fusion to achieve state-of-the-art performance with efficient cross-domain adversarial training. Our approach consists of two stages. In the first stage, we utilize a pre-trained audio–visual representation learning model from the speech recognition domain to extract unimodal features. Comprehensive analysis confirms the efficacy of these features for deepfake detection. In the second stage, we propose a specialized cross-modality fusion module to integrate the unimodal features for multimodal deepfake detection. Furthermore, we utilize a transformer model for final classification and implement an adversarial learning strategy to enhance robustness of the model. Our proposed method achieves 98.9% accuracy and 99.6% AUC on the multimodal deepfake detection benchmark FakeAVCeleb, outperforming the latest multimodal detector NPVForensics by 0.57 percentage points in AUC , while maintaining low training cost and a relatively simple architecture.

新型深度假算法的出现能够生成高度逼真的操纵视听内容，这引发了公众对数字媒体真实性和可信度的极大关注。这种担忧推动了多模态深度假检测方法的发展。在本文中，我们提出了一种新的两阶段多模态检测框架，该框架利用预训练的视听语音识别模型和交叉注意融合，通过高效的跨域对抗训练实现最先进的性能。我们的方法包括两个阶段。在第一阶段，我们利用来自语音识别领域的预训练的视听表示学习模型来提取单峰特征。综合分析证实了这些特征在深度造假检测中的有效性。在第二阶段，我们提出了一个专门的跨模态融合模块来整合单模态特征，用于多模态深度伪造检测。此外，我们利用变压器模型进行最终分类，并实施对抗学习策略来增强模型的鲁棒性。本文提出的方法在多模态深度伪造检测基准FakeAVCeleb上达到了98.9%的准确率和99.6%的AUC， AUC比最新的多模态检测器NPVForensics高出0.57个百分点，同时保持了较低的训练成本和相对简单的架构。

{"title":"CMALDD-PTAF: Cross-modal adversarial learning for deepfake detection by leveraging pre-trained models and cross-attention fusion","authors":"Yuanfan Jin, Yongfang Wang","doi":"10.1016/j.imavis.2025.105885","DOIUrl":"10.1016/j.imavis.2025.105885","url":null,"abstract":"<div><div>The emergence of novel deepfake algorithms capable of generating highly realistic manipulated audio–visual content has sparked significant public concern regarding the authenticity and trustworthiness of digital media. This concern has driven the development of multimodal deepfake detection methods. In this paper, we present a novel two-stage multimodal detection framework that harnesses pre-trained audio–visual speech recognition models and cross-attention fusion to achieve state-of-the-art performance with efficient cross-domain adversarial training. Our approach consists of two stages. In the first stage, we utilize a pre-trained audio–visual representation learning model from the speech recognition domain to extract unimodal features. Comprehensive analysis confirms the efficacy of these features for deepfake detection. In the second stage, we propose a specialized cross-modality fusion module to integrate the unimodal features for multimodal deepfake detection. Furthermore, we utilize a transformer model for final classification and implement an adversarial learning strategy to enhance robustness of the model. Our proposed method achieves 98.9% accuracy and 99.6% AUC on the multimodal deepfake detection benchmark FakeAVCeleb, outperforming the latest multimodal detector NPVForensics by 0.57 percentage points in AUC , while maintaining low training cost and a relatively simple architecture.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"166 ","pages":"Article 105885"},"PeriodicalIF":4.2,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145840004","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

ShadowMamba: State-space model with boundary-region selective scan for shadow removal ShadowMamba：具有边界区域选择性扫描的状态空间模型，用于阴影去除

IF 4.2 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Image and Vision Computing

Pub Date : 2026-02-01 Epub Date: 2025-12-12 DOI: 10.1016/j.imavis.2025.105872

Xiujin Zhu , Chee-Onn Chow , Joon Huang Chuah

Image shadow removal is a typical low-level vision task, as shadows introduce abrupt local brightness variations that degrade the performance of downstream tasks. Due to the quadratic complexity of Transformers, many existing methods adopt local attention to balance accuracy and efficiency. However, restricting attention to local windows prevents true long-range dependency modeling and limits shadow removal performance. Recently, Mamba has shown strong ability in vision tasks by achieving global modeling with linear complexity. Despite this advantage, existing scanning mechanisms in the Mamba architecture are not suitable for shadow removal because they ignore the semantic continuity within the same region. To address this, a boundary-region selective scanning mechanism is proposed that captures local details while enhancing continuity among semantically related pixels, effectively improving shadow removal performance. In addition, a shadow mask denoising preprocessing method is introduced to improve the accuracy of the scanning mechanism and further enhance the data quality. Based on this, this paper presents ShadowMamba, the first Mamba-based model for shadow removal. Experimental results show that the proposed method outperforms existing mainstream approaches on the AISTD, ISTD, SRD, and WSRD+ datasets, and demonstrates good generalization ability in cross-dataset testing on USR and SBU. Meanwhile, the model also has significant advantages in parameter efficiency and computational complexity. Code is available at: https://github.com/ZHUXIUJINChris/ShadowMamba.

图像阴影去除是一项典型的低层次视觉任务，因为阴影会引入突然的局部亮度变化，从而降低下游任务的性能。由于变压器的二次复杂度，现有的许多方法都是局部关注平衡精度和效率。然而，限制对本地窗口的关注妨碍了真正的远程依赖关系建模，并限制了阴影去除的性能。最近，曼巴在视觉任务中表现出了很强的能力，实现了线性复杂性的全局建模。尽管有这样的优势，Mamba架构中现有的扫描机制并不适合阴影去除，因为它们忽略了同一区域内的语义连续性。为了解决这个问题，提出了一种边界区域选择性扫描机制，在捕获局部细节的同时增强语义相关像素之间的连续性，有效提高阴影去除性能。此外，为了提高扫描机构的精度，进一步提高数据质量，还引入了一种阴影掩模去噪预处理方法。基于此，本文提出了ShadowMamba，这是第一个基于mamba的阴影去除模型。实验结果表明，该方法在AISTD、ISTD、SRD和WSRD+数据集上优于现有主流方法，并在USR和SBU上表现出良好的跨数据集测试泛化能力。同时，该模型在参数效率和计算复杂度方面也具有显著的优势。代码可从https://github.com/ZHUXIUJINChris/ShadowMamba获得。

{"title":"ShadowMamba: State-space model with boundary-region selective scan for shadow removal","authors":"Xiujin Zhu , Chee-Onn Chow , Joon Huang Chuah","doi":"10.1016/j.imavis.2025.105872","DOIUrl":"10.1016/j.imavis.2025.105872","url":null,"abstract":"<div><div>Image shadow removal is a typical low-level vision task, as shadows introduce abrupt local brightness variations that degrade the performance of downstream tasks. Due to the quadratic complexity of Transformers, many existing methods adopt local attention to balance accuracy and efficiency. However, restricting attention to local windows prevents true long-range dependency modeling and limits shadow removal performance. Recently, Mamba has shown strong ability in vision tasks by achieving global modeling with linear complexity. Despite this advantage, existing scanning mechanisms in the Mamba architecture are not suitable for shadow removal because they ignore the semantic continuity within the same region. To address this, a boundary-region selective scanning mechanism is proposed that captures local details while enhancing continuity among semantically related pixels, effectively improving shadow removal performance. In addition, a shadow mask denoising preprocessing method is introduced to improve the accuracy of the scanning mechanism and further enhance the data quality. Based on this, this paper presents ShadowMamba, the first Mamba-based model for shadow removal. Experimental results show that the proposed method outperforms existing mainstream approaches on the AISTD, ISTD, SRD, and WSRD+ datasets, and demonstrates good generalization ability in cross-dataset testing on USR and SBU. Meanwhile, the model also has significant advantages in parameter efficiency and computational complexity. Code is available at: <span><span>https://github.com/ZHUXIUJINChris/ShadowMamba</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50374,"journal":{"name":"Image and Vision Computing","volume":"166 ","pages":"Article 105872"},"PeriodicalIF":4.2,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145737140","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0