Digital Signal Processing最新文献

英文中文

Selective collaboration in distributed FxLMS active noise control systems 分布式 FxLMS 主动噪声控制系统中的选择性协作

IF 2.9 3区工程技术 Q2 ENGINEERING, ELECTRICAL & ELECTRONIC

Digital Signal Processing

Pub Date : 2024-10-24 DOI: 10.1016/j.dsp.2024.104829

Víctor M. García-Mollá , Miguel Ferrer , Maria de Diego , Alberto Gonzalez

This paper addresses the selection of the collaboration configuration on distributed active noise control (ANC) systems. ANC systems aim to cancel out acoustic noise within a listening area. In distributed systems, the control task is delegated among multiple acoustic control nodes that generate the control signals by filtering a noise reference signal. The coefficients of each node filter are iteratively calculated using the filtered-X LMS algorithm. The stability is achieved when the adaptive filters computed in each node converge to finite values. However, acoustic coupling among nodes could lead to instability (i.e., divergence). Collaboration among selected nodes may avoid this phenomenon, although not just any collaboration configuration guarantees network stability. On the other hand, a collaborative distributed system presents two drawbacks: the stability assessment is computationally expensive, and communication requirements increase with the number of collaborations among nodes. In this paper, we propose and discuss several methods to establish a collaboration configuration that ensures system stability. The optimal configuration, which is characterized by the minimal number of necessary collaborations between nodes, can be identified through exhaustive search. However, this approach incurs a high computational cost, particularly in networks with many nodes. To address this challenge, we introduce several heuristic methods aimed at efficiently obtaining stable configurations.

本文探讨了分布式主动噪声控制（ANC）系统协作配置的选择问题。ANC 系统旨在消除聆听区域内的声学噪音。在分布式系统中，控制任务分配给多个声学控制节点，这些节点通过过滤噪声参考信号产生控制信号。每个节点滤波器的系数采用滤波-X LMS 算法迭代计算。当每个节点计算出的自适应滤波器收敛到有限值时，就实现了稳定性。然而，节点之间的声耦合可能会导致不稳定性（即发散）。选定节点之间的协作可以避免这种现象，但并非任何协作配置都能保证网络的稳定性。另一方面，协作分布式系统有两个缺点：稳定性评估的计算成本很高，而且通信需求会随着节点间协作数量的增加而增加。在本文中，我们提出并讨论了几种建立确保系统稳定性的协作配置的方法。最优配置的特点是节点间必要协作的数量最少，可通过穷举搜索确定。然而，这种方法的计算成本很高，尤其是在节点众多的网络中。为了应对这一挑战，我们引入了几种启发式方法，旨在高效地获得稳定配置。

{"title":"Selective collaboration in distributed FxLMS active noise control systems","authors":"Víctor M. García-Mollá , Miguel Ferrer , Maria de Diego , Alberto Gonzalez","doi":"10.1016/j.dsp.2024.104829","DOIUrl":"10.1016/j.dsp.2024.104829","url":null,"abstract":"<div><div>This paper addresses the selection of the collaboration configuration on distributed active noise control (ANC) systems. ANC systems aim to cancel out acoustic noise within a listening area. In distributed systems, the control task is delegated among multiple acoustic control nodes that generate the control signals by filtering a noise reference signal. The coefficients of each node filter are iteratively calculated using the filtered-X LMS algorithm. The stability is achieved when the adaptive filters computed in each node converge to finite values. However, acoustic coupling among nodes could lead to instability (i.e., divergence). Collaboration among selected nodes may avoid this phenomenon, although not just any collaboration configuration guarantees network stability. On the other hand, a collaborative distributed system presents two drawbacks: the stability assessment is computationally expensive, and communication requirements increase with the number of collaborations among nodes. In this paper, we propose and discuss several methods to establish a collaboration configuration that ensures system stability. The optimal configuration, which is characterized by the minimal number of necessary collaborations between nodes, can be identified through exhaustive search. However, this approach incurs a high computational cost, particularly in networks with many nodes. To address this challenge, we introduce several heuristic methods aimed at efficiently obtaining stable configurations.</div></div>","PeriodicalId":51011,"journal":{"name":"Digital Signal Processing","volume":"156 ","pages":"Article 104829"},"PeriodicalIF":2.9,"publicationDate":"2024-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142531118","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Physical Knowledge and Data-Driven Technologies for Integrated Sensing, Communication and Computing 用于综合传感、通信和计算的物理知识和数据驱动技术

IF 2.9 3区工程技术 Q2 ENGINEERING, ELECTRICAL & ELECTRONIC

Digital Signal Processing

Pub Date : 2024-10-23 DOI: 10.1016/j.dsp.2024.104832

Sicong Liu , Kaishun Wu , Yongpan Zou , Ruiqi Liu , Marco Di Renzo , Octavia A. Dobre

引用次数: 0

Graph total variation and low-rank regularization for heterogeneous change detection 用于异质变化检测的图总变化和低秩正则化

IF 2.9 3区工程技术 Q2 ENGINEERING, ELECTRICAL & ELECTRONIC

Digital Signal Processing

Pub Date : 2024-10-21 DOI: 10.1016/j.dsp.2024.104825

Jichao Yao , Junzheng Jiang , Fang Zhou

Heterogeneous change detection (HCD) is challenging because different imaging mechanisms for various sensors make images difficult to compare directly. To address this problem, a graph-based regression algorithm is proposed for HCD, by leveraging the Graph Total Variation regularization and Low-Rank matrices decomposition (GTVLR). Utilizing graph signal processing (GSP) theory, a directed graph (digraph) model is employed to effectively represent the orientation and correlation information of images, thereby enabling direct comparison of heterogeneous data within the same domain after graph filtering. The GTVLR framework facilitates the decomposition of post-event images into regression and changed images. This decomposition ensures that the regression image mirrors the structure similarity of the pre-event image, while the changed image highlights areas of alteration, aiding in change detection. The model characterizes the piecewise smoothness and Low-Rank properties of data through GTV regularization and Low-Rank penalty, respectively. Moreover, by integrating the higher-order neighboring information within the digraph to refine the model. Experiments conducted on three real-world datasets and comparison with several state-of-the-art methods demonstrate the effectiveness of the proposed algorithm.

异质变化检测（HCD）具有挑战性，因为各种传感器的成像机制不同，导致图像难以直接比较。为解决这一问题，我们提出了一种基于图的回归算法，利用图总变异正则化和低阶矩阵分解（GTVLR）来进行 HCD 检测。利用图信号处理（GSP）理论，采用有向图（数字图）模型来有效表示图像的方向和相关信息，从而使图过滤后的同域异构数据能够直接比较。GTVLR 框架便于将事件发生后的图像分解为回归图像和变化图像。这种分解可确保回归图像反映出事件发生前图像的结构相似性，而变化图像则突出了变化区域，有助于变化检测。该模型通过 GTV 正则化和 Low-Rank 惩罚分别表征了数据的片状平滑性和 Low-Rank 特性。此外，还通过整合数字图中的高阶相邻信息来完善模型。在三个真实世界数据集上进行的实验以及与几种最先进方法的比较证明了所提算法的有效性。

{"title":"Graph total variation and low-rank regularization for heterogeneous change detection","authors":"Jichao Yao , Junzheng Jiang , Fang Zhou","doi":"10.1016/j.dsp.2024.104825","DOIUrl":"10.1016/j.dsp.2024.104825","url":null,"abstract":"<div><div>Heterogeneous change detection (HCD) is challenging because different imaging mechanisms for various sensors make images difficult to compare directly. To address this problem, a graph-based regression algorithm is proposed for HCD, by leveraging the Graph Total Variation regularization and Low-Rank matrices decomposition (GTVLR). Utilizing graph signal processing (GSP) theory, a directed graph (digraph) model is employed to effectively represent the orientation and correlation information of images, thereby enabling direct comparison of heterogeneous data within the same domain after graph filtering. The GTVLR framework facilitates the decomposition of post-event images into regression and changed images. This decomposition ensures that the regression image mirrors the structure similarity of the pre-event image, while the changed image highlights areas of alteration, aiding in change detection. The model characterizes the piecewise smoothness and Low-Rank properties of data through GTV regularization and Low-Rank penalty, respectively. Moreover, by integrating the higher-order neighboring information within the digraph to refine the model. Experiments conducted on three real-world datasets and comparison with several state-of-the-art methods demonstrate the effectiveness of the proposed algorithm.</div></div>","PeriodicalId":51011,"journal":{"name":"Digital Signal Processing","volume":"156 ","pages":"Article 104825"},"PeriodicalIF":2.9,"publicationDate":"2024-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142531119","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Debiased hybrid contrastive learning with hard negative mining for unsupervised person re-identification 针对无监督人员再识别的有偏差混合对比学习与硬负挖掘

IF 2.9 3区工程技术 Q2 ENGINEERING, ELECTRICAL & ELECTRONIC

Digital Signal Processing

Pub Date : 2024-10-21 DOI: 10.1016/j.dsp.2024.104826

Yu Zhao , Qiaoyuan Shu

The goal of unsupervised person re-identification is to retrieve a specific person across several non-overlapping cameras without the aid of manual labeling information. In recent times, contrastive learning has found extensive application in undertaking the complexities of unsupervised person Re-ID. Nevertheless, prevailing approaches often ignore the bias in negative proxy sampling and the significance of hard negatives in contrastive learning. These limitations have constrained the performance of existing methods. To solve these issues, we introduce a Debiased Hybrid Contrastive Learning with Hard Negative Mining (DHCL-HNM) approach. Particularly, the proposed approach employs an instance-level memory bank to save the class prototypes for all training images. In each training epoch, the memory bank undergoes clustering, dividing the dataset into un-clustered outliers and clustered images with pseudo labels. Then, the debiasing of negative proxies and the hard negative mining are integrated into a hybrid contrastive learning process to enhance the intra-class similarity and the instance discrimination. The debiasing operation is realized during the sampling of negative proxies to reduce the negative effects of false negatives. In the meantime, the hard negative mining can guide the Re-ID model to concentrate on the hard negatives by reweighting negative proxies based on their similarities to the anchor sample. The efficiency of the proposed method in the realm of unsupervised person Re-ID is demonstrated through comprehensive experiment outcomes conducted on several datasets.

无监督人员再识别的目标是在不借助人工标注信息的情况下，在多个非重叠摄像头中检索出一个特定的人。近来，对比学习已被广泛应用于处理复杂的无监督人员再识别问题。然而，目前流行的方法往往忽略了负代理采样的偏差以及对比学习中硬负值的重要性。这些局限性制约了现有方法的性能。为了解决这些问题，我们引入了一种带有硬否定挖掘的去偏差混合对比学习（DHCL-HNM）方法。特别是，所提出的方法采用了实例级存储库来保存所有训练图像的类原型。在每次训练中，记忆库都会进行聚类，将数据集分为未聚类的异常值和带有伪标签的聚类图像。然后，在混合对比学习过程中整合了负面代理去除法和硬负面挖掘法，以增强类内相似性和实例区分度。去重操作是在负代理采样过程中实现的，以减少假否定的负面影响。同时，硬否定挖掘可以引导 Re-ID 模型根据否定代理与锚样本的相似性进行重新加权，从而集中处理硬否定。通过在多个数据集上进行的综合实验结果，证明了所提出的方法在无监督人员再识别领域的效率。

{"title":"Debiased hybrid contrastive learning with hard negative mining for unsupervised person re-identification","authors":"Yu Zhao , Qiaoyuan Shu","doi":"10.1016/j.dsp.2024.104826","DOIUrl":"10.1016/j.dsp.2024.104826","url":null,"abstract":"<div><div>The goal of unsupervised person re-identification is to retrieve a specific person across several non-overlapping cameras without the aid of manual labeling information. In recent times, contrastive learning has found extensive application in undertaking the complexities of unsupervised person Re-ID. Nevertheless, prevailing approaches often ignore the bias in negative proxy sampling and the significance of hard negatives in contrastive learning. These limitations have constrained the performance of existing methods. To solve these issues, we introduce a Debiased Hybrid Contrastive Learning with Hard Negative Mining (DHCL-HNM) approach. Particularly, the proposed approach employs an instance-level memory bank to save the class prototypes for all training images. In each training epoch, the memory bank undergoes clustering, dividing the dataset into un-clustered outliers and clustered images with pseudo labels. Then, the debiasing of negative proxies and the hard negative mining are integrated into a hybrid contrastive learning process to enhance the intra-class similarity and the instance discrimination. The debiasing operation is realized during the sampling of negative proxies to reduce the negative effects of false negatives. In the meantime, the hard negative mining can guide the Re-ID model to concentrate on the hard negatives by reweighting negative proxies based on their similarities to the anchor sample. The efficiency of the proposed method in the realm of unsupervised person Re-ID is demonstrated through comprehensive experiment outcomes conducted on several datasets.</div></div>","PeriodicalId":51011,"journal":{"name":"Digital Signal Processing","volume":"156 ","pages":"Article 104826"},"PeriodicalIF":2.9,"publicationDate":"2024-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142531116","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Norm constraints pyramid for image dehazing 用于图像去毛刺的规范约束金字塔

IF 2.9 3区工程技术 Q2 ENGINEERING, ELECTRICAL & ELECTRONIC

Digital Signal Processing

Pub Date : 2024-10-21 DOI: 10.1016/j.dsp.2024.104828

Lingfan Wu , Haojin Hu , Guoqi Teng , Yifan Yang , Hong Zhang

Dark channel prior-based methods have achieved remarkable performance for image dehazing. However, previous studies are mostly focused on the accuracy of the assumptions used in the target scenes, which incurs color distortion and brightness reduction when the models are used for real-world hazy images. We propose a norm constraints pyramid framework to improve the generalization performance of dehazing. First, a local color adaptive correction approach is devised to ascertain whether there is any color bias and thereafter rectify it automatically. Furthermore, multiple norm constraint methods are developed to improve the transmission and accomplish the first image removal. Finally, a non-linear enhancement method is created via this restriction that precisely modifies the brightness of an image. Through extensive experiments, we demonstrate that our framework establishes the new state-of- the-art performance for real-world dehazing, in terms of visual quality assessed by no-reference quality metrics as well as subjective evaluation and downstream task performance indicator.

基于暗通道先验的方法在图像去毛刺方面取得了显著的性能。然而，以往的研究大多关注目标场景假设的准确性，当模型用于真实世界的雾霾图像时，会导致色彩失真和亮度降低。我们提出了一种规范约束金字塔框架，以提高去雾化的泛化性能。首先，我们设计了一种局部色彩自适应校正方法，以确定是否存在色彩偏差，然后自动纠正偏差。此外，还开发了多种规范约束方法，以改善传输并完成首次图像去除。最后，通过这种限制创建了一种非线性增强方法，可以精确地修改图像的亮度。通过大量的实验，我们证明了我们的框架为现实世界的去毛刺工作建立了新的先进性能，无论是无参考质量指标评估的视觉质量，还是主观评价和下游任务性能指标，都是如此。

引用次数: 0

Damage identification method for jacket platform based on dual-channel model 基于双通道模型的夹套平台损伤识别方法

IF 2.9 3区工程技术 Q2 ENGINEERING, ELECTRICAL & ELECTRONIC

Digital Signal Processing

Pub Date : 2024-10-21 DOI: 10.1016/j.dsp.2024.104827

Wenkai Wu , Junwei Gao , Ankai Wei , Sheng Guan

To address the challenges posed by noisy vibration signals and underutilized time-series data in jacket platforms, this paper proposes a dual-channel damage detection method that integrates Temporal Convolutional Networks (TCN) and Gated Recurrent Units (GRU) in parallel. A multi-head attention mechanism (MA) is employed to reassign feature weights, improving detection accuracy. The optimized features are fused using the Concatenate function for the final output. Two experimental scenarios—isolated noise and ocean noise—were designed to evaluate the method. The results demonstrate that the combination of TCN, GRU and MA effectively detects damage in offshore platforms, surpassing other deep learning models. Although the method shows strong potential for real-world applications, further testing in more complex ocean environments is required to address potential limitations in handling highly variable noise patterns.

为解决夹套平台振动信号噪声大、时间序列数据利用率低等难题，本文提出了一种并行集成时序卷积网络（TCN）和门控递归单元（GRU）的双通道损伤检测方法。采用多头注意机制（MA）重新分配特征权重，提高了检测精度。使用 Concatenate 函数将优化后的特征融合为最终输出。设计了两个实验场景--隔离噪声和海洋噪声--来评估该方法。结果表明，TCN、GRU 和 MA 的组合能有效检测海上平台的损坏情况，超过了其他深度学习模型。虽然该方法在实际应用中显示出强大的潜力，但还需要在更复杂的海洋环境中进行进一步测试，以解决在处理高度多变的噪声模式时可能存在的局限性。

引用次数: 0

Multi exposure fusion for high dynamic range imaging via multi-channel gradient tensor 通过多通道梯度张量实现高动态范围成像的多重曝光融合

IF 2.9 3区工程技术 Q2 ENGINEERING, ELECTRICAL & ELECTRONIC

Digital Signal Processing

Pub Date : 2024-10-21 DOI: 10.1016/j.dsp.2024.104821

Jinyu Li , Yihong Wang , Feng Chen , Yu Wang , Qian Chen , Xiubao Sui

Multi-exposure fusion (MEF) is an effective technique for directly fusing a sequence of low dynamic range (LDR) images from a high dynamic range (HDR) natural scene. The goal is to generate an information enriched LDR image. Despite its effectiveness, current MEF methods often encounter issues such as detail loss and color degradation. Additionally, existing algorithms often struggle to balance image quality and computation time, particularly for large-sized images. This paper introduces an innovative MEF algorithm that address these challenges, offering improved performance and computational time across all image sizes. The algorithm employs a multi-channel gradient tensor on RGB images to effectively capture the contrast information among the three channels. This mechanism allows an edge-preserving image filter to maintain edges while smoothing weight maps. To enhance computational efficiency, the algorithm uses a fast approximation method suitable for large sized images. Our comprehensive experimental results demonstrate that the proposed method outperforms existing MEF techniques both quantitatively and qualitatively. Furthermore, our method reduces computational time by approximately 30% compared to the most recent state-of-the-art techniques.

多重曝光融合（MEF）是一种有效的技术，可直接融合来自高动态范围（HDR）自然场景的低动态范围（LDR）图像序列。其目标是生成信息丰富的 LDR 图像。尽管效果显著，但目前的 MEF 方法经常会遇到细节丢失和色彩退化等问题。此外，现有算法往往难以在图像质量和计算时间之间取得平衡，尤其是在处理大尺寸图像时。本文介绍了一种创新的 MEF 算法，可解决这些难题，在所有尺寸的图像上都能提供更高的性能和更短的计算时间。该算法在 RGB 图像上采用多通道梯度张量，以有效捕捉三个通道之间的对比度信息。这种机制允许边缘保留图像滤波器在平滑权重图的同时保留边缘。为了提高计算效率，该算法采用了适合大尺寸图像的快速近似方法。我们的综合实验结果表明，所提出的方法在数量和质量上都优于现有的 MEF 技术。此外，与最新的先进技术相比，我们的方法减少了约 30% 的计算时间。

{"title":"Multi exposure fusion for high dynamic range imaging via multi-channel gradient tensor","authors":"Jinyu Li , Yihong Wang , Feng Chen , Yu Wang , Qian Chen , Xiubao Sui","doi":"10.1016/j.dsp.2024.104821","DOIUrl":"10.1016/j.dsp.2024.104821","url":null,"abstract":"<div><div>Multi-exposure fusion (MEF) is an effective technique for directly fusing a sequence of low dynamic range (LDR) images from a high dynamic range (HDR) natural scene. The goal is to generate an information enriched LDR image. Despite its effectiveness, current MEF methods often encounter issues such as detail loss and color degradation. Additionally, existing algorithms often struggle to balance image quality and computation time, particularly for large-sized images. This paper introduces an innovative MEF algorithm that address these challenges, offering improved performance and computational time across all image sizes. The algorithm employs a multi-channel gradient tensor on RGB images to effectively capture the contrast information among the three channels. This mechanism allows an edge-preserving image filter to maintain edges while smoothing weight maps. To enhance computational efficiency, the algorithm uses a fast approximation method suitable for large sized images. Our comprehensive experimental results demonstrate that the proposed method outperforms existing MEF techniques both quantitatively and qualitatively. Furthermore, our method reduces computational time by approximately 30% compared to the most recent state-of-the-art techniques.</div></div>","PeriodicalId":51011,"journal":{"name":"Digital Signal Processing","volume":"156 ","pages":"Article 104821"},"PeriodicalIF":2.9,"publicationDate":"2024-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142531117","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

MMCL: Meta-mutual contrastive learning for multi-modal medical image fusion MMCL：用于多模态医学图像融合的元相互对比学习

IF 2.9 3区工程技术 Q2 ENGINEERING, ELECTRICAL & ELECTRONIC

Digital Signal Processing

Pub Date : 2024-10-18 DOI: 10.1016/j.dsp.2024.104806

Ying Zhang , Chaozhen Ma , Hongwei Ding , Yuanjing Zhu

The number of datasets and computational efficiency are always hindrances in the multi-modal medical image fusion (MMIF) research. To address these challenges, we propose a contrastive learning framework inspired meta-mutual, which divides the medical image fusion task into subtasks and pre-trains an optimal meta-representation suitable for all subtasks. We then fine-tune our proposed network using this optimal meta-representation as initialization, achieving the best model with only a few short datasets. Additionally, extracting source image features in pairs can lead to redundant information due to the invariant and unique features of multi-modal images. Therefore, we introduce novelty mutual contrastive coupled pairs to extract both invariant and unique features from source images. Experimental results demonstrate that our method outperforms other state-of-the-art fusion methods.

在多模态医学影像融合（MMIF）研究中，数据集的数量和计算效率始终是个障碍。为了应对这些挑战，我们提出了一个受元互助启发的对比学习框架，它将医学图像融合任务划分为多个子任务，并预先训练出适合所有子任务的最优元表征。然后，我们利用这个最佳元表征作为初始化，对我们提出的网络进行微调，仅用几个简短的数据集就建立了最佳模型。此外，由于多模态图像的不变性和独特性，成对提取源图像特征可能会导致冗余信息。因此，我们引入了新颖性相互对比耦合对，从源图像中提取不变和独特的特征。实验结果表明，我们的方法优于其他最先进的融合方法。

引用次数: 0

Combine multi-order representation learning and frame optimization learning for skeleton-based action recognition 结合多阶表示学习和帧优化学习，实现基于骨骼的动作识别

IF 2.9 3区工程技术 Q2 ENGINEERING, ELECTRICAL & ELECTRONIC

Digital Signal Processing

Pub Date : 2024-10-18 DOI: 10.1016/j.dsp.2024.104823

Liping Nong , Zhuocheng Huang , Junyi Wang , Yanpeng Rong , Jie Peng , Yiping Huang

Skeleton-based action recognition has broad application prospects in many fields such as virtual reality. Currently, the most popular way is to employ Graph Convolutional Networks (GCNs) or Hypergraph Convolutional Networks (HGCNs) for this task. However, GCN-based methods may heavily rely on the physical connectivity relationship between joints while lack the capture of higher-order information about interactions among distant joints, and HGCN-based methods usually introduce unnecessary noise when capturing low-order information of skeleton structures with simple topology. Besides, the current methods do not deal well with redundant frames and confusing frames. These limitations hinder the improvement of recognition accuracy. In this paper, we propose a novel network, called Hyper-Net, which combines multi-order representation learning and frame optimization learning for skeleton-based action recognition. Specifically, the proposed Hyper-Net contains Temporal-Channel Aggregation Graph Convolution (TCA-GC), Spatial-Temporal Aggregation Hypergraph Convolution (STA-HC) and Frame Optimization Learning (F-OL) modules. The TCA-GC aggregates low-order and local information from simple joint and bone topologies across different temporal and channel dimensions. The STA-HC captures high-order and global information from complex motion streams as well as solving the problem of spatial-temporal weight imbalance. The F-OL can adaptively extract key frames and distinguish confusing frames, thus improving the ability of the network to recognize confusing actions. A large number of experiments are conducted on the NTU RGB+D, NTU RGB+D 120 and NW-UCLA datasets for action recognition task. Experimental results demonstrate the superiority and effectiveness of the proposed network.

基于骨骼的动作识别在虚拟现实等许多领域都有广阔的应用前景。目前，最流行的方法是采用图卷积网络（GCN）或超图卷积网络（HGCN）来完成这项任务。然而，基于 GCN 的方法可能会严重依赖关节间的物理连接关系，而缺乏对远处关节间相互作用的高阶信息的捕捉；而基于 HGCN 的方法在捕捉拓扑结构简单的骨架结构的低阶信息时，通常会引入不必要的噪声。此外，目前的方法不能很好地处理冗余帧和混淆帧。这些局限性阻碍了识别准确率的提高。在本文中，我们提出了一种名为 Hyper-Net 的新型网络，它将多阶表示学习和帧优化学习相结合，用于基于骨架的动作识别。具体来说，所提出的 Hyper-Net 包含时空通道聚合图卷积（TCA-GC）、时空聚合超图卷积（STA-HC）和帧优化学习（F-OL）模块。TCA-GC 可在不同的时间和通道维度上聚合来自简单关节和骨骼拓扑的低阶和局部信息。STA-HC 可从复杂的运动流中捕捉高阶和全局信息，并解决时空权重不平衡的问题。F-OL 可以自适应地提取关键帧并区分混淆帧，从而提高网络识别混淆动作的能力。在北师大 RGB+D、北师大 RGB+D 120 和 NW-UCLA 数据集上进行了大量的动作识别任务实验。实验结果证明了所提出网络的优越性和有效性。

{"title":"Combine multi-order representation learning and frame optimization learning for skeleton-based action recognition","authors":"Liping Nong , Zhuocheng Huang , Junyi Wang , Yanpeng Rong , Jie Peng , Yiping Huang","doi":"10.1016/j.dsp.2024.104823","DOIUrl":"10.1016/j.dsp.2024.104823","url":null,"abstract":"<div><div>Skeleton-based action recognition has broad application prospects in many fields such as virtual reality. Currently, the most popular way is to employ Graph Convolutional Networks (GCNs) or Hypergraph Convolutional Networks (HGCNs) for this task. However, GCN-based methods may heavily rely on the physical connectivity relationship between joints while lack the capture of higher-order information about interactions among distant joints, and HGCN-based methods usually introduce unnecessary noise when capturing low-order information of skeleton structures with simple topology. Besides, the current methods do not deal well with redundant frames and confusing frames. These limitations hinder the improvement of recognition accuracy. In this paper, we propose a novel network, called Hyper-Net, which combines multi-order representation learning and frame optimization learning for skeleton-based action recognition. Specifically, the proposed Hyper-Net contains Temporal-Channel Aggregation Graph Convolution (TCA-GC), Spatial-Temporal Aggregation Hypergraph Convolution (STA-HC) and Frame Optimization Learning (F-OL) modules. The TCA-GC aggregates low-order and local information from simple joint and bone topologies across different temporal and channel dimensions. The STA-HC captures high-order and global information from complex motion streams as well as solving the problem of spatial-temporal weight imbalance. The F-OL can adaptively extract key frames and distinguish confusing frames, thus improving the ability of the network to recognize confusing actions. A large number of experiments are conducted on the NTU RGB+D, NTU RGB+D 120 and NW-UCLA datasets for action recognition task. Experimental results demonstrate the superiority and effectiveness of the proposed network.</div></div>","PeriodicalId":51011,"journal":{"name":"Digital Signal Processing","volume":"156 ","pages":"Article 104823"},"PeriodicalIF":2.9,"publicationDate":"2024-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142531120","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

FSIC: Frequency-separated image compression for small object detection FSIC：用于小目标检测的分频图像压缩技术

IF 2.9 3区工程技术 Q2 ENGINEERING, ELECTRICAL & ELECTRONIC

Digital Signal Processing

Pub Date : 2024-10-18 DOI: 10.1016/j.dsp.2024.104822

Chengjie Dai , Tiantian Song , Qiang Chen , Hanshen Gong , Bowei Yang , Guanghua Song

The existing image compression methods are designed for the human visual system. They can achieve good compression quality for low-frequency components of the image that are important to human vision. However, for object detection models, both high and low-frequency components are essential. As a result, the detection metrics on the compressed images obtained by current methods will decline. Particularly for small object detection, the lack of high-frequency signals makes it difficult to distinguish the targets from the background. In this paper, we propose a frequency-separated image compression model, named FSIC. During the training process, the compression of low-frequency components only employs MSE loss, while the compression of high-frequency components additionally incorporates a detection loss. We validate FSIC's image compression capability for the small object detection task on the VisDrone dataset and Dota dataset. Results show that under extremely high compression rates, FSIC demonstrates a better performance compared with current compression methods. Furthermore, FSIC has the fastest encoding speed among current learning-based compression models.

现有的图像压缩方法是针对人类视觉系统设计的。它们可以对图像中对人类视觉非常重要的低频成分实现良好的压缩质量。然而，对于物体检测模型来说，高频和低频成分都是必不可少的。因此，当前方法获得的压缩图像的检测指标会下降。特别是在小物体检测中，由于缺乏高频信号，很难将目标与背景区分开来。本文提出了一种频率分离图像压缩模型，命名为 FSIC。在训练过程中，低频分量的压缩只采用 MSE 损失，而高频分量的压缩则额外加入了检测损失。我们在 VisDrone 数据集和 Dota 数据集上验证了 FSIC 在小物体检测任务中的图像压缩能力。结果表明，在极高的压缩率下，FSIC 的性能优于当前的压缩方法。此外，在目前基于学习的压缩模型中，FSIC 的编码速度最快。

{"title":"FSIC: Frequency-separated image compression for small object detection","authors":"Chengjie Dai , Tiantian Song , Qiang Chen , Hanshen Gong , Bowei Yang , Guanghua Song","doi":"10.1016/j.dsp.2024.104822","DOIUrl":"10.1016/j.dsp.2024.104822","url":null,"abstract":"<div><div>The existing image compression methods are designed for the human visual system. They can achieve good compression quality for low-frequency components of the image that are important to human vision. However, for object detection models, both high and low-frequency components are essential. As a result, the detection metrics on the compressed images obtained by current methods will decline. Particularly for small object detection, the lack of high-frequency signals makes it difficult to distinguish the targets from the background. In this paper, we propose a frequency-separated image compression model, named FSIC. During the training process, the compression of low-frequency components only employs MSE loss, while the compression of high-frequency components additionally incorporates a detection loss. We validate FSIC's image compression capability for the small object detection task on the VisDrone dataset and Dota dataset. Results show that under extremely high compression rates, FSIC demonstrates a better performance compared with current compression methods. Furthermore, FSIC has the fastest encoding speed among current learning-based compression models.</div></div>","PeriodicalId":51011,"journal":{"name":"Digital Signal Processing","volume":"156 ","pages":"Article 104822"},"PeriodicalIF":2.9,"publicationDate":"2024-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142531115","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

首页上一页

下一页尾页

类型

全部化学•材料生命科学医学物理工程技术环境•农林材料科学地球科学法学管理学化学环境科学与生态学计算机科学教育学经济学农林科学人文科学生物学数学物理与天体物理心理学综合性期刊其他工业工程理学历史学农学文学信息工程

数据库

全部 ACS Publications Elsevier ieeexplore Springer The Royal Society of Chemistry Wiley

期刊

Digital Signal Processing

全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.

﹀