Pub Date : 2024-12-18DOI: 10.1109/LSP.2024.3519884
Ziying Li;Xiongjun Fu;Jian Dong;Min Xie
As a key technology in radar reconnaissance systems, radar signal sorting aims to separate multiple radar pulses from an interleaved pulse stream. Supervised signal sorting methods based on deep learning depend on a large volume of training data to optimize model parameters. However, acquiring labeled pulses in practice is challenging. In this letter, a semi-supervised learning (SSL) framework is proposed to address this issue. First, a Self-Organizing Map (SOM) is used to learn the spatial distribution of impulse features, and an anchor graph is constructed based on SOM nodes. A pseudo-label set is then generated using the SOM based on pulse discrepancy information. Finally, a three-layer Weighted Residual Graph Convolutional Network (WRGCN) is designed for signal sorting, with its parameters pre-trained on pseudo-labels and fine-tuned with a limited number of true labels. Experiments on a simulated radar pulse dataset demonstrate that this framework outperforms several existing methods for radar signal sorting with limited labeled pulses.
{"title":"Radar Signal Sorting via Graph Convolutional Network and Semi-Supervised Learning","authors":"Ziying Li;Xiongjun Fu;Jian Dong;Min Xie","doi":"10.1109/LSP.2024.3519884","DOIUrl":"https://doi.org/10.1109/LSP.2024.3519884","url":null,"abstract":"As a key technology in radar reconnaissance systems, radar signal sorting aims to separate multiple radar pulses from an interleaved pulse stream. Supervised signal sorting methods based on deep learning depend on a large volume of training data to optimize model parameters. However, acquiring labeled pulses in practice is challenging. In this letter, a semi-supervised learning (SSL) framework is proposed to address this issue. First, a Self-Organizing Map (SOM) is used to learn the spatial distribution of impulse features, and an anchor graph is constructed based on SOM nodes. A pseudo-label set is then generated using the SOM based on pulse discrepancy information. Finally, a three-layer Weighted Residual Graph Convolutional Network (WRGCN) is designed for signal sorting, with its parameters pre-trained on pseudo-labels and fine-tuned with a limited number of true labels. Experiments on a simulated radar pulse dataset demonstrate that this framework outperforms several existing methods for radar signal sorting with limited labeled pulses.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"421-425"},"PeriodicalIF":3.2,"publicationDate":"2024-12-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142925371","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-18DOI: 10.1109/LSP.2024.3520010
Yuxuan Xiong;Zhou Zhao;Yongchao Xu;Yan Zhang;Bo Du
Cervical optical coherence tomography (OCT) imaging serves as an effective diagnostic tool, and the development of deep learning classification models for OCT has the potential to enhance diagnosis. However, the complex imaging patterns of OCT data, significant noise, and the substantial domain gap from multi-center data result in high uncertainty and low accuracy in classification networks. To address these challenges, we propose a Multi-scale Prototype-Guided Diffusion learning method (MPGD), which is constructed with the Multi-scale Feature Condition (MFC) , Diffusion-based Classification Calibrator (DCC) , and Multi-scale Prototype Bank (MPB) modules. Specifically, MFC provides initial classification based on multi-scale features, DCC calibrates MFC's classification results through a diffusion model, and MPB refines DCC's visual guidance using prototypes obtained from clustering. Extensive experiments demonstrate that MPGD outperforms widely-used competitors for cervical OCT image classification, showing excellent generalization performance.
{"title":"Calibration Matters: Prototype-Aware Diffusion for OCT Cervical Classification With Calibration","authors":"Yuxuan Xiong;Zhou Zhao;Yongchao Xu;Yan Zhang;Bo Du","doi":"10.1109/LSP.2024.3520010","DOIUrl":"https://doi.org/10.1109/LSP.2024.3520010","url":null,"abstract":"Cervical optical coherence tomography (OCT) imaging serves as an effective diagnostic tool, and the development of deep learning classification models for OCT has the potential to enhance diagnosis. However, the complex imaging patterns of OCT data, significant noise, and the substantial domain gap from multi-center data result in high uncertainty and low accuracy in classification networks. To address these challenges, we propose a Multi-scale Prototype-Guided Diffusion learning method (MPGD), which is constructed with the \u0000<bold>Multi-scale Feature Condition (MFC)</b>\u0000, \u0000<bold>Diffusion-based Classification Calibrator (DCC)</b>\u0000, and \u0000<bold>Multi-scale Prototype Bank (MPB)</b>\u0000 modules. Specifically, MFC provides initial classification based on multi-scale features, DCC calibrates MFC's classification results through a diffusion model, and MPB refines DCC's visual guidance using prototypes obtained from clustering. Extensive experiments demonstrate that MPGD outperforms widely-used competitors for cervical OCT image classification, showing excellent generalization performance.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"396-400"},"PeriodicalIF":3.2,"publicationDate":"2024-12-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142925369","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-17DOI: 10.1109/LSP.2024.3519459
Zheng Zhai;Mingxin Wu;Jialu Xu;Xiaohui Li
This letter introduces a sparse regularized projection matrix approximation (SPMA) model to recover cluster structures from affinity matrices. The model is formulated as a projection approximation problem with an entry-wise sparsity penalty to encourage sparse solutions. We propose two algorithms to solve this problem: one involves direct optimization on the Stiefel manifold using the Cayley transformation, while the other employs the Alternating Direction Method of Multipliers (ADMM). Numerical experiments on synthetic and real-world datasets demonstrate that our regularized projection matrix approximation approach significantly outperforms state-of-the-art methods in clustering accuracy and performance.
{"title":"Sparse Projection Matrix Approximation and Its Applications","authors":"Zheng Zhai;Mingxin Wu;Jialu Xu;Xiaohui Li","doi":"10.1109/LSP.2024.3519459","DOIUrl":"https://doi.org/10.1109/LSP.2024.3519459","url":null,"abstract":"This letter introduces a sparse regularized projection matrix approximation (SPMA) model to recover cluster structures from affinity matrices. The model is formulated as a projection approximation problem with an entry-wise sparsity penalty to encourage sparse solutions. We propose two algorithms to solve this problem: one involves direct optimization on the Stiefel manifold using the Cayley transformation, while the other employs the Alternating Direction Method of Multipliers (ADMM). Numerical experiments on synthetic and real-world datasets demonstrate that our regularized projection matrix approximation approach significantly outperforms state-of-the-art methods in clustering accuracy and performance.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"351-355"},"PeriodicalIF":3.2,"publicationDate":"2024-12-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142890168","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-17DOI: 10.1109/LSP.2024.3519261
Andrzej Borys
This letter shows that the description of signal sampling operation that uses a weighted Dirac comb plays only a supporting role. It must be supplemented with a relation that transfers this description into the world of ordinary functions. In this case, these will be the weighted step functions. It is shown that the description mentioned, together with the complementary relation, form a joint model of the signal sampling operation. To achieve this, an idea of a Schrödinger's cat locked in a black box, which opens at sampling instants, was used.
{"title":"How to Understand Generation of Dirac Weighted Combs in Signal Sampling Operation?","authors":"Andrzej Borys","doi":"10.1109/LSP.2024.3519261","DOIUrl":"https://doi.org/10.1109/LSP.2024.3519261","url":null,"abstract":"This letter shows that the description of signal sampling operation that uses a weighted Dirac comb plays only a supporting role. It must be supplemented with a relation that transfers this description into the world of ordinary functions. In this case, these will be the weighted step functions. It is shown that the description mentioned, together with the complementary relation, form a joint model of the signal sampling operation. To achieve this, an idea of a Schrödinger's cat locked in a black box, which opens at sampling instants, was used.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"311-315"},"PeriodicalIF":3.2,"publicationDate":"2024-12-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142890397","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-17DOI: 10.1109/LSP.2024.3509335
Hai Chen;Shu Zhao;Yuanting Yan;Fulan Qian
One of the most popular methods for analyzing the robustness of 3D Deep Neural Networks (DNNs) is the transfer-based adversarial attack method, as it allows to analyze the robustness of an unknown model by generating an adversarial point cloud on an alternative model. However, the adversarial point clouds generated by current methods may overfit the surrogate models that generated them, thus limiting their performance in transfer attacks against different target 3D classifiers. To enhance the transferability of the adversarial point cloud, we propose in this letter an adversarial attack method by Initializing the Transferable Adversarial Noise, which named as ITAN . Specifically, we pre-train on the training set a generator capable of generating the adversarial noise with transferability and diversity, and then the noise generated by the generator serves as the initial adversarial noise to be integrated into the iterations of the attack. Extensive experiments on well-recognized benchmark datasets demonstrate that the adversarial point clouds generated by the proposed ITAN could be effectively transferred across unknown 3D classifiers.
分析三维深度神经网络（DNN）鲁棒性的最流行方法之一是基于转移的对抗攻击法，因为它可以通过在替代模型上生成对抗点云来分析未知模型的鲁棒性。然而，当前方法生成的对抗点云可能会过度拟合生成它们的代用模型，从而限制了它们在针对不同目标三维分类器的转移攻击中的性能。为了提高对抗点云的可转移性，我们在这封信中提出了一种通过初始化可转移对抗噪声的对抗攻击方法，并将其命名为 ITAN。具体来说，我们在训练集上预先训练一个能够生成具有可转移性和多样性的对抗噪声的生成器，然后将生成器生成的噪声作为初始对抗噪声，整合到攻击的迭代中。在公认的基准数据集上进行的大量实验证明，由所提出的 ITAN 生成的对抗性点云可以在未知的 3D 分类器之间有效转移。
{"title":"Enhancing the Transferability of Adversarial Point Clouds by Initializing Transferable Adversarial Noise","authors":"Hai Chen;Shu Zhao;Yuanting Yan;Fulan Qian","doi":"10.1109/LSP.2024.3509335","DOIUrl":"https://doi.org/10.1109/LSP.2024.3509335","url":null,"abstract":"One of the most popular methods for analyzing the robustness of 3D Deep Neural Networks (DNNs) is the transfer-based adversarial attack method, as it allows to analyze the robustness of an unknown model by generating an adversarial point cloud on an alternative model. However, the adversarial point clouds generated by current methods may overfit the surrogate models that generated them, thus limiting their performance in transfer attacks against different target 3D classifiers. To enhance the transferability of the adversarial point cloud, we propose in this letter an adversarial attack method by Initializing the Transferable Adversarial Noise, which named as \u0000<bold>ITAN</b>\u0000. Specifically, we pre-train on the training set a generator capable of generating the adversarial noise with transferability and diversity, and then the noise generated by the generator serves as the initial adversarial noise to be integrated into the iterations of the attack. Extensive experiments on well-recognized benchmark datasets demonstrate that the adversarial point clouds generated by the proposed ITAN could be effectively transferred across unknown 3D classifiers.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"201-205"},"PeriodicalIF":3.2,"publicationDate":"2024-12-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142844601","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-17DOI: 10.1109/LSP.2024.3519265
Minhyeok Ko;Abdollah Shafieezadeh
State estimation from noisy observations is crucial across various fields. Traditional methods such as Kalman, Extended Kalman, and Unscented Kalman Filter often struggle with nonlinearities, model inaccuracies, and high observation noise. This letter introduces Cholesky-KalmanNet (CKN), a model-based deep learning approach that considerably enhances state estimation by providing and enforcing transiently precise error covariance estimation. Specifically, the CKN embeds mathematical constraints associated with the positive definiteness of error covariance in a recurrent DNN architecture through the Cholesky decomposition. This architecture enhances statistical reliability and mitigates numerical instabilities. Furthermore, introducing a novel loss function that minimizes discrepancies between the estimated and empirical error covariance ensures a comprehensive minimization of estimation errors, accounting for interdependencies among state variables. Extensive evaluations on both synthetic and real-world datasets affirm CKN's superior performance vis-a-vis state estimation accuracy, robustness against system inaccuracies and observation noise, as well as stability across varying training data partitions, an essential feature for practical scenarios with suboptimal data conditions.
{"title":"Cholesky-KalmanNet: Model-Based Deep Learning With Positive Definite Error Covariance Structure","authors":"Minhyeok Ko;Abdollah Shafieezadeh","doi":"10.1109/LSP.2024.3519265","DOIUrl":"https://doi.org/10.1109/LSP.2024.3519265","url":null,"abstract":"State estimation from noisy observations is crucial across various fields. Traditional methods such as Kalman, Extended Kalman, and Unscented Kalman Filter often struggle with nonlinearities, model inaccuracies, and high observation noise. This letter introduces Cholesky-KalmanNet (CKN), a model-based deep learning approach that considerably enhances state estimation by providing and enforcing transiently precise error covariance estimation. Specifically, the CKN embeds mathematical constraints associated with the positive definiteness of error covariance in a recurrent DNN architecture through the Cholesky decomposition. This architecture enhances statistical reliability and mitigates numerical instabilities. Furthermore, introducing a novel loss function that minimizes discrepancies between the estimated and empirical error covariance ensures a comprehensive minimization of estimation errors, accounting for interdependencies among state variables. Extensive evaluations on both synthetic and real-world datasets affirm CKN's superior performance vis-a-vis state estimation accuracy, robustness against system inaccuracies and observation noise, as well as stability across varying training data partitions, an essential feature for practical scenarios with suboptimal data conditions.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"326-330"},"PeriodicalIF":3.2,"publicationDate":"2024-12-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142890169","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-17DOI: 10.1109/LSP.2024.3519258
Fatemeh Yaghoobi;Simo Särkkä
This paper presents parallel-in-time state estimation methods for systems with Slow-Rate inTegrated Measurements (SRTM). Integrated measurements are common in various applications, and they appear in analysis of data resulting from processes that require material collection or integration over the sampling period. Current state estimation methods for SRTM are inherently sequential, preventing temporal parallelization in their standard form. This paper proposes parallel Bayesian filters and smoothers for linear Gaussian SRTM models. For that purpose, we develop a novel smoother for SRTM models and develop parallel-in-time filters and smoother for them using an associative scan-based parallel formulation. Empirical experiments ran on a GPU demonstrate the superior time complexity of the proposed methods over traditional sequential approaches.
{"title":"Parallel State Estimation for Systems With Integrated Measurements","authors":"Fatemeh Yaghoobi;Simo Särkkä","doi":"10.1109/LSP.2024.3519258","DOIUrl":"https://doi.org/10.1109/LSP.2024.3519258","url":null,"abstract":"This paper presents parallel-in-time state estimation methods for systems with Slow-Rate inTegrated Measurements (SRTM). Integrated measurements are common in various applications, and they appear in analysis of data resulting from processes that require material collection or integration over the sampling period. Current state estimation methods for SRTM are inherently sequential, preventing temporal parallelization in their standard form. This paper proposes parallel Bayesian filters and smoothers for linear Gaussian SRTM models. For that purpose, we develop a novel smoother for SRTM models and develop parallel-in-time filters and smoother for them using an associative scan-based parallel formulation. Empirical experiments ran on a GPU demonstrate the superior time complexity of the proposed methods over traditional sequential approaches.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"371-375"},"PeriodicalIF":3.2,"publicationDate":"2024-12-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10804629","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142912512","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-13DOI: 10.1109/LSP.2024.3517418
Rong Zhang;Junneng Feng;Cun Feng;Yirui Wang;Lijun Guo
Most of existing Human Pose Estimation (HPE) methods struggle to handle with challenges such as changeable poses, complex backgrounds, and occlusion encountered in complex scenes. To address these problems, a novel HPE network, called Part2Pose, is proposed in this paper. In our Part2Pose, instead of focusing on small-sized keypoints like existing HPE methods do, we first extract image features based on human body parts to expand the detection scope. This strategy enhances the robustness of the extracted features to variations and distractions in complex scenes. Then, a Transformer-based Global Part Relation Module (GPRM) and a graph convolutional network-based Local Part Relation Module (LPRM) are used to capture global and local relationships among different body parts to help infer the position of keypoints. Extensive experiments on challenging datasets, including COCO, CrowdPose and OCHuman, show that the proposed Part2Pose can surpass existing popular state-of-the-art HPE methods. The combination with lightweight networks confirms the robustness and generalizability of our Part2Pose.
{"title":"Part2Pose: Inferring Human Pose From Parts in Complex Scenes","authors":"Rong Zhang;Junneng Feng;Cun Feng;Yirui Wang;Lijun Guo","doi":"10.1109/LSP.2024.3517418","DOIUrl":"https://doi.org/10.1109/LSP.2024.3517418","url":null,"abstract":"Most of existing Human Pose Estimation (HPE) methods struggle to handle with challenges such as changeable poses, complex backgrounds, and occlusion encountered in complex scenes. To address these problems, a novel HPE network, called Part2Pose, is proposed in this paper. In our Part2Pose, instead of focusing on small-sized keypoints like existing HPE methods do, we first extract image features based on human body parts to expand the detection scope. This strategy enhances the robustness of the extracted features to variations and distractions in complex scenes. Then, a Transformer-based Global Part Relation Module (GPRM) and a graph convolutional network-based Local Part Relation Module (LPRM) are used to capture global and local relationships among different body parts to help infer the position of keypoints. Extensive experiments on challenging datasets, including COCO, CrowdPose and OCHuman, show that the proposed Part2Pose can surpass existing popular state-of-the-art HPE methods. The combination with lightweight networks confirms the robustness and generalizability of our Part2Pose.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"441-445"},"PeriodicalIF":3.2,"publicationDate":"2024-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142938332","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Recently, diffusion models have achieved remarkable success in blind image super-resolution. However, most existing methods rely solely on uni-modal degraded low-resolution images to guide diffusion models for restoring high-fidelity images, resulting in inferior realism. In this letter, we propose a Multi-modal Prior-Guided diffusion model for blind image Super-Resolution (MPGSR), which fine-tunes Stable Diffusion (SD) by utilizing the superior visual-and-textual guidance for restoring realistic high-resolution images. Specifically, our MPGSR involves two stages, i.e., multi-modal guidance extraction and adaptive guidance injection. For the former, we propose a composited transformer and further incorporate it with GPT-CLIP to extract the representative visual-and-textual guidance. For the latter, we design a feature calibration ControlNet to inject the visual guidance and employ the cross-attention layer provided by the frozen SD to inject the textual guidance, thus effectively activating the powerful text-to-image generation potential. Extensive experiments show that our MPGSR outperforms state-of-the-art methods in restoration quality and convergence time.
{"title":"Multi-Modal Prior-Guided Diffusion Model for Blind Image Super-Resolution","authors":"Detian Huang;Jiaxun Song;Xiaoqian Huang;Zhenzhen Hu;Huanqiang Zeng","doi":"10.1109/LSP.2024.3516699","DOIUrl":"https://doi.org/10.1109/LSP.2024.3516699","url":null,"abstract":"Recently, diffusion models have achieved remarkable success in blind image super-resolution. However, most existing methods rely solely on uni-modal degraded low-resolution images to guide diffusion models for restoring high-fidelity images, resulting in inferior realism. In this letter, we propose a Multi-modal Prior-Guided diffusion model for blind image Super-Resolution (MPGSR), which fine-tunes Stable Diffusion (SD) by utilizing the superior visual-and-textual guidance for restoring realistic high-resolution images. Specifically, our MPGSR involves two stages, i.e., multi-modal guidance extraction and adaptive guidance injection. For the former, we propose a composited transformer and further incorporate it with GPT-CLIP to extract the representative visual-and-textual guidance. For the latter, we design a feature calibration ControlNet to inject the visual guidance and employ the cross-attention layer provided by the frozen SD to inject the textual guidance, thus effectively activating the powerful text-to-image generation potential. Extensive experiments show that our MPGSR outperforms state-of-the-art methods in restoration quality and convergence time.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"316-320"},"PeriodicalIF":3.2,"publicationDate":"2024-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142890396","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-12DOI: 10.1109/LSP.2024.3516700
Xuyi Zhuang;Yukun Qian;Mingjiang Wang
Recently, the performance of non-autoregressive ASR models has made significant progress but still lags behind hybrid CTC/attention systems. This paper introduces Hypformer, a fast hypothesis-driven rescoring speech recognition framework. Multiple hypothetical prefixes are realized by fast prefix generation algorithm. With two different rescoring methods, nar-ar rescoring and nar $^{2}$ rescoring, Hypformer can flexibly switch between autoregressive and non-autoregressive decoding modes to perform rescoring of hypothesis prefixes. Experiments on the standard Mandarin datasets AISHELL-1 and AISHELL-2 demonstrate that Hypformer outperforms the state-of-the-art Hybrid CTC/Attention systems in ASR performance while achieving a speedup of over six times. Experiments on the Mandarin sub-dialect dataset KeSpeech indicate that Hypformer achieves more accurate recognition by leveraging richer contextual information.
{"title":"Hypformer: A Fast Hypothesis-Driven Rescoring Speech Recognition Framework","authors":"Xuyi Zhuang;Yukun Qian;Mingjiang Wang","doi":"10.1109/LSP.2024.3516700","DOIUrl":"https://doi.org/10.1109/LSP.2024.3516700","url":null,"abstract":"Recently, the performance of non-autoregressive ASR models has made significant progress but still lags behind hybrid CTC/attention systems. This paper introduces Hypformer, a fast hypothesis-driven rescoring speech recognition framework. Multiple hypothetical prefixes are realized by fast prefix generation algorithm. With two different rescoring methods, nar-ar rescoring and nar\u0000<inline-formula><tex-math>$^{2}$</tex-math></inline-formula>\u0000 rescoring, Hypformer can flexibly switch between autoregressive and non-autoregressive decoding modes to perform rescoring of hypothesis prefixes. Experiments on the standard Mandarin datasets AISHELL-1 and AISHELL-2 demonstrate that Hypformer outperforms the state-of-the-art Hybrid CTC/Attention systems in ASR performance while achieving a speedup of over six times. Experiments on the Mandarin sub-dialect dataset KeSpeech indicate that Hypformer achieves more accurate recognition by leveraging richer contextual information.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"471-475"},"PeriodicalIF":3.2,"publicationDate":"2024-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142937864","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：