Pub Date : 2023-12-08DOI: 10.1109/OJSP.2023.3338430
{"title":"List of Reviewers","authors":"","doi":"10.1109/OJSP.2023.3338430","DOIUrl":"https://doi.org/10.1109/OJSP.2023.3338430","url":null,"abstract":"","PeriodicalId":73300,"journal":{"name":"IEEE open journal of signal processing","volume":"4 ","pages":"489-490"},"PeriodicalIF":0.0,"publicationDate":"2023-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10352169","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138558055","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-06DOI: 10.1109/OJSP.2023.3340063
Marvin Borsdorf;Siqi Cai;Saurav Pahuja;Dashanka De Silva;Haizhou Li;Tanja Schultz
For the development of neuro-steered hearing aids, it is important to study the relationship between a speech stimulus and the elicited EEG response of a human listener. The recent Auditory EEG Decoding Challenge 2023 (Signal Processing Grand Challenge, IEEE International Conference on Acoustics, Speech and Signal Processing) dealt with this relationship in the context of a match-mismatch classification task. The challenge's task was to find the speech stimulus that elicited a specific EEG response from two given speech stimuli. Participating in the challenge, we adopted the challenge's baseline model and explored an attention encoder to replace the spatial convolution in the EEG processing pipeline, as well as additional sequence modeling methods based on RNN, LSTM, and GRU to preprocess the speech stimuli. We compared speech envelopes and mel-spectrograms as two different types of input speech stimulus and evaluated our models on a test set as well as held-out stories and held-out subjects benchmark sets. In this work, we show that the mel-spectrograms generally offer better results. Replacing the spatial convolution with an attention encoder helps to capture better spatial and temporal information in the EEG response. Additionally, the sequence modeling methods can further enhance the performance, when mel-spectrograms are used. Consequently, both lead to higher performances on the test set and held-out stories benchmark set. Our best model outperforms the baseline by 1.91% on the test set and 1.35% on the total ranking score. We ranked second in the challenge.
{"title":"Attention and Sequence Modeling for Match-Mismatch Classification of Speech Stimulus and EEG Response","authors":"Marvin Borsdorf;Siqi Cai;Saurav Pahuja;Dashanka De Silva;Haizhou Li;Tanja Schultz","doi":"10.1109/OJSP.2023.3340063","DOIUrl":"https://doi.org/10.1109/OJSP.2023.3340063","url":null,"abstract":"For the development of neuro-steered hearing aids, it is important to study the relationship between a speech stimulus and the elicited EEG response of a human listener. The recent Auditory EEG Decoding Challenge 2023 (Signal Processing Grand Challenge, IEEE International Conference on Acoustics, Speech and Signal Processing) dealt with this relationship in the context of a match-mismatch classification task. The challenge's task was to find the speech stimulus that elicited a specific EEG response from two given speech stimuli. Participating in the challenge, we adopted the challenge's baseline model and explored an attention encoder to replace the spatial convolution in the EEG processing pipeline, as well as additional sequence modeling methods based on RNN, LSTM, and GRU to preprocess the speech stimuli. We compared speech envelopes and mel-spectrograms as two different types of input speech stimulus and evaluated our models on a test set as well as held-out stories and held-out subjects benchmark sets. In this work, we show that the mel-spectrograms generally offer better results. Replacing the spatial convolution with an attention encoder helps to capture better spatial and temporal information in the EEG response. Additionally, the sequence modeling methods can further enhance the performance, when mel-spectrograms are used. Consequently, both lead to higher performances on the test set and held-out stories benchmark set. Our best model outperforms the baseline by 1.91% on the test set and 1.35% on the total ranking score. We ranked second in the challenge.","PeriodicalId":73300,"journal":{"name":"IEEE open journal of signal processing","volume":"5 ","pages":"799-809"},"PeriodicalIF":2.9,"publicationDate":"2023-12-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10345738","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142013430","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-06DOI: 10.1109/OJSP.2023.3340106
Miguel Ferrer;María de Diego;Alberto Gonzalez
The LMS algorithm is widely employed in adaptive systems due to its robustness, simplicity, and reasonable performance. However, it is well known that this algorithm suffers from a slow convergence speed when dealing with colored reference signals. Numerous variants and alternative algorithms have been proposed to address this issue, though all of them entail an increase in computational cost. Among the proposed alternatives, the affine projection algorithm stands out. This algorithm has the peculiarity of starting from $N$ data vectors of the reference signal. It transforms these vectors into as many data vectors suitably normalized in energy and mutually orthogonal. In this work, we propose a version of the LMS algorithm that, similar to the affine projection algorithm, starts from $N$ data vectors of the reference signal but corrects them by using only a scalar factor that functions as a convergence step. Our goal is to align the behavior of this algorithm with the behavior of the affine projection algorithm without significantly increasing the computational cost of the LMS.
{"title":"Low Cost Variable Step-Size LMS With Maximum Similarity to the Affine Projection Algorithm","authors":"Miguel Ferrer;María de Diego;Alberto Gonzalez","doi":"10.1109/OJSP.2023.3340106","DOIUrl":"https://doi.org/10.1109/OJSP.2023.3340106","url":null,"abstract":"The LMS algorithm is widely employed in adaptive systems due to its robustness, simplicity, and reasonable performance. However, it is well known that this algorithm suffers from a slow convergence speed when dealing with colored reference signals. Numerous variants and alternative algorithms have been proposed to address this issue, though all of them entail an increase in computational cost. Among the proposed alternatives, the affine projection algorithm stands out. This algorithm has the peculiarity of starting from \u0000<inline-formula><tex-math>$N$</tex-math></inline-formula>\u0000 data vectors of the reference signal. It transforms these vectors into as many data vectors suitably normalized in energy and mutually orthogonal. In this work, we propose a version of the LMS algorithm that, similar to the affine projection algorithm, starts from \u0000<inline-formula><tex-math>$N$</tex-math></inline-formula>\u0000 data vectors of the reference signal but corrects them by using only a scalar factor that functions as a convergence step. Our goal is to align the behavior of this algorithm with the behavior of the affine projection algorithm without significantly increasing the computational cost of the LMS.","PeriodicalId":73300,"journal":{"name":"IEEE open journal of signal processing","volume":"5 ","pages":"82-91"},"PeriodicalIF":0.0,"publicationDate":"2023-12-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10345730","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139060180","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-06DOI: 10.1109/OJSP.2023.3340057
Eric Grinstein;Christopher M. Hicks;Toon van Waterschoot;Mike Brookes;Patrick A. Naylor
Neural networks have achieved state-of-the-art performance on the task of acoustic Direction-of-Arrival (DOA) estimation using microphone arrays. Neural models can be classified as end-to-end or hybrid, each class showing advantages and disadvantages. This work introduces Neural-SRP, an end-to-end neural network architecture for DOA estimation inspired by the classical Steered Response Power (SRP) method, which overcomes limitations of current neural models. We evaluate the architecture on multiple scenarios, namely, multi-source DOA tracking and single-source DOA tracking under the presence of directional and diffuse noise. The experiments demonstrate that our proposed method compares favourably in terms of computational and localization performance with established neural methods on various recorded and simulated benchmark datasets.
神经网络在利用麦克风阵列进行声波到达方向（DOA）估算的任务中取得了最先进的性能。神经模型可分为端到端和混合模型，每一类都各有优缺点。本研究介绍了 Neural-SRP，这是一种用于 DOA 估计的端到端神经网络架构，其灵感来自经典的转向响应功率（SRP）方法，克服了当前神经模型的局限性。我们在多种情况下对该架构进行了评估，即在存在定向噪声和漫反射噪声的情况下进行多源 DOA 跟踪和单源 DOA 跟踪。实验证明，在各种记录和模拟基准数据集上，我们提出的方法在计算和定位性能方面优于已有的神经方法。
{"title":"The Neural-SRP Method for Universal Robust Multi-Source Tracking","authors":"Eric Grinstein;Christopher M. Hicks;Toon van Waterschoot;Mike Brookes;Patrick A. Naylor","doi":"10.1109/OJSP.2023.3340057","DOIUrl":"https://doi.org/10.1109/OJSP.2023.3340057","url":null,"abstract":"Neural networks have achieved state-of-the-art performance on the task of acoustic Direction-of-Arrival (DOA) estimation using microphone arrays. Neural models can be classified as end-to-end or hybrid, each class showing advantages and disadvantages. This work introduces Neural-SRP, an end-to-end neural network architecture for DOA estimation inspired by the classical Steered Response Power (SRP) method, which overcomes limitations of current neural models. We evaluate the architecture on multiple scenarios, namely, multi-source DOA tracking and single-source DOA tracking under the presence of directional and diffuse noise. The experiments demonstrate that our proposed method compares favourably in terms of computational and localization performance with established neural methods on various recorded and simulated benchmark datasets.","PeriodicalId":73300,"journal":{"name":"IEEE open journal of signal processing","volume":"5 ","pages":"19-28"},"PeriodicalIF":0.0,"publicationDate":"2023-12-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10345765","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139060204","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-06DOI: 10.1109/OJSP.2023.3340111
AMIT EFRAIM;Joseph M. Francos
In solving a model fitting problem, the existence of outliers in the set of measurements can have a devastating effect on the solution accuracy. Traditionally, in order to overcome this problem, robust point cloud registration algorithms are composed of transformation hypothesis generation, followed by hypothesis evaluation aimed at selecting the best hypothesized estimate. Hypotheses evaluation is commonly performed using the sample consensus criterion. However, since this criterion accounts only for the consensus size, it fails when the maximal sample consensus is incorrect. We propose a new hypothesis evaluation approach, generalizing the sample consensus approach, where instead of the sample consensus, the transformation that maximizes the point clouds feature correlation is selected as the best hypothesis. The feature vector at each point contains information such as on local geometry and semantic context. Utilizing this information in the hypotheses evaluation and selection procedure allows for a correct decision even when the hypothesis yielding the maximal sample consensus is false. Consequently, the probability of selecting the correct model increases. We show both mathematically and empirically that substituting the sample consensus criterion with the proposed point cloud feature correlation hypothesis test (PC-FCHT) lowers the probability of large registration errors, compared to using the special case of sample consensus. The proposed PC-FCHT is applicable to any algorithm that follows the hypothesis generation and evaluation scheme, potentially improving the success rates of a wide family of point cloud registration algorithms.
{"title":"On Minimizing the Probability of Large Errors in Robust Point Cloud Registration","authors":"AMIT EFRAIM;Joseph M. Francos","doi":"10.1109/OJSP.2023.3340111","DOIUrl":"https://doi.org/10.1109/OJSP.2023.3340111","url":null,"abstract":"In solving a model fitting problem, the existence of outliers in the set of measurements can have a devastating effect on the solution accuracy. Traditionally, in order to overcome this problem, robust point cloud registration algorithms are composed of transformation hypothesis generation, followed by hypothesis evaluation aimed at selecting the best hypothesized estimate. Hypotheses evaluation is commonly performed using the sample consensus criterion. However, since this criterion accounts only for the consensus size, it fails when the maximal sample consensus is incorrect. We propose a new hypothesis evaluation approach, generalizing the sample consensus approach, where instead of the sample consensus, the transformation that maximizes the point clouds feature correlation is selected as the best hypothesis. The feature vector at each point contains information such as on local geometry and semantic context. Utilizing this information in the hypotheses evaluation and selection procedure allows for a correct decision even when the hypothesis yielding the maximal sample consensus is false. Consequently, the probability of selecting the correct model increases. We show both mathematically and empirically that substituting the sample consensus criterion with the proposed point cloud feature correlation hypothesis test (PC-FCHT) lowers the probability of large registration errors, compared to using the special case of sample consensus. The proposed PC-FCHT is applicable to any algorithm that follows the hypothesis generation and evaluation scheme, potentially improving the success rates of a wide family of point cloud registration algorithms.","PeriodicalId":73300,"journal":{"name":"IEEE open journal of signal processing","volume":"5 ","pages":"39-47"},"PeriodicalIF":0.0,"publicationDate":"2023-12-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10345750","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139060220","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-06DOI: 10.1109/OJSP.2023.3340064
Anh Minh Truong;Wilfried Philips;Peter Veelaert
Many image-based rendering (IBR) methods rely on depth estimates obtained from structured light or time-of-flight depth sensors to synthesize novel views from sparse camera networks. However, these estimates often contain missing or noisy regions, resulting in an incorrect mapping between source and target views. This situation makes the fusion process more challenging, as the visual information is misaligned, inconsistent, or missing. In this work, we first implement a lightweight network based on the transformer, which is well-known for its capability to model long-range relationships within input data, to extract spatial features from color images. These features are then used to enhance the quality of completed depth maps. Furthermore, we combine a sequential deep neural network with a spatial attention mechanism to effectively fuse the projected features from multiple source viewpoints. This approach enables us to integrate information from an arbitrary number of source viewpoints as well as improve accuracy in synthesized views. Experimental results on challenging datasets demonstrate that our method achieves superior synthesized image quality compared to state-of-the-art (SOTA) methods.
{"title":"Exploiting a Spatial Attention Mechanism for Improved Depth Completion and Feature Fusion in Novel View Synthesis","authors":"Anh Minh Truong;Wilfried Philips;Peter Veelaert","doi":"10.1109/OJSP.2023.3340064","DOIUrl":"https://doi.org/10.1109/OJSP.2023.3340064","url":null,"abstract":"Many image-based rendering (IBR) methods rely on depth estimates obtained from structured light or time-of-flight depth sensors to synthesize novel views from sparse camera networks. However, these estimates often contain missing or noisy regions, resulting in an incorrect mapping between source and target views. This situation makes the fusion process more challenging, as the visual information is misaligned, inconsistent, or missing. In this work, we first implement a lightweight network based on the transformer, which is well-known for its capability to model long-range relationships within input data, to extract spatial features from color images. These features are then used to enhance the quality of completed depth maps. Furthermore, we combine a sequential deep neural network with a spatial attention mechanism to effectively fuse the projected features from multiple source viewpoints. This approach enables us to integrate information from an arbitrary number of source viewpoints as well as improve accuracy in synthesized views. Experimental results on challenging datasets demonstrate that our method achieves superior synthesized image quality compared to state-of-the-art (SOTA) methods.","PeriodicalId":73300,"journal":{"name":"IEEE open journal of signal processing","volume":"5 ","pages":"204-212"},"PeriodicalIF":0.0,"publicationDate":"2023-12-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10345792","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139060244","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-05DOI: 10.1109/OJSP.2023.3339433
Gal Itzhak;Israel Cohen
This article presents a Kronecker-product (KP) beamforming approach incorporating sparse concentric circular arrays (SCCAs). The locations of the microphones on the SCCA are optimized concerning the broadband array directivity over a wide range of direction-of-arrival (DOA) deviations of a desired signal. A maximum directivity factor (MDF) sub-beamformer is derived accordingly with the optimal locations. Then, we propose two global beamformers obtained as a Kronecker product of a uniform linear array (ULA) and the SCCA sub-beamformer. The global beamformers differ by the type of the ULA, which is designed either as an MDF sub-beamformer along the $mathsf {x}$ -axis or as a maximum white noise gain sub-beamformer along the $mathsf {y}$ -axis. We analyze the performance of the proposed beamformers in terms of the directivity factor, the white noise gain, and their spatial beampatterns. Compared to traditional beamformers, the proposed beamformers exhibit considerably larger tolerance to DOA deviations concerning both the azimuth and elevation angles. Experimental results with speech signals in noisy and reverberant environments demonstrate that the proposed approach outperforms traditional beamformers regarding the perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI) scores when the desired speech signals deviate from the nominal DOA.
{"title":"Kronecker-Product Beamforming With Sparse Concentric Circular Arrays","authors":"Gal Itzhak;Israel Cohen","doi":"10.1109/OJSP.2023.3339433","DOIUrl":"https://doi.org/10.1109/OJSP.2023.3339433","url":null,"abstract":"This article presents a Kronecker-product (KP) beamforming approach incorporating sparse concentric circular arrays (SCCAs). The locations of the microphones on the SCCA are optimized concerning the broadband array directivity over a wide range of direction-of-arrival (DOA) deviations of a desired signal. A maximum directivity factor (MDF) sub-beamformer is derived accordingly with the optimal locations. Then, we propose two global beamformers obtained as a Kronecker product of a uniform linear array (ULA) and the SCCA sub-beamformer. The global beamformers differ by the type of the ULA, which is designed either as an MDF sub-beamformer along the \u0000<inline-formula><tex-math>$mathsf {x}$</tex-math></inline-formula>\u0000-axis or as a maximum white noise gain sub-beamformer along the \u0000<inline-formula><tex-math>$mathsf {y}$</tex-math></inline-formula>\u0000-axis. We analyze the performance of the proposed beamformers in terms of the directivity factor, the white noise gain, and their spatial beampatterns. Compared to traditional beamformers, the proposed beamformers exhibit considerably larger tolerance to DOA deviations concerning both the azimuth and elevation angles. Experimental results with speech signals in noisy and reverberant environments demonstrate that the proposed approach outperforms traditional beamformers regarding the perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI) scores when the desired speech signals deviate from the nominal DOA.","PeriodicalId":73300,"journal":{"name":"IEEE open journal of signal processing","volume":"5 ","pages":"64-72"},"PeriodicalIF":0.0,"publicationDate":"2023-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10342869","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139060221","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-05DOI: 10.1109/OJSP.2023.3339376
Chengen Liu;Geert Leus;Elvin Isufi
The edge flow reconstruction task consists of retreiving edge flow signals from corrupted or incomplete measurements. This is typically solved by a regularized optimization problem on higher-order networks such as simplicial complexes and the corresponding regularizers are chosen based on prior knowledge. Tailoring this prior to the setting of interest can be challenging or it may not even be possible. Thus, we consider to learn this prior knowledge via a model-based deep learning approach. We propose a new regularized optimization problem for the simplicial edge flow reconstruction task, the simplicial ElasticNet, which combines the advantages of the $ell _{1}$ and $ell _{2}$ norms. We solve the simplicial ElasticNet problem via the multi-block alternating direction method of multipliers (ADMM) algorithm and provide conditions on its convergence. By unrolling the ADMM iterative steps, we develop a model-based neural network with a low requirement on the number of training data. This unrolling network replaces the fixed parameters in the iterative algorithm by learnable weights, thus exploiting the neural network's learning capability while preserving the iterative algorithm's interpretability. We enhance this unrolling network via simplicial convolutional filters to aggregate information from the edge flow neighbors, ultimately, improving the network learning expressivity. Extensive experiments on real-world and synthetic datasets validate the proposed approaches and show considerable improvements over both baselines and traditional non-model-based neural networks.
{"title":"Unrolling of Simplicial ElasticNet for Edge Flow Signal Reconstruction","authors":"Chengen Liu;Geert Leus;Elvin Isufi","doi":"10.1109/OJSP.2023.3339376","DOIUrl":"https://doi.org/10.1109/OJSP.2023.3339376","url":null,"abstract":"The edge flow reconstruction task consists of retreiving edge flow signals from corrupted or incomplete measurements. This is typically solved by a regularized optimization problem on higher-order networks such as simplicial complexes and the corresponding regularizers are chosen based on prior knowledge. Tailoring this prior to the setting of interest can be challenging or it may not even be possible. Thus, we consider to learn this prior knowledge via a model-based deep learning approach. We propose a new regularized optimization problem for the simplicial edge flow reconstruction task, the simplicial ElasticNet, which combines the advantages of the \u0000<inline-formula><tex-math>$ell _{1}$</tex-math></inline-formula>\u0000 and \u0000<inline-formula><tex-math>$ell _{2}$</tex-math></inline-formula>\u0000 norms. We solve the simplicial ElasticNet problem via the multi-block alternating direction method of multipliers (ADMM) algorithm and provide conditions on its convergence. By unrolling the ADMM iterative steps, we develop a model-based neural network with a low requirement on the number of training data. This unrolling network replaces the fixed parameters in the iterative algorithm by learnable weights, thus exploiting the neural network's learning capability while preserving the iterative algorithm's interpretability. We enhance this unrolling network via simplicial convolutional filters to aggregate information from the edge flow neighbors, ultimately, improving the network learning expressivity. Extensive experiments on real-world and synthetic datasets validate the proposed approaches and show considerable improvements over both baselines and traditional non-model-based neural networks.","PeriodicalId":73300,"journal":{"name":"IEEE open journal of signal processing","volume":"5 ","pages":"186-194"},"PeriodicalIF":0.0,"publicationDate":"2023-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10342735","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139060255","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-05DOI: 10.1109/OJSP.2023.3339428
Karn N. Watcharasupat;Chih-Wei Wu;Yiwei Ding;Iroro Orife;Aaron J. Hipple;Phillip A. Williams;Scott Kramer;Alexander Lerch;William Wolcott
Cinematic audio source separation is a relatively new subtask of audio source separation, with the aim of extracting the dialogue, music, and effects stems from their mixture. In this work, we developed a model generalizing the Bandsplit RNN for any complete or overcomplete partitions of the frequency axis. Psychoacoustically motivated frequency scales were used to inform the band definitions which are now defined with redundancy for more reliable feature extraction. A loss function motivated by the signal-to-noise ratio and the sparsity-promoting property of the 1-norm was proposed. We additionally exploit the information-sharing property of a common-encoder setup to reduce computational complexity during both training and inference, improve separation performance for hard-to-generalize classes of sounds, and allow flexibility during inference time with detachable decoders. Our best model sets the state of the art on the Divide and Remaster dataset with performance above the ideal ratio mask for the dialogue stem.
电影音源分离是音源分离中一个相对较新的子任务，其目的是从混合物中提取对白、音乐和特效。在这项工作中，我们开发了一个模型，对频带分割 RNN 进行了概括，适用于任何完整或过度完整的频轴分割。以心理声学为动机的频率标度被用来为频段定义提供信息，而现在的频段定义具有冗余性，可以进行更可靠的特征提取。我们提出了一个由信噪比和 1-norm 的稀疏性促进特性激发的损失函数。此外，我们还利用共同编码器设置的信息共享特性，降低了训练和推理过程中的计算复杂度，提高了难以归纳的声音类别的分离性能，并允许在推理过程中使用可拆卸解码器的灵活性。我们的最佳模型在 "Divide and Remaster "数据集上树立了技术典范，其性能高于对话干的理想比率掩码。
{"title":"A Generalized Bandsplit Neural Network for Cinematic Audio Source Separation","authors":"Karn N. Watcharasupat;Chih-Wei Wu;Yiwei Ding;Iroro Orife;Aaron J. Hipple;Phillip A. Williams;Scott Kramer;Alexander Lerch;William Wolcott","doi":"10.1109/OJSP.2023.3339428","DOIUrl":"https://doi.org/10.1109/OJSP.2023.3339428","url":null,"abstract":"Cinematic audio source separation is a relatively new subtask of audio source separation, with the aim of extracting the dialogue, music, and effects stems from their mixture. In this work, we developed a model generalizing the Bandsplit RNN for any complete or overcomplete partitions of the frequency axis. Psychoacoustically motivated frequency scales were used to inform the band definitions which are now defined with redundancy for more reliable feature extraction. A loss function motivated by the signal-to-noise ratio and the sparsity-promoting property of the 1-norm was proposed. We additionally exploit the information-sharing property of a common-encoder setup to reduce computational complexity during both training and inference, improve separation performance for hard-to-generalize classes of sounds, and allow flexibility during inference time with detachable decoders. Our best model sets the state of the art on the Divide and Remaster dataset with performance above the ideal ratio mask for the dialogue stem.","PeriodicalId":73300,"journal":{"name":"IEEE open journal of signal processing","volume":"5 ","pages":"73-81"},"PeriodicalIF":0.0,"publicationDate":"2023-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10342812","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139060182","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-30DOI: 10.1109/OJSP.2023.3338113
Alexander Venus;Erik Leitinger;Stefan Tertinek;Klaus Witrisal
This paper presents a neural-enhanced probabilistic model and corresponding factor graph-based sum-product algorithm for robust localization and tracking in multipath-prone environments. The introduced hybrid probabilistic model consists of physics-based and data-driven measurement models capturing the information contained in both, the line-of-sight (LOS) component as well as in multipath components (NLOS components). The physics-based and data-driven models are embedded in a joint Bayesian framework allowing to derive from first principles a factor graph-based algorithm that fuses the information of these models. The proposed algorithm uses radio signal measurements from multiple base stations to robustly estimate the mobile agent's position together with all model parameters. It provides high localization accuracy by exploiting the position-related information of the LOS component via the physics-based model and robustness by exploiting the geometric imprint of multipath components independent of the propagation channel via the data-driven model. In a challenging numerical experiment involving obstructed LOS situations to all anchors, we show that the proposed sequential algorithm significantly outperforms state-of-the-art methods and attains the posterior Cramér-Rao lower bound even with training data limited to local regions.
本文提出了一种神经增强概率模型和相应的基于因子图的和积算法，用于多径环境中的鲁棒定位和跟踪。引入的混合概率模型由基于物理和数据驱动的测量模型组成，可捕捉视线（LOS）分量和多径分量（NLOS 分量）中包含的信息。基于物理和数据驱动的模型被嵌入到一个联合贝叶斯框架中，从而可以从第一原理推导出一种基于因子图的算法，将这些模型的信息融合在一起。所提出的算法使用来自多个基站的无线电信号测量结果来稳健地估计移动代理的位置以及所有模型参数。该算法通过基于物理的模型，利用 LOS 分量的位置相关信息，实现了高定位精度；通过数据驱动模型，利用独立于传播信道的多径分量的几何印记，实现了鲁棒性。在一个具有挑战性的数值实验中，在所有锚点的 LOS 均受阻的情况下，我们发现所提出的序列算法明显优于最先进的方法，即使训练数据仅限于局部区域，也能达到后 Cramér-Rao 下限。
{"title":"A Neural-Enhanced Factor Graph-Based Algorithm for Robust Positioning in Obstructed LOS Situations","authors":"Alexander Venus;Erik Leitinger;Stefan Tertinek;Klaus Witrisal","doi":"10.1109/OJSP.2023.3338113","DOIUrl":"https://doi.org/10.1109/OJSP.2023.3338113","url":null,"abstract":"This paper presents a neural-enhanced probabilistic model and corresponding factor graph-based sum-product algorithm for robust localization and tracking in multipath-prone environments. The introduced hybrid probabilistic model consists of physics-based and data-driven measurement models capturing the information contained in both, the line-of-sight (LOS) component as well as in multipath components (NLOS components). The physics-based and data-driven models are embedded in a joint Bayesian framework allowing to derive from first principles a factor graph-based algorithm that fuses the information of these models. The proposed algorithm uses radio signal measurements from multiple base stations to robustly estimate the mobile agent's position together with all model parameters. It provides high localization accuracy by exploiting the position-related information of the LOS component via the physics-based model and robustness by exploiting the geometric imprint of multipath components independent of the propagation channel via the data-driven model. In a challenging numerical experiment involving obstructed LOS situations to all anchors, we show that the proposed sequential algorithm significantly outperforms state-of-the-art methods and attains the posterior Cramér-Rao lower bound even with training data limited to local regions.","PeriodicalId":73300,"journal":{"name":"IEEE open journal of signal processing","volume":"5 ","pages":"29-38"},"PeriodicalIF":0.0,"publicationDate":"2023-11-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10336409","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139060334","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：