Pub Date : 2024-09-16DOI: 10.1109/TSP.2024.3460690
Nicolò Michelusi
Implementing Decentralized Gradient Descent (DGD) in wireless systems is challenging due to noise, fading, and limited bandwidth, necessitating topology awareness, transmission scheduling, and the acquisition of channel state information (CSI) to mitigate interference and maintain reliable communications. These operations may result in substantial signaling overhead and scalability challenges in large networks lacking central coordination. This paper introduces a scalable DGD algorithm that eliminates the need for scheduling, topology information, or CSI (both average and instantaneous). At its core is a Non-Coherent Over-The-Air (NCOTA) consensus scheme that exploits a noisy energy superposition property of wireless channels. Nodes encode their local optimization signals into energy levels within an OFDM frame and transmit simultaneously, without coordination. The key insight is that the received energy equals, on average , the sum of the energies of the transmitted signals, scaled by their respective average channel gains, akin to a consensus step. This property enables unbiased consensus estimation, utilizing average channel gains as mixing weights, thereby removing the need for their explicit design or for CSI. Introducing a consensus stepsize mitigates consensus estimation errors due to energy fluctuations around their expected values. For strongly-convex problems, it is shown that the expected squared distance between the local and globally optimum models vanishes at a rate of $mathcal{O}(1/sqrt{k})$ after $k$ iterations, with suitable decreasing learning and consensus stepsizes. Extensions accommodate a broad class of fading models and frequency-selective channels. Numerical experiments on image classification demonstrate faster convergence in terms of running time compared to state-of-the-art schemes, especially in dense network scenarios.
{"title":"Non-Coherent Over-the-Air Decentralized Gradient Descent","authors":"Nicolò Michelusi","doi":"10.1109/TSP.2024.3460690","DOIUrl":"10.1109/TSP.2024.3460690","url":null,"abstract":"Implementing Decentralized Gradient Descent (DGD) in wireless systems is challenging due to noise, fading, and limited bandwidth, necessitating topology awareness, transmission scheduling, and the acquisition of channel state information (CSI) to mitigate interference and maintain reliable communications. These operations may result in substantial signaling overhead and scalability challenges in large networks lacking central coordination. This paper introduces a scalable DGD algorithm that eliminates the need for scheduling, topology information, or CSI (both average and instantaneous). At its core is a Non-Coherent Over-The-Air (NCOTA) consensus scheme that exploits a noisy energy superposition property of wireless channels. Nodes encode their local optimization signals into energy levels within an OFDM frame and transmit simultaneously, without coordination. The key insight is that the received energy equals, \u0000<italic>on average</i>\u0000, the sum of the energies of the transmitted signals, scaled by their respective average channel gains, akin to a consensus step. This property enables unbiased consensus estimation, utilizing average channel gains as mixing weights, thereby removing the need for their explicit design or for CSI. Introducing a consensus stepsize mitigates consensus estimation errors due to energy fluctuations around their expected values. For strongly-convex problems, it is shown that the expected squared distance between the local and globally optimum models vanishes at a rate of \u0000<inline-formula><tex-math>$mathcal{O}(1/sqrt{k})$</tex-math></inline-formula>\u0000 after \u0000<inline-formula><tex-math>$k$</tex-math></inline-formula>\u0000 iterations, with suitable decreasing learning and consensus stepsizes. Extensions accommodate a broad class of fading models and frequency-selective channels. Numerical experiments on image classification demonstrate faster convergence in terms of running time compared to state-of-the-art schemes, especially in dense network scenarios.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"72 ","pages":"4618-4634"},"PeriodicalIF":4.6,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142236321","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-16DOI: 10.1109/TSP.2024.3457817
Jiageng Wu;Zhiguo Wang;Ya-Feng Liu;Fan Liu
In this paper, we propose a MIMO transmit beamforming optimization model for joint radar sensing and multi-user communications, where the design of the beamformers is formulated as an optimization problem whose objective is a weighted combination of the sum rate and the Cramér-Rao bound, subject to the transmit power budget. Obtaining the global solution for the formulated nonconvex problem is a challenging task, since the sum-rate maximization problem itself (even without considering the sensing metric) is known to be NP-hard. The main contributions of this paper are threefold. Firstly, we derive an optimal closed-form solution to the formulated problem in the single-user case and the multi-user case where the channel vectors of different users are orthogonal. Secondly, for the general multi-user case, we propose a novel branch and bound (B&B) algorithm based on the McCormick envelope relaxation. The proposed algorithm is guaranteed to find the globally optimal solution to the formulated problem. Thirdly, we design a graph neural network (GNN) based pruning policy to determine irrelevant nodes that can be directly pruned in the proposed B&B algorithm, thereby significantly reducing the number of unnecessary enumerations therein and improving its computational efficiency. Simulation results show the efficiency of the proposed vanilla and GNN-based accelerated B&B algorithms.
{"title":"Efficient Global Algorithms for Transmit Beamforming Design in ISAC Systems","authors":"Jiageng Wu;Zhiguo Wang;Ya-Feng Liu;Fan Liu","doi":"10.1109/TSP.2024.3457817","DOIUrl":"https://doi.org/10.1109/TSP.2024.3457817","url":null,"abstract":"In this paper, we propose a MIMO transmit beamforming optimization model for joint radar sensing and multi-user communications, where the design of the beamformers is formulated as an optimization problem whose objective is a weighted combination of the sum rate and the Cramér-Rao bound, subject to the transmit power budget. Obtaining the global solution for the formulated nonconvex problem is a challenging task, since the sum-rate maximization problem itself (even without considering the sensing metric) is known to be NP-hard. The main contributions of this paper are threefold. Firstly, we derive an optimal closed-form solution to the formulated problem in the single-user case and the multi-user case where the channel vectors of different users are orthogonal. Secondly, for the general multi-user case, we propose a novel branch and bound (B&B) algorithm based on the McCormick envelope relaxation. The proposed algorithm is guaranteed to find the globally optimal solution to the formulated problem. Thirdly, we design a graph neural network (GNN) based pruning policy to determine irrelevant nodes that can be directly pruned in the proposed B&B algorithm, thereby significantly reducing the number of unnecessary enumerations therein and improving its computational efficiency. Simulation results show the efficiency of the proposed vanilla and GNN-based accelerated B&B algorithms.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"72 ","pages":"4493-4508"},"PeriodicalIF":4.6,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142452673","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-16DOI: 10.1109/TSP.2024.3457778
Tao Wang;Junli Liang;H. C. So;Hongwei Gao;Yongkang Li
To detect high-speed moving targets using multiple-transmitter systems, the transmit sequence set should possess Doppler tolerance performance similar to that of linear frequency modulated signal, characterized by a diagonal-ridge-type auto-ambiguity function (AF). Although they provide space and waveform diversities to enhance sensing performance compared to single-transmitter systems, the cross-AFs introduced by the multiple transmit sequences decrease the degrees-of-freedom (DOFs), and thus the sidelobe levels of both the auto- and cross- AFs are not sufficiently low. In this paper, we first establish the relative level (between AF sidelobe and mainlobe) -based fractional models to jointly design the transmit sequence set and mismatched filter bank with diagonal-ridge-type Doppler tolerance and low sidelobe AFs. Moreover, a novel similarity constraint between the sequence and mismatched filter is devised to flexibly adjust the receiver output loss. Together with the relative-level scheme, it also reduces the DOF loss incurred by the cross AFs. Furthermore, the resultant challenging sum-of-fraction formulations with nonlinear and nonconvex constraints are effectively tackled via fraction separation and decoupling. Finally, we extend our design to solve the joint design problem of sequence set and mismatched filter bank with low sidelobe level in correlation function. Numerical results demonstrate the excellent performance of the proposed methods.
要利用多发射机系统探测高速移动目标，发射序列集应具有类似于线性频率调制信号的多普勒容差性能，其特征是对角线脊型自动模糊函数（AF）。虽然与单发射机系统相比，多发射机系统提供了空间和波形多样性以提高传感性能，但多发射序列引入的交叉自动模糊函数降低了自由度（DOFs），因此自动模糊函数和交叉自动模糊函数的侧叶电平都不够低。在本文中，我们首先建立了基于分数模型的相对水平（AF 边瓣和主瓣之间的相对水平），以联合设计具有对角线脊型多普勒容限和低边瓣 AF 的发射序列集和错配滤波器组。此外，还设计了序列与失配滤波器之间的新型相似性约束，以灵活调整接收器输出损耗。与相对级方案一起，它还能减少交叉自动变频造成的 DOF 损失。此外，我们还通过分数分离和解耦，有效地解决了由此产生的具有非线性和非凸约束的具有挑战性的分数总和公式。最后，我们对设计进行了扩展，以解决序列集和失配滤波器组的联合设计问题，并在相关函数中实现低边音水平。数值结果证明了所提方法的卓越性能。
{"title":"Designing Sequence Set and Mismatched Filter Bank With Diagonal-Ridge-Type Doppler Tolerance via Relative Level Optimization","authors":"Tao Wang;Junli Liang;H. C. So;Hongwei Gao;Yongkang Li","doi":"10.1109/TSP.2024.3457778","DOIUrl":"10.1109/TSP.2024.3457778","url":null,"abstract":"To detect high-speed moving targets using multiple-transmitter systems, the transmit sequence set should possess Doppler tolerance performance similar to that of linear frequency modulated signal, characterized by a diagonal-ridge-type auto-ambiguity function (AF). Although they provide space and waveform diversities to enhance sensing performance compared to single-transmitter systems, the cross-AFs introduced by the multiple transmit sequences decrease the degrees-of-freedom (DOFs), and thus the sidelobe levels of both the auto- and cross- AFs are not sufficiently low. In this paper, we first establish the relative level (between AF sidelobe and mainlobe) -based fractional models to jointly design the transmit sequence set and mismatched filter bank with diagonal-ridge-type Doppler tolerance and low sidelobe AFs. Moreover, a novel similarity constraint between the sequence and mismatched filter is devised to flexibly adjust the receiver output loss. Together with the relative-level scheme, it also reduces the DOF loss incurred by the cross AFs. Furthermore, the resultant challenging sum-of-fraction formulations with nonlinear and nonconvex constraints are effectively tackled via fraction separation and decoupling. Finally, we extend our design to solve the joint design problem of sequence set and mismatched filter bank with low sidelobe level in correlation function. Numerical results demonstrate the excellent performance of the proposed methods.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"72 ","pages":"4412-4427"},"PeriodicalIF":4.6,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142236320","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-16DOI: 10.1109/TSP.2024.3460194
Bishwadeep Das;Elvin Isufi
Graph filters are a staple tool for processing signals over graphs in a multitude of downstream tasks. However, they are commonly designed for graphs with a fixed number of nodes, despite real-world networks typically grow over time. This topological evolution is often known up to a stochastic model, thus, making conventional graph filters ill-equipped to withstand such topological changes, their uncertainty, as well as the dynamic nature of the incoming data. To tackle these issues, we propose an online graph filtering framework by relying on online learning principles. We design filters for scenarios where the topology is both known and unknown, including a learner adaptive to such evolution. We conduct a regret analysis to highlight the role played by the different components such as the online algorithm, the filter order, and the growing graph model. Numerical experiments with synthetic and real data corroborate the proposed approach for graph signal inference tasks and show a competitive performance w.r.t. baselines and state-of-the-art alternatives.
{"title":"Online Graph Filtering Over Expanding Graphs","authors":"Bishwadeep Das;Elvin Isufi","doi":"10.1109/TSP.2024.3460194","DOIUrl":"https://doi.org/10.1109/TSP.2024.3460194","url":null,"abstract":"Graph filters are a staple tool for processing signals over graphs in a multitude of downstream tasks. However, they are commonly designed for graphs with a fixed number of nodes, despite real-world networks typically grow over time. This topological evolution is often known up to a stochastic model, thus, making conventional graph filters ill-equipped to withstand such topological changes, their uncertainty, as well as the dynamic nature of the incoming data. To tackle these issues, we propose an online graph filtering framework by relying on online learning principles. We design filters for scenarios where the topology is both known and unknown, including a learner adaptive to such evolution. We conduct a regret analysis to highlight the role played by the different components such as the online algorithm, the filter order, and the growing graph model. Numerical experiments with synthetic and real data corroborate the proposed approach for graph signal inference tasks and show a competitive performance w.r.t. baselines and state-of-the-art alternatives.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"72 ","pages":"4698-4712"},"PeriodicalIF":4.6,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142524130","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-13DOI: 10.1109/TSP.2024.3460176
Aamir Hussain Chughtai;Muhammad Tahir;Momin Uppal
In this article, we consider the problem of outlier-robust state estimation where the measurement noise can be correlated. Outliers in data arise due to many reasons like sensor malfunctioning, environmental behaviors, communication glitches, etc. Moreover, noise correlation emerges in several real-world applications e.g. sensor networks, radar data, GPS-based systems, etc. We consider these effects in system modeling which is subsequently used for inference. We employ the Expectation-Maximization (EM) framework to derive both outlier-resilient filtering and smoothing methods, suitable for online and offline estimation respectively. The standard Gaussian filtering and the Gaussian Rauch–Tung–Striebel (RTS) smoothing results are leveraged to devise the estimators. In addition, Bayesian Cramer-Rao Bounds (BCRBs) for a filter and a smoother which can perfectly detect and reject outliers are presented. These serve as useful theoretical benchmarks to gauge the error performance of different estimators. Lastly, different numerical experiments, for an illustrative target tracking application, are carried out that indicate performance gains compared to similarly engineered state-of-the-art outlier-rejecting state estimators. The advantages are in terms of simpler implementation, enhanced estimation quality, and competitive computational performance.
{"title":"EMORF/S: EM-Based Outlier-Robust Filtering and Smoothing With Correlated Measurement Noise","authors":"Aamir Hussain Chughtai;Muhammad Tahir;Momin Uppal","doi":"10.1109/TSP.2024.3460176","DOIUrl":"10.1109/TSP.2024.3460176","url":null,"abstract":"In this article, we consider the problem of outlier-robust state estimation where the measurement noise can be correlated. Outliers in data arise due to many reasons like sensor malfunctioning, environmental behaviors, communication glitches, etc. Moreover, noise correlation emerges in several real-world applications e.g. sensor networks, radar data, GPS-based systems, etc. We consider these effects in system modeling which is subsequently used for inference. We employ the Expectation-Maximization (EM) framework to derive both outlier-resilient filtering and smoothing methods, suitable for online and offline estimation respectively. The standard Gaussian filtering and the Gaussian Rauch–Tung–Striebel (RTS) smoothing results are leveraged to devise the estimators. In addition, Bayesian Cramer-Rao Bounds (BCRBs) for a filter and a smoother which can perfectly detect and reject outliers are presented. These serve as useful theoretical benchmarks to gauge the error performance of different estimators. Lastly, different numerical experiments, for an illustrative target tracking application, are carried out that indicate performance gains compared to similarly engineered state-of-the-art outlier-rejecting state estimators. The advantages are in terms of simpler implementation, enhanced estimation quality, and competitive computational performance.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"72 ","pages":"4318-4331"},"PeriodicalIF":4.6,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142231632","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-13DOI: 10.1109/TSP.2024.3458175
Yun Chen;Zixuan Chen;Yunhua Zhang;Jiefang Yang;Dong Li
Design of discrete phase sequence waveform (DPSW) with desirable co- and cross-ambiguity function (AF) properties has been a longstanding and critical challenge in the field of high-performance multichannel electronic systems, e.g. radar systems. This paper focuses on the joint design of Doppler-resilient DPSW and receiving filter with low weighted integrated sidelobe level (WISL) for multichannel radar system. This design aims to construct DPSWs of “thumbtack” shape and all-zero AFs within the desired Range-Doppler region for both co-channels and cross-channels, respectively. A peak constraint function, i.e. the penalty function, is incorporated into the objective function to control the signal-to-noise ratio loss (SNRL) due to mismatched filtering. In the design, unimodular and discrete phase constraints are imposed on each element of the sequences, while the receiving filters are subject to the energy constraint and the mismatch constraint of SNRL. Different constraints on transmitted sequences and receiving filters make the optimization problem difficult to solve. Here, an alternatively iterative algorithm based on the majorization-minimization (MM) and the coordinate descent (CD) frameworks is proposed to handle the differently constrained optimization problem. Moreover, by incorporating a general acceleration scheme and the fast Fourier transform (FFT), the computational efficiency of the proposed algorithm can be further improved. Simulation and practical experiments are conducted to validate the designed DPSWs showing superior performance when compared to that by the latest and representative methods.
{"title":"Joint Design of Doppler Resilient Unimodular Discrete Phase Sequence Waveform and Receiving Filter for Multichannel Radar","authors":"Yun Chen;Zixuan Chen;Yunhua Zhang;Jiefang Yang;Dong Li","doi":"10.1109/TSP.2024.3458175","DOIUrl":"10.1109/TSP.2024.3458175","url":null,"abstract":"Design of discrete phase sequence waveform (DPSW) with desirable co- and cross-ambiguity function (AF) properties has been a longstanding and critical challenge in the field of high-performance multichannel electronic systems, e.g. radar systems. This paper focuses on the joint design of Doppler-resilient DPSW and receiving filter with low weighted integrated sidelobe level (WISL) for multichannel radar system. This design aims to construct DPSWs of “thumbtack” shape and all-zero AFs within the desired Range-Doppler region for both co-channels and cross-channels, respectively. A peak constraint function, i.e. the penalty function, is incorporated into the objective function to control the signal-to-noise ratio loss (SNRL) due to mismatched filtering. In the design, unimodular and discrete phase constraints are imposed on each element of the sequences, while the receiving filters are subject to the energy constraint and the mismatch constraint of SNRL. Different constraints on transmitted sequences and receiving filters make the optimization problem difficult to solve. Here, an alternatively iterative algorithm based on the majorization-minimization (MM) and the coordinate descent (CD) frameworks is proposed to handle the differently constrained optimization problem. Moreover, by incorporating a general acceleration scheme and the fast Fourier transform (FFT), the computational efficiency of the proposed algorithm can be further improved. Simulation and practical experiments are conducted to validate the designed DPSWs showing superior performance when compared to that by the latest and representative methods.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"72 ","pages":"4207-4221"},"PeriodicalIF":4.6,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142231633","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-12DOI: 10.1109/TSP.2024.3459422
Ali Abdelbadie;Mona Mostafa;Salime Bameri;Ramy H. Gohary;Dimple Thomas
We develop two novel algorithms for estimating the direction of arrival (DoA) of multiple sources in fully-connected and partially-connected hybrid analog/digital (HAD) receivers. The first algorithm is based on the observation that the analog combiner projects received signals on a particular subspace, causing the signals corresponding to particular DoAs to be heavily attenuated. Thus, an analog combiner defines spatial sectors, beyond which the DoAs are practically undetectable. To address this difficulty, we perform DoA estimation over an exhaustive set of analog combiners spanning distinct subspaces. To refine the estimates generated by this algorithm, we develop an exponentially-converging algorithm wherein the search window is successively narrowed until convergence. Cramér-Rao lower bounds on the root-mean-square error of the proposed algorithms are derived and the superiority of these algorithms over their existing counterparts is established through numerical simulations.
我们开发了两种新算法，用于估计全连接和部分连接模拟/数字混合（HAD）接收器中多个信号源的到达方向（DoA）。第一种算法基于以下观察结果：模拟合路器将接收到的信号投射到一个特定的子空间，导致与特定到达方向相对应的信号被严重衰减。因此，模拟合路器定义了空间扇区，超出该扇区的 DoAs 几乎无法检测。为了解决这一难题，我们在一组跨越不同子空间的详尽模拟合路器上执行 DoA 估计。为了完善该算法生成的估计值，我们开发了一种指数收敛算法，在该算法中，搜索窗口连续缩小，直至收敛。我们推导出了所提算法均方根误差的克拉梅尔-拉奥（Cramér-Rao）下限，并通过数值模拟证明了这些算法优于现有算法。
{"title":"DoA Estimation for Hybrid Receivers: Full Spatial Coverage and Successive Refinement","authors":"Ali Abdelbadie;Mona Mostafa;Salime Bameri;Ramy H. Gohary;Dimple Thomas","doi":"10.1109/TSP.2024.3459422","DOIUrl":"10.1109/TSP.2024.3459422","url":null,"abstract":"We develop two novel algorithms for estimating the direction of arrival (DoA) of multiple sources in fully-connected and partially-connected hybrid analog/digital (HAD) receivers. The first algorithm is based on the observation that the analog combiner projects received signals on a particular subspace, causing the signals corresponding to particular DoAs to be heavily attenuated. Thus, an analog combiner defines spatial sectors, beyond which the DoAs are practically undetectable. To address this difficulty, we perform DoA estimation over an exhaustive set of analog combiners spanning distinct subspaces. To refine the estimates generated by this algorithm, we develop an exponentially-converging algorithm wherein the search window is successively narrowed until convergence. Cramér-Rao lower bounds on the root-mean-square error of the proposed algorithms are derived and the superiority of these algorithms over their existing counterparts is established through numerical simulations.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"72 ","pages":"4730-4744"},"PeriodicalIF":4.6,"publicationDate":"2024-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142174784","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-11DOI: 10.1109/TSP.2024.3458008
Junbin Liu;Ya Liu;Wing-Kin Ma;Mingjie Shao;Anthony Man-Cho So
This study develops a framework for a class of constant modulus (CM) optimization problems, which covers binary constraints, discrete phase constraints, semi-orthogonal matrix constraints, non-negative semi-orthogonal matrix constraints, and several types of binary assignment constraints. Capitalizing on the basic principles of concave minimization and error bounds, we study a convex-constrained penalized formulation for general CM problems. The advantage of such formulation is that it allows us to leverage non-convex optimization techniques, such as the simple projected gradient method, to build algorithms. As the first part of this study, we explore the theory of this framework. We study conditions under which the formulation provides exact penalization results. We also examine computational aspects relating to the use of the projected gradient method for each type of CM constraint. Our study suggests that the proposed framework has a broad scope of applicability.
本研究为一类常模（CM）优化问题建立了一个框架，其中包括二元约束、离散相位约束、半正交矩阵约束、非负半正交矩阵约束以及几种类型的二元赋值约束。利用凹最小化和误差约束的基本原理，我们研究了一般 CM 问题的凸约束惩罚公式。这种表述的优势在于，它允许我们利用非凸优化技术（如简单的投影梯度法）来构建算法。作为本研究的第一部分，我们探讨了这一框架的理论。我们将研究在哪些条件下，该公式能提供精确的惩罚结果。我们还研究了针对各类 CM 约束条件使用投影梯度法的相关计算问题。我们的研究表明，所提出的框架具有广泛的适用性。
{"title":"Extreme Point Pursuit—Part I: A Framework for Constant Modulus Optimization","authors":"Junbin Liu;Ya Liu;Wing-Kin Ma;Mingjie Shao;Anthony Man-Cho So","doi":"10.1109/TSP.2024.3458008","DOIUrl":"10.1109/TSP.2024.3458008","url":null,"abstract":"This study develops a framework for a class of constant modulus (CM) optimization problems, which covers binary constraints, discrete phase constraints, semi-orthogonal matrix constraints, non-negative semi-orthogonal matrix constraints, and several types of binary assignment constraints. Capitalizing on the basic principles of concave minimization and error bounds, we study a convex-constrained penalized formulation for general CM problems. The advantage of such formulation is that it allows us to leverage non-convex optimization techniques, such as the simple projected gradient method, to build algorithms. As the first part of this study, we explore the theory of this framework. We study conditions under which the formulation provides exact penalization results. We also examine computational aspects relating to the use of the projected gradient method for each type of CM constraint. Our study suggests that the proposed framework has a broad scope of applicability.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"72 ","pages":"4541-4556"},"PeriodicalIF":4.6,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142170621","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-11DOI: 10.1109/TSP.2024.3459808
Xiaonan Liu;Tharmalingam Ratnarajah;Mathini Sellathurai;Yonina C. Eldar
Federated learning (FL) enables distributed learning across edge devices while protecting data privacy. However, the learning accuracy decreases due to the heterogeneity of devices’ data, and the computation and communication latency increase when updating large-scale learning models on devices with limited computational capability and wireless resources. We consider a FL framework with partial model pruning and personalization to overcome these challenges. This framework splits the learning model into a global part with model pruning shared with all devices to learn data representations and a personalized part to be fine-tuned for a specific device, which adapts the model size during FL. Our approach reduces both computation and communication latency and increases the learning accuracy for devices with non-independent and identically distributed data. The computation and communication latency and convergence of the proposed FL framework are mathematically analyzed. To maximize the convergence rate and guarantee learning accuracy, Karush–Kuhn–Tucker (KKT) conditions are deployed to jointly optimize the pruning ratio and bandwidth allocation. Finally, experimental results demonstrate that the proposed FL framework achieves similar learning accuracy and a remarkable reduction of approximately $50%$ computation and communication latency compared with FL with partial model personalization.
{"title":"Adaptive Model Pruning and Personalization for Federated Learning Over Wireless Networks","authors":"Xiaonan Liu;Tharmalingam Ratnarajah;Mathini Sellathurai;Yonina C. Eldar","doi":"10.1109/TSP.2024.3459808","DOIUrl":"10.1109/TSP.2024.3459808","url":null,"abstract":"Federated learning (FL) enables distributed learning across edge devices while protecting data privacy. However, the learning accuracy decreases due to the heterogeneity of devices’ data, and the computation and communication latency increase when updating large-scale learning models on devices with limited computational capability and wireless resources. We consider a FL framework with partial model pruning and personalization to overcome these challenges. This framework splits the learning model into a global part with model pruning shared with all devices to learn data representations and a personalized part to be fine-tuned for a specific device, which adapts the model size during FL. Our approach reduces both computation and communication latency and increases the learning accuracy for devices with non-independent and identically distributed data. The computation and communication latency and convergence of the proposed FL framework are mathematically analyzed. To maximize the convergence rate and guarantee learning accuracy, Karush–Kuhn–Tucker (KKT) conditions are deployed to jointly optimize the pruning ratio and bandwidth allocation. Finally, experimental results demonstrate that the proposed FL framework achieves similar learning accuracy and a remarkable reduction of approximately \u0000<inline-formula><tex-math>$50%$</tex-math></inline-formula>\u0000 computation and communication latency compared with FL with partial model personalization.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"72 ","pages":"4395-4411"},"PeriodicalIF":4.6,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142170619","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-11DOI: 10.1109/TSP.2024.3458015
Junbin Liu;Ya Liu;Wing-Kin Ma;Mingjie Shao;Anthony Man-Cho So
In the first part of this study, a convex-constrained penalized formulation was studied for a class of constant modulus (CM) problems. In particular, the error bound techniques were shown to play a vital role in providing exact penalization results. In this second part of the study, we continue our error bound analysis for the cases of partial permutation matrices, size-constrained assignment matrices and non-negative semi-orthogonal matrices. We develop new error bounds and penalized formulations for these three cases, and the new formulations possess good structures for building computationally efficient algorithms. Moreover, we provide numerical results to demonstrate our framework in a variety of applications such as the densest $k$ -subgraph problem, graph matching, size-constrained clustering, non-negative orthogonal matrix factorization and sparse fair principal component analysis.
{"title":"Extreme Point Pursuit—Part II: Further Error Bound Analysis and Applications","authors":"Junbin Liu;Ya Liu;Wing-Kin Ma;Mingjie Shao;Anthony Man-Cho So","doi":"10.1109/TSP.2024.3458015","DOIUrl":"10.1109/TSP.2024.3458015","url":null,"abstract":"In the first part of this study, a convex-constrained penalized formulation was studied for a class of constant modulus (CM) problems. In particular, the error bound techniques were shown to play a vital role in providing exact penalization results. In this second part of the study, we continue our error bound analysis for the cases of partial permutation matrices, size-constrained assignment matrices and non-negative semi-orthogonal matrices. We develop new error bounds and penalized formulations for these three cases, and the new formulations possess good structures for building computationally efficient algorithms. Moreover, we provide numerical results to demonstrate our framework in a variety of applications such as the densest \u0000<inline-formula><tex-math>$k$</tex-math></inline-formula>\u0000-subgraph problem, graph matching, size-constrained clustering, non-negative orthogonal matrix factorization and sparse fair principal component analysis.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"72 ","pages":"4557-4572"},"PeriodicalIF":4.6,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142170620","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：