Pub Date : 2024-02-01DOI: 10.1109/JSTSP.2024.3361154
Marc Martinez-Gost;Ana Pérez-Neira;Miguel Ángel Lagunas
The expressiveness of neural networks highly depends on the nature of the activation function, although these are usually assumed predefined and fixed during the training stage. Under a signal processing perspective, in this article we present Expressive Neural Network (ENN), a novel model in which the non-linear activation functions are modeled using the Discrete Cosine Transform (DCT) and adapted using backpropagation during training. This parametrization keeps the number of trainable parameters low, is appropriate for gradient-based schemes, and adapts to different learning tasks. This is the first non-linear model for activation functions that relies on a signal processing perspective, providing high flexibility and expressiveness to the network. We contribute with insights in the explainability of the network at convergence by recovering the concept of bump, this is, the response of each activation function in the output space. Finally, through exhaustive experiments we show that the model can adapt to classification and regression tasks. The performance of ENN outperforms state of the art benchmarks, providing above a 40% gap in accuracy in some scenarios.
{"title":"ENN: A Neural Network With DCT Adaptive Activation Functions","authors":"Marc Martinez-Gost;Ana Pérez-Neira;Miguel Ángel Lagunas","doi":"10.1109/JSTSP.2024.3361154","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3361154","url":null,"abstract":"The expressiveness of neural networks highly depends on the nature of the activation function, although these are usually assumed predefined and fixed during the training stage. Under a signal processing perspective, in this article we present Expressive Neural Network (ENN), a novel model in which the non-linear activation functions are modeled using the Discrete Cosine Transform (DCT) and adapted using backpropagation during training. This parametrization keeps the number of trainable parameters low, is appropriate for gradient-based schemes, and adapts to different learning tasks. This is the first non-linear model for activation functions that relies on a signal processing perspective, providing high flexibility and expressiveness to the network. We contribute with insights in the explainability of the network at convergence by recovering the concept of bump, this is, the response of each activation function in the output space. Finally, through exhaustive experiments we show that the model can adapt to classification and regression tasks. The performance of ENN outperforms state of the art benchmarks, providing above a 40% gap in accuracy in some scenarios.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 2","pages":"232-241"},"PeriodicalIF":8.7,"publicationDate":"2024-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10418453","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141500178","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In order to realize advanced system design for the sophisticated mobile networks, predictive digital twin (DT) channel is constructed via data-driven approaches to provide high-accuracy channel prediction. However, lacking sufficient time-series datasets leads to overfitting, which degrades the prediction accuracy of the DT channel. In this article, data augmentation is investigated for constructing the predictive DT channel, while enhancing its capability of tackling channel aging problem. The feature space needs to be learned by guaranteeing that the synthetic datasets have the same channel coefficient distribution and time-frequency-space domain correlations as the original ones. Therefore, convolutional time-series generative adversarial network (TimeGAN) is proposed to capture the intrinsic features of the original datasets and then generate synthetic samples. Specifically, the embedding network and recovery network provide a latent space by reducing the dimensions of the original channel datasets, while adversarial learning operates in this space via sequence generator and sequence discriminator. Simulation results demonstrate that the synthetic dataset has the same channel coefficient distribution and multi-domain correlations as the original one. Moreover, the proposed data augmentation scheme effectively improves the prediction accuracy of the DT channel in a dynamic wireless environment, thereby increasing the achievable spectral efficiency in an aging channel.
{"title":"Data Augmentation for Predictive Digital Twin Channel: Learning Multi-Domain Correlations by Convolutional TimeGAN","authors":"Guangming Liang;Jie Hu;Kun Yang;Siyao Song;Tingcai Liu;Ning Xie;Yijun Yu","doi":"10.1109/JSTSP.2024.3358980","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3358980","url":null,"abstract":"In order to realize advanced system design for the sophisticated mobile networks, predictive digital twin (DT) channel is constructed via data-driven approaches to provide high-accuracy channel prediction. However, lacking sufficient time-series datasets leads to overfitting, which degrades the prediction accuracy of the DT channel. In this article, data augmentation is investigated for constructing the predictive DT channel, while enhancing its capability of tackling channel aging problem. The feature space needs to be learned by guaranteeing that the synthetic datasets have the same channel coefficient distribution and time-frequency-space domain correlations as the original ones. Therefore, convolutional time-series generative adversarial network (TimeGAN) is proposed to capture the intrinsic features of the original datasets and then generate synthetic samples. Specifically, the embedding network and recovery network provide a latent space by reducing the dimensions of the original channel datasets, while adversarial learning operates in this space via sequence generator and sequence discriminator. Simulation results demonstrate that the synthetic dataset has the same channel coefficient distribution and multi-domain correlations as the original one. Moreover, the proposed data augmentation scheme effectively improves the prediction accuracy of the DT channel in a dynamic wireless environment, thereby increasing the achievable spectral efficiency in an aging channel.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 1","pages":"18-33"},"PeriodicalIF":7.5,"publicationDate":"2024-01-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140641635","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-26DOI: 10.1109/JSTSP.2024.3359009
Zhixiong Chen;Wenqiang Yi;Arumugam Nallanathan;Jonathon A. Chambers
At the network edges, the multi-tier computing framework provides mobile users with efficient cloud-like computing and signal processing capabilities. Deploying digital twins in the multi-tier computing system helps to realize ultra-reliable and low-latency interactions between users and their virtual objects. Considering users in the system may roam between edge servers with limited coverage and increase the data synchronization latency to their digital twins, it is crucial to address the digital twin migration problem to enable real-time synchronization between digital twins and users. To this end, we formulate a joint digital twin migration, communication and computation resource management problem to minimize the data synchronization latency, where the time-varying network states and user mobility are considered. By decoupling edge servers under a deterministic migration strategy, we first derive the optimal communication and computation resource management policies at each server using convex optimization methods. For the digital twin migration problem between different servers, we transform it as a decentralized partially observable Markov decision process (Dec-POMDP). To solve this problem, we propose a novel agent-contribution-enabled multi-agent reinforcement learning (AC-MARL) algorithm to enable distributed digital twin migration for users, in which the counterfactual baseline method is adopted to characterize the contribution of each agent and facilitate cooperation among agents. In addition, we utilize embedding matrices to code agents' actions and states to release the scalability issue under the high dimensional state in AC-MARL. Simulation results based on two real-world taxi mobility trace datasets show that the proposed digital twin migration scheme is able to reduce 23%–30% data synchronization latency for users compared to the benchmark schemes.
{"title":"Distributed Digital Twin Migration in Multi-Tier Computing Systems","authors":"Zhixiong Chen;Wenqiang Yi;Arumugam Nallanathan;Jonathon A. Chambers","doi":"10.1109/JSTSP.2024.3359009","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3359009","url":null,"abstract":"At the network edges, the multi-tier computing framework provides mobile users with efficient cloud-like computing and signal processing capabilities. Deploying digital twins in the multi-tier computing system helps to realize ultra-reliable and low-latency interactions between users and their virtual objects. Considering users in the system may roam between edge servers with limited coverage and increase the data synchronization latency to their digital twins, it is crucial to address the digital twin migration problem to enable real-time synchronization between digital twins and users. To this end, we formulate a joint digital twin migration, communication and computation resource management problem to minimize the data synchronization latency, where the time-varying network states and user mobility are considered. By decoupling edge servers under a deterministic migration strategy, we first derive the optimal communication and computation resource management policies at each server using convex optimization methods. For the digital twin migration problem between different servers, we transform it as a decentralized partially observable Markov decision process (Dec-POMDP). To solve this problem, we propose a novel agent-contribution-enabled multi-agent reinforcement learning (AC-MARL) algorithm to enable distributed digital twin migration for users, in which the counterfactual baseline method is adopted to characterize the contribution of each agent and facilitate cooperation among agents. In addition, we utilize embedding matrices to code agents' actions and states to release the scalability issue under the high dimensional state in AC-MARL. Simulation results based on two real-world taxi mobility trace datasets show that the proposed digital twin migration scheme is able to reduce 23%–30% data synchronization latency for users compared to the benchmark schemes.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 1","pages":"109-123"},"PeriodicalIF":7.5,"publicationDate":"2024-01-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140641641","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
6G has emerged as a feasible solution to enable intelligent electric vehicle (EV) energy management. It can be further combined with digital twin (DT) to optimize resource management under unobservable information. However, the lack of reliable information timeliness guarantee increases DT inconsistency and undermines resource management optimality. To address this challenge, we investigate DT-empowered resource management from the perspective of age of information (AoI) optimization. We utilize AoI as an effective information timeliness metric to measure DT consistency, and construct an AoI-optimal DT (AoIo-DT) to assist resource management by providing more accurate state estimates. A joint optimization algorithm of signal processing, communication, and computing integration based on AoI-aware deep actor critic (DAC) with DT assistance is proposed to achieve balanced tradeoff between DT consistency and precision improvement of EV energy management. It further improves learning convergence and optimality of DAC by enforcing training with data samples of smaller AoI. Numerical results verify its performance gain in AoI minimization and EV energy management optimization.
{"title":"Integration of 6G Signal Processing, Communication, and Computing Based on Information Timeliness-Aware Digital Twin","authors":"Haijun Liao;Jiaxuan Lu;Yiling Shu;Zhenyu Zhou;Muhammad Tariq;Shahid Mumtaz","doi":"10.1109/JSTSP.2023.3341353","DOIUrl":"https://doi.org/10.1109/JSTSP.2023.3341353","url":null,"abstract":"6G has emerged as a feasible solution to enable intelligent electric vehicle (EV) energy management. It can be further combined with digital twin (DT) to optimize resource management under unobservable information. However, the lack of reliable information timeliness guarantee increases DT inconsistency and undermines resource management optimality. To address this challenge, we investigate DT-empowered resource management from the perspective of age of information (AoI) optimization. We utilize AoI as an effective information timeliness metric to measure DT consistency, and construct an AoI-optimal DT (AoIo-DT) to assist resource management by providing more accurate state estimates. A joint optimization algorithm of signal processing, communication, and computing integration based on AoI-aware deep actor critic (DAC) with DT assistance is proposed to achieve balanced tradeoff between DT consistency and precision improvement of EV energy management. It further improves learning convergence and optimality of DAC by enforcing training with data samples of smaller AoI. Numerical results verify its performance gain in AoI minimization and EV energy management optimization.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 1","pages":"98-108"},"PeriodicalIF":7.5,"publicationDate":"2024-01-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140641644","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-10DOI: 10.1109/JSTSP.2024.3350508
O. Deniz Kose;Gonzalo Mateos;Yanning Shen
Graphs are mathematical tools that can be used to represent complex real-world interconnected systems, such as financial markets and social networks. Hence, machine learning (ML) over graphs has attracted significant attention recently. However, it has been demonstrated that ML over graphs amplifies the already existing bias towards certain under-represented groups in various decision-making problems due to the information aggregation over biased graph structures. Faced with this challenge, here we take a fresh look at the problem of bias mitigation in graph-based learning by borrowing insights from graph signal processing. Our idea is to introduce predesigned graph filters within an ML pipeline to reduce a novel unsupervised bias measure, namely the correlation between sensitive attributes and the underlying graph connectivity. We show that the optimal design of said filters can be cast as a convex problem in the graph spectral domain. We also formulate a linear programming (LP) problem informed by a theoretical bias analysis, which attains a closed-form solution and leads to a more efficient fairness-aware graph filter. Finally, for a design whose degrees of freedom are independent of the input graph size, we minimize the bias metric over the family of polynomial graph convolutional filters. Our optimal filter designs offer complementary strengths to explore favorable fairness-utility-complexity tradeoffs. For performance evaluation, we conduct extensive and reproducible node classification experiments over real-world networks. Our results show that the proposed framework leads to better fairness measures together with similar utility compared to state-of-the-art fairness-aware baselines.
{"title":"Fairness-Aware Optimal Graph Filter Design","authors":"O. Deniz Kose;Gonzalo Mateos;Yanning Shen","doi":"10.1109/JSTSP.2024.3350508","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3350508","url":null,"abstract":"Graphs are mathematical tools that can be used to represent complex real-world interconnected systems, such as financial markets and social networks. Hence, machine learning (ML) over graphs has attracted significant attention recently. However, it has been demonstrated that ML over graphs amplifies the already existing bias towards certain under-represented groups in various decision-making problems due to the information aggregation over biased graph structures. Faced with this challenge, here we take a fresh look at the problem of bias mitigation in graph-based learning by borrowing insights from graph signal processing. Our idea is to introduce predesigned graph filters within an ML pipeline to reduce a novel unsupervised bias measure, namely the correlation between sensitive attributes and the underlying graph connectivity. We show that the optimal design of said filters can be cast as a convex problem in the graph spectral domain. We also formulate a linear programming (LP) problem informed by a theoretical bias analysis, which attains a closed-form solution and leads to a more efficient fairness-aware graph filter. Finally, for a design whose degrees of freedom are independent of the input graph size, we minimize the bias metric over the family of polynomial graph convolutional filters. Our optimal filter designs offer complementary strengths to explore favorable fairness-utility-complexity tradeoffs. For performance evaluation, we conduct extensive and reproducible node classification experiments over real-world networks. Our results show that the proposed framework leads to better fairness measures together with similar utility compared to state-of-the-art fairness-aware baselines.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 2","pages":"142-154"},"PeriodicalIF":8.7,"publicationDate":"2024-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141500314","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-01DOI: 10.1109/JSTSP.2024.3365415
{"title":"IEEE Signal Processing Society Information","authors":"","doi":"10.1109/JSTSP.2024.3365415","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3365415","url":null,"abstract":"","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 1","pages":"C3-C3"},"PeriodicalIF":7.5,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10507871","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140641610","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-01DOI: 10.1109/JSTSP.2024.3365411
{"title":"IEEE Signal Processing Society Information","authors":"","doi":"10.1109/JSTSP.2024.3365411","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3365411","url":null,"abstract":"","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 1","pages":"C2-C2"},"PeriodicalIF":7.5,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10507870","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140641642","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-01DOI: 10.1109/JSTSP.2024.3369289
Kunlun Wang;Trung Q. Duong;Saeed R. Khosravirad;Octavia A. Dobre;George K. Karagiannidis
Digital twin (DT) has become a game-changing tech- nology in many smart applications, including smart cities, manufacturing, automotive, gaming, entertainment and climate resilience. DTs help push the boundaries of system reliability and are used to support a wide range of func- tions such as diagnostics and fault prediction. Keeping DT up-to-date requires communication means with low latency, high reliability, and high data security protection. The digital virtual twins of physical systems are then used to optimize performance of the system in real time, and one example for such systems is the sixth-generation (6G) wireless networks. There are many challenges in representing a physical system virtually, such as true reflection