End-edge-cloud (EEC) collaborative computing is regarded as one of the most promising technologies for the Industrial Internet of Things (IIoT). It offers effective solutions for managing computationally intensive and delay-sensitive tasks efficiently. Indeed, achieving intelligent manufacturing in the context of 6G networks requires the development of efficient resource scheduling schemes. However, improving the quality of service and resource management in the face of challenges like time-varying physical operating environments of IIoT, task heterogeneity, and the coupling of different resource types is undoubtedly a complex task. In this work, we propose a digital twin (DT) assisted EEC collaborative computing scheme, where DT is utilized to monitor the physical operating environment in real-time and determine the optimal strategy, and the potential deviation between the real values and DT estimates is also considered. We aim to minimize the system cost by optimizing device association, offloading mode, bandwidth allocation, and task split ratio. Our optimization is constrained by the maximum tolerable latency of the task while considering both latency and energy consumption. To solve the collaborative computation and resource allocation (CCRA) problem in the EEC, we propose an algorithm with DT based on Multi-Agent Deep Deterministic Policy Gradient (MADDPG), where each user end (UE) in DT operates as an independent agent to determine the optimum offloading decision autonomously. Simulation results demonstrate the effectiveness of the proposed scheme, which can significantly improve the task success rate compared to benchmark schemes, while reducing the latency and energy consumption of task offloading with the assistance of DT.
{"title":"Cooperative End-Edge-Cloud Computing and Resource Allocation for Digital Twin Enabled 6G Industrial IoT","authors":"Yuao Wang;Jingjing Fang;Yao Cheng;Hao She;Yongan Guo;Gan Zheng","doi":"10.1109/JSTSP.2023.3345154","DOIUrl":"https://doi.org/10.1109/JSTSP.2023.3345154","url":null,"abstract":"End-edge-cloud (EEC) collaborative computing is regarded as one of the most promising technologies for the Industrial Internet of Things (IIoT). It offers effective solutions for managing computationally intensive and delay-sensitive tasks efficiently. Indeed, achieving intelligent manufacturing in the context of 6G networks requires the development of efficient resource scheduling schemes. However, improving the quality of service and resource management in the face of challenges like time-varying physical operating environments of IIoT, task heterogeneity, and the coupling of different resource types is undoubtedly a complex task. In this work, we propose a digital twin (DT) assisted EEC collaborative computing scheme, where DT is utilized to monitor the physical operating environment in real-time and determine the optimal strategy, and the potential deviation between the real values and DT estimates is also considered. We aim to minimize the system cost by optimizing device association, offloading mode, bandwidth allocation, and task split ratio. Our optimization is constrained by the maximum tolerable latency of the task while considering both latency and energy consumption. To solve the collaborative computation and resource allocation (CCRA) problem in the EEC, we propose an algorithm with DT based on Multi-Agent Deep Deterministic Policy Gradient (MADDPG), where each user end (UE) in DT operates as an independent agent to determine the optimum offloading decision autonomously. Simulation results demonstrate the effectiveness of the proposed scheme, which can significantly improve the task success rate compared to benchmark schemes, while reducing the latency and energy consumption of task offloading with the assistance of DT.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 1","pages":"124-137"},"PeriodicalIF":7.5,"publicationDate":"2023-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140641617","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Multicast short video streaming (MSVS) can effectively reduce network traffic load by delivering identical video sequences to multiple users simultaneously. The existing MSVS schemes mainly rely on the aggregated video requests to reserve bandwidth and computing resources, which cannot satisfy users' diverse and dynamic service requirements, particularly when users' swipe behaviors exhibit spatiotemporal fluctuation. In this article, we propose a user-centric resource management scheme based on the digital twin (DT) technique, which aims to enhance user satisfaction as well as reduce resource consumption. Firstly, we design a user DT (UDT)-assisted resource reservation framework. Specifically, UDTs are constructed for individual users, which store users' historical data for updating multicast groups and abstracting useful information. The swipe probability distributions and recommended video lists are abstracted from UDTs to predict bandwidth and computing resource demands. Parameterized sigmoid functions are leveraged to characterize multicast groups' user satisfaction. Secondly, we formulate a joint non-convex bandwidth and computing resource reservation problem which is transformed into a convex piecewise problem by utilizing a tangent function to approximately substitute the concave part. A low-complexity scheduling algorithm is then developed to find the optimal resource reservation decisions. Simulation results based on the real-world dataset demonstrate that the proposed scheme outperforms benchmark schemes in terms of user satisfaction and resource consumption.
{"title":"Digital Twin Based User-Centric Resource Management for Multicast Short Video Streaming","authors":"Xinyu Huang;Wen Wu;Shisheng Hu;Mushu Li;Conghao Zhou;Xuemin Shen","doi":"10.1109/JSTSP.2023.3343626","DOIUrl":"https://doi.org/10.1109/JSTSP.2023.3343626","url":null,"abstract":"Multicast short video streaming (MSVS) can effectively reduce network traffic load by delivering identical video sequences to multiple users simultaneously. The existing MSVS schemes mainly rely on the aggregated video requests to reserve bandwidth and computing resources, which cannot satisfy users' diverse and dynamic service requirements, particularly when users' swipe behaviors exhibit spatiotemporal fluctuation. In this article, we propose a user-centric resource management scheme based on the digital twin (DT) technique, which aims to enhance user satisfaction as well as reduce resource consumption. Firstly, we design a user DT (UDT)-assisted resource reservation framework. Specifically, UDTs are constructed for individual users, which store users' historical data for updating multicast groups and abstracting useful information. The swipe probability distributions and recommended video lists are abstracted from UDTs to predict bandwidth and computing resource demands. Parameterized sigmoid functions are leveraged to characterize multicast groups' user satisfaction. Secondly, we formulate a joint non-convex bandwidth and computing resource reservation problem which is transformed into a convex piecewise problem by utilizing a tangent function to approximately substitute the concave part. A low-complexity scheduling algorithm is then developed to find the optimal resource reservation decisions. Simulation results based on the real-world dataset demonstrate that the proposed scheme outperforms benchmark schemes in terms of user satisfaction and resource consumption.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 1","pages":"50-65"},"PeriodicalIF":7.5,"publicationDate":"2023-12-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140641593","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-06DOI: 10.1109/JSTSP.2023.3340107
Anal Paul;Keshav Singh;Minh-Hien T. Nguyen;Cunhua Pan;Chih-Peng Li
In this paper, we present a framework that integrates digital twin (DT) technology into space-air-ground integrated networks (SAGINs) to enhance vehicular edge computing (VEC). Our objective is to efficiently offload tasks in ultra-reliable low-latency communications (URLLC)-enabled vehicular networks, focusing on minimizing overall latency for requested tasks by reducing transmission time for task offloading and edge processing requirements. The proposed framework leverages DT-assisted SAGINs to minimize task offloading latency, expand network coverage, and reduce energy consumption. Key components of our framework include partial task offloading, distributed edge computing, latency modeling, energy consumption analysis, mobility, and channel modeling. We formulate a non-convex optimization problem considering various network constraints to achieve the system objective. To solve this optimization problem, we develop a novel multi-agent deep reinforcement learning (DRL) algorithm, enabling intelligent decision-making by individual agents. Through extensive simulations, we validate the effectiveness of our proposed system in advancing VEC by integrating DT technology into SAGINs.
{"title":"Digital Twin-Assisted Space-Air-Ground Integrated Networks for Vehicular Edge Computing","authors":"Anal Paul;Keshav Singh;Minh-Hien T. Nguyen;Cunhua Pan;Chih-Peng Li","doi":"10.1109/JSTSP.2023.3340107","DOIUrl":"https://doi.org/10.1109/JSTSP.2023.3340107","url":null,"abstract":"In this paper, we present a framework that integrates digital twin (DT) technology into space-air-ground integrated networks (SAGINs) to enhance vehicular edge computing (VEC). Our objective is to efficiently offload tasks in ultra-reliable low-latency communications (URLLC)-enabled vehicular networks, focusing on minimizing overall latency for requested tasks by reducing transmission time for task offloading and edge processing requirements. The proposed framework leverages DT-assisted SAGINs to minimize task offloading latency, expand network coverage, and reduce energy consumption. Key components of our framework include partial task offloading, distributed edge computing, latency modeling, energy consumption analysis, mobility, and channel modeling. We formulate a non-convex optimization problem considering various network constraints to achieve the system objective. To solve this optimization problem, we develop a novel multi-agent deep reinforcement learning (DRL) algorithm, enabling intelligent decision-making by individual agents. Through extensive simulations, we validate the effectiveness of our proposed system in advancing VEC by integrating DT technology into SAGINs.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 1","pages":"66-82"},"PeriodicalIF":7.5,"publicationDate":"2023-12-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140641643","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-16DOI: 10.1109/JSTSP.2023.3333552
Jun Ling;Xu Tan;Liyang Chen;Runnan Li;Yuchao Zhang;Sheng Zhao;Li Song
While previous methods for speech-driven talking face generation have shown significant advances in improving the visual and lip-sync quality of the synthesized videos, they have paid less attention to lip motion jitters which can substantially undermine the perceived quality of talking face videos. What causes motion jitters, and how to mitigate the problem? In this article, we conduct systematic analyses to investigate the motion jittering problem based on a state-of-the-art pipeline that utilizes 3D face representations to bridge the input audio and output video, and implement several effective designs to improve motion stability. This study finds that several factors can lead to jitters in the synthesized talking face video, including jitters from the input face representations, training-inference mismatch, and a lack of dependency modeling in the generation network. Accordingly, we propose three effective solutions: 1) a Gaussian-based adaptive smoothing module to smooth the 3D face representations to eliminate jitters in the input; 2) augmented erosions added to the input data of the neural renderer in training to simulate the inference distortion to reduce mismatch; 3) an audio-fused transformer generator to model inter-frame dependency. In addition, considering there is no off-the-shelf metric that can measures motion jitters of talking face video, we devise an objective metric (Motion Stability Index, MSI) to quantitatively measure the motion jitters. Extensive experimental results show the superiority of the proposed method on motion-stable talking video generation, with superior quality to previous systems.
{"title":"StableFace: Analyzing and Improving Motion Stability for Talking Face Generation","authors":"Jun Ling;Xu Tan;Liyang Chen;Runnan Li;Yuchao Zhang;Sheng Zhao;Li Song","doi":"10.1109/JSTSP.2023.3333552","DOIUrl":"https://doi.org/10.1109/JSTSP.2023.3333552","url":null,"abstract":"While previous methods for speech-driven talking face generation have shown significant advances in improving the visual and lip-sync quality of the synthesized videos, they have paid less attention to lip motion jitters which can substantially undermine the perceived quality of talking face videos. What causes motion jitters, and how to mitigate the problem? In this article, we conduct systematic analyses to investigate the motion jittering problem based on a state-of-the-art pipeline that utilizes 3D face representations to bridge the input audio and output video, and implement several effective designs to improve motion stability. This study finds that several factors can lead to jitters in the synthesized talking face video, including jitters from the input face representations, training-inference mismatch, and a lack of dependency modeling in the generation network. Accordingly, we propose three effective solutions: 1) a Gaussian-based adaptive smoothing module to smooth the 3D face representations to eliminate jitters in the input; 2) augmented erosions added to the input data of the neural renderer in training to simulate the inference distortion to reduce mismatch; 3) an audio-fused transformer generator to model inter-frame dependency. In addition, considering there is no off-the-shelf metric that can measures motion jitters of talking face video, we devise an objective metric (Motion Stability Index, MSI) to quantitatively measure the motion jitters. Extensive experimental results show the superiority of the proposed method on motion-stable talking video generation, with superior quality to previous systems.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"17 6","pages":"1232-1247"},"PeriodicalIF":7.5,"publicationDate":"2023-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139081210","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-13DOI: 10.1109/JSTSP.2023.3332455
Yapeng Zhao;Qingqing Wu;Guangji Chen;Wen Chen;Ruiqi Liu;Ming-Min Zhao;Yuan Wu;Shaodan Ma
The digital twin edge network (DITEN) aims to integrate mobile edge computing (MEC) and digital twin (DT) to provide real-time system configuration and flexible resource allocation for the sixth-generation network. This paper investigates an intelligent reflecting surface (IRS)-aided multi-tier hybrid computing system that can achieve mutual benefits for DT and MEC in the DITEN. For the first time, this paper presents the opportunity to realize the network-wide convergence of DT and MEC. In the considered system, specifically, over-the-air computation (AirComp) is employed to monitor the status of the DT system, while MEC is performed with the assistance of DT to provide low-latency computing services. Besides, the IRS is utilized to enhance signal transmission and mitigate interference among heterogeneous nodes. We propose a framework for designing the hybrid computing system, aiming to maximize the sum computation rate under communication and computation resources constraints. To tackle the non-convex optimization problem, alternative optimization and successive convex approximation techniques are leveraged to decouple variables and then transform the problem into a more tractable form. Simulation results verify the effectiveness of the proposed algorithm and demonstrate the IRS can significantly improve the system performance with appropriate phase shift configurations. Moreover, the results indicate that the DT assisted MEC system can precisely achieve the balance between local computing and task offloading since real-time system status can be obtained with the help of DT.
{"title":"Intelligent Reflecting Surface Aided Multi-Tier Hybrid Computing","authors":"Yapeng Zhao;Qingqing Wu;Guangji Chen;Wen Chen;Ruiqi Liu;Ming-Min Zhao;Yuan Wu;Shaodan Ma","doi":"10.1109/JSTSP.2023.3332455","DOIUrl":"10.1109/JSTSP.2023.3332455","url":null,"abstract":"The digital twin edge network (DITEN) aims to integrate mobile edge computing (MEC) and digital twin (DT) to provide real-time system configuration and flexible resource allocation for the sixth-generation network. This paper investigates an intelligent reflecting surface (IRS)-aided multi-tier hybrid computing system that can achieve mutual benefits for DT and MEC in the DITEN. For the first time, this paper presents the opportunity to realize the network-wide convergence of DT and MEC. In the considered system, specifically, over-the-air computation (AirComp) is employed to monitor the status of the DT system, while MEC is performed with the assistance of DT to provide low-latency computing services. Besides, the IRS is utilized to enhance signal transmission and mitigate interference among heterogeneous nodes. We propose a framework for designing the hybrid computing system, aiming to maximize the sum computation rate under communication and computation resources constraints. To tackle the non-convex optimization problem, alternative optimization and successive convex approximation techniques are leveraged to decouple variables and then transform the problem into a more tractable form. Simulation results verify the effectiveness of the proposed algorithm and demonstrate the IRS can significantly improve the system performance with appropriate phase shift configurations. Moreover, the results indicate that the DT assisted MEC system can precisely achieve the balance between local computing and task offloading since real-time system status can be obtained with the help of DT.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 1","pages":"83-97"},"PeriodicalIF":7.5,"publicationDate":"2023-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135613321","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-01DOI: 10.1109/JSTSP.2023.3340490
{"title":"List of Reviewers","authors":"","doi":"10.1109/JSTSP.2023.3340490","DOIUrl":"https://doi.org/10.1109/JSTSP.2023.3340490","url":null,"abstract":"","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"17 6","pages":"1277-1280"},"PeriodicalIF":7.5,"publicationDate":"2023-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10378866","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139081209","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-01DOI: 10.1109/JSTSP.2023.3324780
{"title":"IEEE Signal Processing Society Information","authors":"","doi":"10.1109/JSTSP.2023.3324780","DOIUrl":"https://doi.org/10.1109/JSTSP.2023.3324780","url":null,"abstract":"","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"17 6","pages":"C3-C3"},"PeriodicalIF":7.5,"publicationDate":"2023-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10378863","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139081241","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-01DOI: 10.1109/JSTSP.2024.3350358
{"title":"2023 Index IEEE Journal of Selected Topics in Signal Processing Vol. 17","authors":"","doi":"10.1109/JSTSP.2024.3350358","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3350358","url":null,"abstract":"","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"17 6","pages":"1281-1299_lpage_"},"PeriodicalIF":7.5,"publicationDate":"2023-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10381590","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139109602","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}