Pub Date : 2024-06-19DOI: 10.1109/JSTSP.2024.3416681
Kangjian Chen;Chenhao Qi;Geoffrey Ye Li;Octavia A. Dobre
This paper considers near-field multiuser communications based on sparse arrays (SAs). First, for the uniform SAs (USAs), we analyze the beam gains of channel steering vectors, which shows that increasing the antenna spacings can effectively improve the spatial resolution of the antenna arrays to enhance the sum rate of multiuser communications. Then, we investigate nonuniform SAs (NSAs) to mitigate the high multiuser interference from the grating lobes of the USAs. To maximize the sum rate of near-field multiuser communications, we optimize the antenna positions of the NSAs, where a successive convex approximation-based antenna position optimization algorithm is proposed. Moreover, we find that the channels of both the USAs and the NSAs show uniform sparsity in the defined surrogate distance-angle (SD-A) domain. Based on the channel sparsity, an on-grid SD-A-domain orthogonal matching pursuit (SDA-OMP) algorithm is developed to estimate multiuser channels. To further improve the resolution of the SDA-OMP, we also design an off-grid SD-A-domain iterative super-resolution channel estimation algorithm. Simulation results demonstrate the superior performance of the proposed methods.
{"title":"Near-Field Multiuser Communications Based on Sparse Arrays","authors":"Kangjian Chen;Chenhao Qi;Geoffrey Ye Li;Octavia A. Dobre","doi":"10.1109/JSTSP.2024.3416681","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3416681","url":null,"abstract":"This paper considers near-field multiuser communications based on sparse arrays (SAs). First, for the uniform SAs (USAs), we analyze the beam gains of channel steering vectors, which shows that increasing the antenna spacings can effectively improve the spatial resolution of the antenna arrays to enhance the sum rate of multiuser communications. Then, we investigate nonuniform SAs (NSAs) to mitigate the high multiuser interference from the grating lobes of the USAs. To maximize the sum rate of near-field multiuser communications, we optimize the antenna positions of the NSAs, where a successive convex approximation-based antenna position optimization algorithm is proposed. Moreover, we find that the channels of both the USAs and the NSAs show uniform sparsity in the defined surrogate distance-angle (SD-A) domain. Based on the channel sparsity, an on-grid SD-A-domain orthogonal matching pursuit (SDA-OMP) algorithm is developed to estimate multiuser channels. To further improve the resolution of the SDA-OMP, we also design an off-grid SD-A-domain iterative super-resolution channel estimation algorithm. Simulation results demonstrate the superior performance of the proposed methods.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 4","pages":"619-632"},"PeriodicalIF":8.7,"publicationDate":"2024-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142587511","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Despite the current surge of interest in autonomous robotic systems, robot activity recognition within restricted indoor environments remains a formidable challenge. Conventional methods for detecting and recognizing robotic arms' activities often rely on vision-based or light detection and ranging (LiDAR) sensors, which require line-of-sight (LoS) access and may raise privacy concerns, for example, in nursing facilities. This research pioneers an innovative approach harnessing channel state information (CSI) measured from WiFi signals, subtly influenced by the activity of robotic arms. We developed an attention-based network to classify eight distinct activities performed by a Franka Emika robotic arm in different situations. Our proposed bidirectional vision transformer-concatenated (BiVTC) methodology aspires to predict robotic arm activities accurately, even when trained on activities with different velocities, all without dependency on external or internal sensors or visual aids. Considering the high dependency of CSI data on the environment motivated us study the problem of sniffer location selection, by systematically changing the sniffer's location and collecting different sets of data. Finally, this paper also marks the first publication of the CSI data of eight distinct robotic arm activities, collectively referred to as RoboFiSense. This initiative aims to provide a benchmark dataset and baselines to the research community, fostering advancements in the field of robotics sensing.
{"title":"RoboFiSense: Attention-Based Robotic Arm Activity Recognition With WiFi Sensing","authors":"Rojin Zandi;Kian Behzad;Elaheh Motamedi;Hojjat Salehinejad;Milad Siami","doi":"10.1109/JSTSP.2024.3416851","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3416851","url":null,"abstract":"Despite the current surge of interest in autonomous robotic systems, robot activity recognition within restricted indoor environments remains a formidable challenge. Conventional methods for detecting and recognizing robotic arms' activities often rely on vision-based or light detection and ranging (LiDAR) sensors, which require line-of-sight (LoS) access and may raise privacy concerns, for example, in nursing facilities. This research pioneers an innovative approach harnessing channel state information (CSI) measured from WiFi signals, subtly influenced by the activity of robotic arms. We developed an attention-based network to classify eight distinct activities performed by a Franka Emika robotic arm in different situations. Our proposed bidirectional vision transformer-concatenated (BiVTC) methodology aspires to predict robotic arm activities accurately, even when trained on activities with different velocities, all without dependency on external or internal sensors or visual aids. Considering the high dependency of CSI data on the environment motivated us study the problem of sniffer location selection, by systematically changing the sniffer's location and collecting different sets of data. Finally, this paper also marks the first publication of the CSI data of eight distinct robotic arm activities, collectively referred to as RoboFiSense. This initiative aims to provide a benchmark dataset and baselines to the research community, fostering advancements in the field of robotics sensing.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 3","pages":"396-406"},"PeriodicalIF":8.7,"publicationDate":"2024-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137541","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In the context of single-base station (BS) non-line-of-sight (NLoS) single-epoch localization with the aid of a reflective reconfigurable intelligent surface (RIS), this paper introduces a novel three-step algorithm that jointly estimates the position and velocity of a mobile user equipment (UE), while compensating for the Doppler effects observed in near-field (NF) at the RIS elements over the short transmission duration of a sequence of downlink (DL) pilot symbols. First, a low-complexity initialization procedure is proposed, relying in part on far-field (FF) approximation and a static user assumption. Then, an alternating optimization procedure is designed to iteratively refine the velocity and position estimates, as well as the channel gain. The refinement routines leverage small angle approximations and the linearization of the RIS response, accounting for both NF and mobility effects. We evaluate the performance of the proposed algorithm through extensive simulations under diverse operating conditions with regard to signal-to-noise ratio (SNR), UE mobility, uncontrolled multipath and RIS-UE distance. Our results reveal remarkable performance improvements over the state-of-the-art (SoTA) mobility-agnostic benchmark algorithm, while indicating convergence of the proposed algorithm to respective theoretical bounds on position and velocity estimation.
{"title":"RIS-Enabled NLoS Near-Field Joint Position and Velocity Estimation Under User Mobility","authors":"Moustafa Rahal;Benoit Denis;Musa Furkan Keskin;Bernard Uguen;Henk Wymeersch","doi":"10.1109/JSTSP.2024.3414110","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3414110","url":null,"abstract":"In the context of single-base station (BS) non-line-of-sight (NLoS) single-epoch localization with the aid of a reflective reconfigurable intelligent surface (RIS), this paper introduces a novel three-step algorithm that jointly estimates the position and velocity of a mobile user equipment (UE), while compensating for the Doppler effects observed in near-field (NF) at the RIS elements over the short transmission duration of a sequence of downlink (DL) pilot symbols. First, a low-complexity initialization procedure is proposed, relying in part on far-field (FF) approximation and a static user assumption. Then, an alternating optimization procedure is designed to iteratively refine the velocity and position estimates, as well as the channel gain. The refinement routines leverage small angle approximations and the linearization of the RIS response, accounting for both NF and mobility effects. We evaluate the performance of the proposed algorithm through extensive simulations under diverse operating conditions with regard to signal-to-noise ratio (SNR), UE mobility, uncontrolled multipath and RIS-UE distance. Our results reveal remarkable performance improvements over the state-of-the-art (SoTA) mobility-agnostic benchmark algorithm, while indicating convergence of the proposed algorithm to respective theoretical bounds on position and velocity estimation.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 4","pages":"633-645"},"PeriodicalIF":8.7,"publicationDate":"2024-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142587538","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-13DOI: 10.1109/JSTSP.2024.3414137
Wentao Yu;Hengtao He;Xianghao Yu;Shenghui Song;Jun Zhang;Ross Murch;Khaled B. Letaief
Holographic MIMO (HMIMO) is being increasingly recognized as a key enabling technology for 6G wireless systems through the deployment of an extremely large number of antennas within a compact space to fully exploit the potentials of the electromagnetic (EM) channel. Nevertheless, the benefits of HMIMO systems cannot be fully unleashed without an efficient means to estimate the high-dimensional channel, whose distribution becomes increasingly complicated due to the accessibility of the near-field region. In this paper, we address the fundamental challenge of designing a low-complexity Bayes-optimal channel estimator in near-field HMIMO systems operating in unknown EM environments. The core idea is to estimate the HMIMO channels solely based on the Stein' s score function of the received pilot signals and an estimated noise level, without relying on priors or supervision that is not feasible in practical deployment. A neural network is trained with the unsupervised denoising score matching objective to learn the parameterized score function. Meanwhile, a principal component analysis (PCA)-based algorithm is proposed to estimate the noise level leveraging the low-rank near-field spatial correlation. Building upon these techniques, we develop a Bayes-optimal score-based channel estimator for fully-digital HMIMO transceivers in a closed form. The optimal score-based estimator is also extended to hybrid analog-digital HMIMO systems by incorporating it into a low-complexity message passing algorithm. The (quasi-) Bayes-optimality of the proposed estimators is validated both in theory and by extensive simulation results. In addition to optimality, it is shown that our proposal is robust to various mismatches and can quickly adapt to dynamic EM environments in an online manner thanks to its unsupervised nature, demonstrating its potential in real-world deployment.
{"title":"Bayes-Optimal Unsupervised Learning for Channel Estimation in Near-Field Holographic MIMO","authors":"Wentao Yu;Hengtao He;Xianghao Yu;Shenghui Song;Jun Zhang;Ross Murch;Khaled B. Letaief","doi":"10.1109/JSTSP.2024.3414137","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3414137","url":null,"abstract":"Holographic MIMO (HMIMO) is being increasingly recognized as a key enabling technology for 6G wireless systems through the deployment of an extremely large number of antennas within a compact space to fully exploit the potentials of the electromagnetic (EM) channel. Nevertheless, the benefits of HMIMO systems cannot be fully unleashed without an efficient means to estimate the high-dimensional channel, whose distribution becomes increasingly complicated due to the accessibility of the near-field region. In this paper, we address the fundamental challenge of designing a \u0000<italic>low-complexity Bayes-optimal</i>\u0000 channel estimator in near-field HMIMO systems operating in \u0000<italic>unknown</i>\u0000 EM environments. The core idea is to estimate the HMIMO channels solely based on the Stein' s score function of the received pilot signals and an estimated noise level, \u0000<italic>without</i>\u0000 relying on priors or supervision that is not feasible in practical deployment. A neural network is trained with the unsupervised denoising score matching objective to learn the parameterized score function. Meanwhile, a principal component analysis (PCA)-based algorithm is proposed to estimate the noise level leveraging the low-rank near-field spatial correlation. Building upon these techniques, we develop a Bayes-optimal score-based channel estimator for fully-digital HMIMO transceivers in a closed form. The optimal score-based estimator is also extended to hybrid analog-digital HMIMO systems by incorporating it into a low-complexity message passing algorithm. The (quasi-) Bayes-optimality of the proposed estimators is validated both in theory and by extensive simulation results. In addition to optimality, it is shown that our proposal is robust to various mismatches and can quickly adapt to dynamic EM environments in an online manner thanks to its unsupervised nature, demonstrating its potential in real-world deployment.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 4","pages":"714-729"},"PeriodicalIF":8.7,"publicationDate":"2024-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10556727","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142587616","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-12DOI: 10.1109/JSTSP.2024.3386054
Boqun Zhao;Chongjun Ouyang;Yuanwei Liu;Xingqi Zhang;H. Vincent Poor
As technologies envisioned for next-generation wireless networks significantly extend the near-field region, it is of interest to reevaluate integrated sensing and communications (ISAC) with an appropriate channel model to account for the effects introduced by the near field. In this article, a near-field ISAC framework is proposed for both downlink and uplink scenarios based on such a channel model. We consider a base station equipped with a uniform planar array, and the impacts of the effective aperture and polarization of antennas are considered. For the downlink case, three distinct designs are studied: a communications-centric (C-C) design, a sensing-centric (S-C) design, and a Pareto optimal design. Regarding the uplink case, the C-C design, the S-C design and a time-sharing strategy are considered. Within each design, sensing rates (SRs) and communication rates (CRs) are derived. To gain further insights, high signal-to-noise ratio slopes and rate scaling laws concerning the number of antennas are examined. The attainable near-field SR-CR regions of ISAC and the baseline frequency-division S&C are also characterized. Numerical results reveal that, as the number of antennas in the array grows, the SRs and CRs under our model converge to finite values, while those under conventional far- and near-field models exhibit unbounded growth, highlighting the importance of precise channel modeling for near-field ISAC.
{"title":"Modeling and Analysis of Near-Field ISAC","authors":"Boqun Zhao;Chongjun Ouyang;Yuanwei Liu;Xingqi Zhang;H. Vincent Poor","doi":"10.1109/JSTSP.2024.3386054","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3386054","url":null,"abstract":"As technologies envisioned for next-generation wireless networks significantly extend the near-field region, it is of interest to reevaluate integrated sensing and communications (ISAC) with an appropriate channel model to account for the effects introduced by the near field. In this article, a near-field ISAC framework is proposed for both downlink and uplink scenarios based on such a channel model. We consider a base station equipped with a uniform planar array, and the impacts of the effective aperture and polarization of antennas are considered. For the downlink case, three distinct designs are studied: a communications-centric (C-C) design, a sensing-centric (S-C) design, and a Pareto optimal design. Regarding the uplink case, the C-C design, the S-C design and a time-sharing strategy are considered. Within each design, sensing rates (SRs) and communication rates (CRs) are derived. To gain further insights, high signal-to-noise ratio slopes and rate scaling laws concerning the number of antennas are examined. The attainable near-field SR-CR regions of ISAC and the baseline frequency-division S&C are also characterized. Numerical results reveal that, as the number of antennas in the array grows, the SRs and CRs under our model converge to finite values, while those under conventional far- and near-field models exhibit unbounded growth, highlighting the importance of precise channel modeling for near-field ISAC.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 4","pages":"678-693"},"PeriodicalIF":8.7,"publicationDate":"2024-04-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142587621","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper proposes a vision-based tactile sensor (VBTS) embedded into the soft hand palm, named PaLmTac. We adopt a distributed modality design instead of overlaying function layers. On the one hand, the problem of unrelated modality integration (texture and temperature) is solved. On the other hand, combining regional recognition can avoid mixed unrelated information. Herein, a Level-Regional Feature Extraction Network (LRFE-Net) is presented to match the modality design. We leverage feature mapping, regional convolution, and regional vectorization to construct the regional recognition mechanism, which can extract features in parallel and control fusion degrees. The level recognition mechanism balances the learning difficulty of each modality. Compared with the existing VBTSs, the PaLmTac optimizes unrelated modality integration and reduces fusion interference. This paper provides a novel idea of multimodal VBTS design and sensing mechanism, which is expected to be applied to human-computer interaction scenarios based on multimodal fusion.
{"title":"PaLmTac: A Vision-Based Tactile Sensor Leveraging Distributed-Modality Design and Modal-Matching Recognition for Soft Hand Perception","authors":"Shixin Zhang;Yiyong Yang;Jianhua Shan;Fuchun Sun;Hongxiang Xue;Bin Fang","doi":"10.1109/JSTSP.2024.3386070","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3386070","url":null,"abstract":"This paper proposes a vision-based tactile sensor (VBTS) embedded into the soft hand palm, named PaLmTac. We adopt a distributed modality design instead of overlaying function layers. On the one hand, the problem of unrelated modality integration (texture and temperature) is solved. On the other hand, combining regional recognition can avoid mixed unrelated information. Herein, a Level-Regional Feature Extraction Network (LRFE-Net) is presented to match the modality design. We leverage feature mapping, regional convolution, and regional vectorization to construct the regional recognition mechanism, which can extract features in parallel and control fusion degrees. The level recognition mechanism balances the learning difficulty of each modality. Compared with the existing VBTSs, the PaLmTac optimizes unrelated modality integration and reduces fusion interference. This paper provides a novel idea of multimodal VBTS design and sensing mechanism, which is expected to be applied to human-computer interaction scenarios based on multimodal fusion.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 3","pages":"288-298"},"PeriodicalIF":8.7,"publicationDate":"2024-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137509","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-08DOI: 10.1109/JSTSP.2024.3386056
Nisha Varghese;A. N. Rajagopalan;Zahir Ahmed Ansari
Robotic systems employed in tasks such as navigation, target tracking, security, and surveillance often use camera gimbal systems to enhance their monitoring and security capabilities. These camera gimbal systems undergo fast to-and-fro rotational motion to surveil the extended field of view (FOV). A high steering rate (rotation angle per second) of the gimbal is essential to revisit a given scene as fast as possible, which results in significant motion blur in the captured video frames. Real-time motion deblurring is essential in surveillance robots since the subsequent image-processing tasks demand immediate availability of blur-free images. Existing deep learning (DL) based motion deblurring methods either lack real-time performance due to network complexity or suffer from poor deblurring quality for large motion blurs. In this work, we propose a Gyro-guided Network for Real-time motion deblurring (GRNet) which makes effective use of existing prior information to improve deblurring without increasing the complexity of the network. The steering rate of the gimbal is taken as a prior for data generation. A contrastive learning scheme is introduced for the network to learn the amount of blur in an image by utilizing the knowledge of blur content in images during training. To the GRNet, a sharp reference image is additionally given as input to guide the deblurring process. The most relevant features from the reference image are selected using a cross-attention module. Our method works in real-time at 30 fps. As a first, we propose a Gimbal Yaw motion Real-wOrld (GYRO) dataset of infrared (IR) as well as color images with significant motion blur along with the inertial measurements of camera rotation, captured by a gimbal-based imaging setup where the gimbal undergoes rotational yaw motion. Both qualitative and quantitative evaluations on our proposed GYRO dataset, demonstrate the practical utility of our method.
{"title":"Real-Time Large-Motion Deblurring for Gimbal-Based Imaging Systems","authors":"Nisha Varghese;A. N. Rajagopalan;Zahir Ahmed Ansari","doi":"10.1109/JSTSP.2024.3386056","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3386056","url":null,"abstract":"Robotic systems employed in tasks such as navigation, target tracking, security, and surveillance often use camera gimbal systems to enhance their monitoring and security capabilities. These camera gimbal systems undergo fast to-and-fro rotational motion to surveil the extended field of view (FOV). A high steering rate (rotation angle per second) of the gimbal is essential to revisit a given scene as fast as possible, which results in significant motion blur in the captured video frames. Real-time motion deblurring is essential in surveillance robots since the subsequent image-processing tasks demand immediate availability of blur-free images. Existing deep learning (DL) based motion deblurring methods either lack real-time performance due to network complexity or suffer from poor deblurring quality for large motion blurs. In this work, we propose a Gyro-guided Network for Real-time motion deblurring (GRNet) which makes effective use of existing prior information to improve deblurring without increasing the complexity of the network. The steering rate of the gimbal is taken as a prior for data generation. A contrastive learning scheme is introduced for the network to learn the amount of blur in an image by utilizing the knowledge of blur content in images during training. To the GRNet, a sharp reference image is additionally given as input to guide the deblurring process. The most relevant features from the reference image are selected using a cross-attention module. Our method works in real-time at 30 fps. As a first, we propose a Gimbal Yaw motion Real-wOrld (GYRO) dataset of infrared (IR) as well as color images with significant motion blur along with the inertial measurements of camera rotation, captured by a gimbal-based imaging setup where the gimbal undergoes rotational yaw motion. Both qualitative and quantitative evaluations on our proposed GYRO dataset, demonstrate the practical utility of our method.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 3","pages":"346-357"},"PeriodicalIF":8.7,"publicationDate":"2024-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137573","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Distributed stochastic gradient descent (D-SGD) with gradient compression has become a popular communication-efficient solution for accelerating optimization procedures in distributed learning systems like multi-robot systems. One commonly used method for gradient compression is Top-K sparsification, which sparsifies the gradients by a fixed degree during model training. However, there has been a lack of an adaptive approach with a systematic treatment and analysis to adjust the sparsification degree to maximize the potential of the model's performance or training speed. This paper proposes a novel adaptive Top-K in Stochastic Gradient Descent framework that enables an adaptive degree of sparsification for each gradient descent step to optimize the convergence performance by balancing the trade-off between communication cost and convergence error with respect to the norm of gradients and the communication budget. Firstly, an upper bound of convergence error is derived for the adaptive sparsification scheme and the loss function. Secondly, we consider communication budget constraints and propose an optimization formulation for minimizing the deep model's convergence error under such constraints. We obtain an enhanced compression algorithm that significantly improves model accuracy under given communication budget constraints. Finally, we conduct numerical experiments on general image classification tasks using the MNIST, CIFAR-10 datasets. For the multi-robot collaboration tasks, we choose the object detection task on the PASCAL VOC dataset. The results demonstrate that the proposed adaptive Top-K algorithm in SGD achieves a significantly better convergence rate compared to state-of-the-art methods, even after considering error compensation.
{"title":"Adaptive Top-K in SGD for Communication-Efficient Distributed Learning in Multi-Robot Collaboration","authors":"Mengzhe Ruan;Guangfeng Yan;Yuanzhang Xiao;Linqi Song;Weitao Xu","doi":"10.1109/JSTSP.2024.3381373","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3381373","url":null,"abstract":"Distributed stochastic gradient descent (D-SGD) with gradient compression has become a popular communication-efficient solution for accelerating optimization procedures in distributed learning systems like multi-robot systems. One commonly used method for gradient compression is Top-K sparsification, which sparsifies the gradients by a fixed degree during model training. However, there has been a lack of an adaptive approach with a systematic treatment and analysis to adjust the sparsification degree to maximize the potential of the model's performance or training speed. This paper proposes a novel adaptive Top-K in Stochastic Gradient Descent framework that enables an adaptive degree of sparsification for each gradient descent step to optimize the convergence performance by balancing the trade-off between communication cost and convergence error with respect to the norm of gradients and the communication budget. Firstly, an upper bound of convergence error is derived for the adaptive sparsification scheme and the loss function. Secondly, we consider communication budget constraints and propose an optimization formulation for minimizing the deep model's convergence error under such constraints. We obtain an enhanced compression algorithm that significantly improves model accuracy under given communication budget constraints. Finally, we conduct numerical experiments on general image classification tasks using the MNIST, CIFAR-10 datasets. For the multi-robot collaboration tasks, we choose the object detection task on the PASCAL VOC dataset. The results demonstrate that the proposed adaptive Top-K algorithm in SGD achieves a significantly better convergence rate compared to state-of-the-art methods, even after considering error compensation.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 3","pages":"487-501"},"PeriodicalIF":8.7,"publicationDate":"2024-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137601","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-03DOI: 10.1109/JSTSP.2024.3376962
Sichun Luo;Yuxuan Yao;Haohan Zhao;Linqi Song
Accurate address resolution plays a vital role in UAV delivery systems. Existing address resolution systems heavily rely on user-provided Point of Interest (POI) information. However, such information often lacks precision, making it challenging to obtain fine-grained details for further processing. In this paper, we present an end-to-end L anguage M odel-based fine-grained A ddress R esolution framework ( LMAR ). Instead of solely relying on POI information, we introduce a language model to process the user input text information. Specifically, we start by collecting data and constructing two datasets, which are then used to fine-tune a pre-trained language model. Additionally, our pipeline incorporates pre-processing and post-processing modules to handle data processing and regularization. We combine the output of the language model with the POI information to perform a database match and derive the final outcome. To evaluate our proposed LMAR, we conduct offline and online experiments. In both offline and online testing, our proposed model achieves an overall performance of over 90% accuracy, while in the online pressure test, it achieves satisfactory performance, demonstrating its effectiveness and practicality. The proposed LMAR has passed the internal test and will be deployed into the Meituan UAV delivery system in the near future.
准确的地址解析在无人机投递系统中起着至关重要的作用。现有的地址解析系统在很大程度上依赖于用户提供的兴趣点（POI）信息。然而，这些信息往往缺乏精确性，因此要获得用于进一步处理的细粒度细节具有挑战性。在本文中，我们提出了一个基于语言模型的端到端细粒度地址解析框架（LMAR）。我们不再单纯依赖 POI 信息，而是引入了语言模型来处理用户输入的文本信息。具体来说，我们首先收集数据并构建两个数据集，然后利用这两个数据集对预先训练好的语言模型进行微调。此外，我们的管道还包含预处理和后处理模块，以处理数据处理和正则化。我们将语言模型的输出与 POI 信息相结合，进行数据库匹配并得出最终结果。为了评估我们提出的 LMAR，我们进行了离线和在线实验。在离线和在线测试中，我们提出的模型总体准确率超过了 90%，而在在线压力测试中，它也取得了令人满意的成绩，证明了它的有效性和实用性。所提出的 LMAR 已通过内部测试，并将在不久的将来部署到美团无人机配送系统中。
{"title":"A Language Model-Based Fine-Grained Address Resolution Framework in UAV Delivery System","authors":"Sichun Luo;Yuxuan Yao;Haohan Zhao;Linqi Song","doi":"10.1109/JSTSP.2024.3376962","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3376962","url":null,"abstract":"Accurate address resolution plays a vital role in UAV delivery systems. Existing address resolution systems heavily rely on user-provided Point of Interest (POI) information. However, such information often lacks precision, making it challenging to obtain fine-grained details for further processing. In this paper, we present an end-to-end \u0000<underline>L</u>\u0000anguage \u0000<underline>M</u>\u0000odel-based fine-grained \u0000<underline>A</u>\u0000ddress \u0000<underline>R</u>\u0000esolution framework (\u0000<underline>LMAR</u>\u0000). Instead of solely relying on POI information, we introduce a language model to process the user input text information. Specifically, we start by collecting data and constructing two datasets, which are then used to fine-tune a pre-trained language model. Additionally, our pipeline incorporates pre-processing and post-processing modules to handle data processing and regularization. We combine the output of the language model with the POI information to perform a database match and derive the final outcome. To evaluate our proposed LMAR, we conduct offline and online experiments. In both offline and online testing, our proposed model achieves an overall performance of over 90% accuracy, while in the online pressure test, it achieves satisfactory performance, demonstrating its effectiveness and practicality. The proposed LMAR has passed the internal test and will be deployed into the Meituan UAV delivery system in the near future.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 3","pages":"529-539"},"PeriodicalIF":8.7,"publicationDate":"2024-04-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137542","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-31DOI: 10.1109/JSTSP.2024.3408100
Shuzhan Hu;Yiping Duan;Xiaoming Tao;Jian Chu;Jianhua Lu
Vision, as the primary perceptual mode for intelligent robots, plays a crucial role in various human-robot interaction (HRI) scenarios. In certain situations, it is essential to utilize the visual sensors to capture videos for humans, assisting them in tasks like exploration missions. However, the increasing amount of video information brings great challenges for data transmission and storage. Therefore, there is an urgent need to develop more efficient video compression strategies to address this challenge. When perceiving a video, humans tend to pay more attention to some specific clips, which may occupy a small part of the whole video content, but largely affect the perceptual quality. This human visual attention (VA) mechanism provides valuable inspiration for optimizing video compression methods for HRI scenarios. Therefore, we combine psychophysiological paradigms and machine learning methods to model human VA and introduce it into the bitrate allocation to fully utilize the limited resources. Specifically, we collect electroencephalographic (EEG) data when humans watch videos, constructing an EEG dataset reflecting VA. Based on the dataset, we propose a VA measurement model to determine the VA states of humans in their underlying brain responses. Then, a brain-inspired VA prediction model is established to obtain VA metrics directly from the videos. Finally, based on the VA metric, more bitrates are allocated to the clips that humans pay more attention to. The experimental results show that our proposed methods can accurately determine the humans' VA states and predict the VA metrics evoked by different video clips. Furthermore, the bitrate allocation method based on the VA metric can achieve better perceptual quality at low bitrates.
视觉作为智能机器人的主要感知模式，在各种人机交互（HRI）场景中发挥着至关重要的作用。在某些情况下，必须利用视觉传感器为人类捕捉视频，协助人类执行探索任务等任务。然而，视频信息量的不断增加给数据传输和存储带来了巨大挑战。因此，迫切需要开发更高效的视频压缩策略来应对这一挑战。人类在感知视频时，往往会对一些特定片段给予更多关注，这些片段可能只占整个视频内容的一小部分，但却在很大程度上影响着感知质量。这种人类视觉注意力（VA）机制为优化 HRI 场景下的视频压缩方法提供了宝贵的灵感。因此，我们结合心理生理学范式和机器学习方法，对人类视觉注意力进行建模，并将其引入比特率分配，以充分利用有限的资源。具体来说，我们收集了人类观看视频时的脑电图（EEG）数据，构建了一个反映 VA 的 EEG 数据集。基于该数据集，我们提出了一个 VA 测量模型，以确定人类大脑底层反应中的 VA 状态。然后，建立大脑启发的 VA 预测模型，直接从视频中获取 VA 指标。最后，根据 VA 指标，为人类更关注的片段分配更多比特率。实验结果表明，我们提出的方法可以准确判断人类的 VA 状态，并预测不同视频片段所唤起的 VA 指标。此外，基于 VA 指标的比特率分配方法可以在低比特率下获得更好的感知质量。
{"title":"Brain-Inspired Visual Attention Modeling Based on EEG for Intelligent Robotics","authors":"Shuzhan Hu;Yiping Duan;Xiaoming Tao;Jian Chu;Jianhua Lu","doi":"10.1109/JSTSP.2024.3408100","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3408100","url":null,"abstract":"Vision, as the primary perceptual mode for intelligent robots, plays a crucial role in various human-robot interaction (HRI) scenarios. In certain situations, it is essential to utilize the visual sensors to capture videos for humans, assisting them in tasks like exploration missions. However, the increasing amount of video information brings great challenges for data transmission and storage. Therefore, there is an urgent need to develop more efficient video compression strategies to address this challenge. When perceiving a video, humans tend to pay more attention to some specific clips, which may occupy a small part of the whole video content, but largely affect the perceptual quality. This human visual attention (VA) mechanism provides valuable inspiration for optimizing video compression methods for HRI scenarios. Therefore, we combine psychophysiological paradigms and machine learning methods to model human VA and introduce it into the bitrate allocation to fully utilize the limited resources. Specifically, we collect electroencephalographic (EEG) data when humans watch videos, constructing an EEG dataset reflecting VA. Based on the dataset, we propose a VA measurement model to determine the VA states of humans in their underlying brain responses. Then, a brain-inspired VA prediction model is established to obtain VA metrics directly from the videos. Finally, based on the VA metric, more bitrates are allocated to the clips that humans pay more attention to. The experimental results show that our proposed methods can accurately determine the humans' VA states and predict the VA metrics evoked by different video clips. Furthermore, the bitrate allocation method based on the VA metric can achieve better perceptual quality at low bitrates.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 3","pages":"431-443"},"PeriodicalIF":8.7,"publicationDate":"2024-03-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137557","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：