Delivery drones provide a promising sensing platform for smart cities thanks to their city-wide infrastructure and large-scale deployment. However, due to limited battery lifetime and available resources, it is challenging to schedule delivery drones to derive both high sensing and delivery performance, which is a highly complicated optimization problem with several coupled decision variables. Meanwhile, this complex optimization problem involves multiple interconnected decision variables, making it even more complex. In this paper, we first propose a delivery drone-based sensing system and formulate a mixed-integer non-linear programming problem (MINLP) that jointly optimizes the sensing utility and delivery time, considering practical factors including energy capacity and available delivery drones. Then we provide an efficient solution that integrates the strength of deep reinforcement learning (DRL) and heuristic, which decouples the highly complicated optimization search process and replaces the heavy computation with a rapid approximation. Evaluation results compared with the state-of-the-art baselines show that DDL improves the scheduling quality by at least 46% on average. More importantly, our proposed method could effectively improve the computational efficiency, which is up to 98 times higher than the best baseline.
{"title":"DDL: Empowering Delivery Drones With Large-Scale Urban Sensing Capability","authors":"Xuecheng Chen;Haoyang Wang;Yuhan Cheng;Haohao Fu;Yuxuan Liu;Fan Dang;Yunhao Liu;Jinqiang Cui;Xinlei Chen","doi":"10.1109/JSTSP.2024.3427371","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3427371","url":null,"abstract":"Delivery drones provide a promising sensing platform for smart cities thanks to their city-wide infrastructure and large-scale deployment. However, due to limited battery lifetime and available resources, it is challenging to schedule delivery drones to derive both high sensing and delivery performance, which is a highly complicated optimization problem with several coupled decision variables. Meanwhile, this complex optimization problem involves multiple interconnected decision variables, making it even more complex. In this paper, we first propose a delivery drone-based sensing system and formulate a mixed-integer non-linear programming problem (MINLP) that jointly optimizes the sensing utility and delivery time, considering practical factors including energy capacity and available delivery drones. Then we provide an efficient solution that integrates the strength of deep reinforcement learning (DRL) and heuristic, which decouples the highly complicated optimization search process and replaces the heavy computation with a rapid approximation. Evaluation results compared with the state-of-the-art baselines show that \u0000<italic>DDL</i>\u0000 improves the scheduling quality by at least 46% on average. More importantly, our proposed method could effectively improve the computational efficiency, which is up to 98 times higher than the best baseline.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 3","pages":"502-515"},"PeriodicalIF":8.7,"publicationDate":"2024-07-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137524","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-12DOI: 10.1109/JSTSP.2024.3427424
Yinggang Liu;Yuanjie Deng;Ying Wei
Speech separation as a fundamental task in signal processing can be used in many types of intelligent robots, and audio-visual (AV) speech separation has been proven to be superior to audio-only speech separation. In current AV speech separation methods, visual information plays a pivotal role not only during network training but also during testing. However, due to various factors in real environments, sensors do not always possible to obtain high-quality visual signals. In this paper, we propose an effective two-stage AV speech separation model that introduces a new approach of visual feature embedding, where visual information is used to optimize the separation network during training, but no visual input is required during testing. Different from the current methods which fuse visual features and audio features together as the input of the separation network, in this model, visual features are embedded into AV matching block to calculate the cross-modal consistency loss, which is used as part of the loss function for network optimization. A novel tuples loss function with a learnable dynamic margin is proposed for better AV matching, and two margin change strategies are given. The proposed two-stage AV speech separation method is evaluated on the widely used GRID and VoxCeleb2 datasets. Experimental results show that the performance outperforms current AV speech separation methods.
语音分离作为信号处理中的一项基本任务，可用于多种类型的智能机器人，而视听（AV）语音分离已被证明优于纯音频语音分离。在目前的视听语音分离方法中，视觉信息不仅在网络训练过程中起着关键作用，在测试过程中也是如此。然而，由于实际环境中的各种因素，传感器并不总能获得高质量的视觉信号。本文提出了一种有效的两阶段视听语音分离模型，引入了视觉特征嵌入的新方法，即在训练过程中使用视觉信息优化分离网络，而在测试过程中不需要视觉输入。与目前将视觉特征和音频特征融合在一起作为分离网络输入的方法不同，在该模型中，视觉特征被嵌入到影音匹配块中，以计算跨模态一致性损失，并将其作为网络优化损失函数的一部分。为了更好地进行视听匹配，提出了一种具有可学习动态余量的新型元组损失函数，并给出了两种余量变化策略。在广泛使用的 GRID 和 VoxCeleb2 数据集上评估了所提出的两阶段影音语音分离方法。实验结果表明，该方法的性能优于当前的 AV 语音分离方法。
{"title":"A Two-Stage Audio-Visual Speech Separation Method Without Visual Signals for Testing and Tuples Loss With Dynamic Margin","authors":"Yinggang Liu;Yuanjie Deng;Ying Wei","doi":"10.1109/JSTSP.2024.3427424","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3427424","url":null,"abstract":"Speech separation as a fundamental task in signal processing can be used in many types of intelligent robots, and audio-visual (AV) speech separation has been proven to be superior to audio-only speech separation. In current AV speech separation methods, visual information plays a pivotal role not only during network training but also during testing. However, due to various factors in real environments, sensors do not always possible to obtain high-quality visual signals. In this paper, we propose an effective two-stage AV speech separation model that introduces a new approach of visual feature embedding, where visual information is used to optimize the separation network during training, but no visual input is required during testing. Different from the current methods which fuse visual features and audio features together as the input of the separation network, in this model, visual features are embedded into AV matching block to calculate the cross-modal consistency loss, which is used as part of the loss function for network optimization. A novel tuples loss function with a learnable dynamic margin is proposed for better AV matching, and two margin change strategies are given. The proposed two-stage AV speech separation method is evaluated on the widely used GRID and VoxCeleb2 datasets. Experimental results show that the performance outperforms current AV speech separation methods.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 3","pages":"459-472"},"PeriodicalIF":8.7,"publicationDate":"2024-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137555","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-11DOI: 10.1109/JSTSP.2024.3426484
I-Chen Sang;William R. Norris
Breakthroughs in autonomous vehicle technology have ignited diverse topics within engineering research. Among these, the focus on conducting inspections through autonomous underwater vehicles (AUVs) stands out as particularly influential, owing to the substantial investments directed towards offshore infrastructures. Leveraging the capabilities of onboard sensors, AUVs hold the potential to adeptly trace and examine pipelines with high levels of accuracy. However, the complicated and varying underwater environment presents a formidable challenge to ensuring the robustness of the localization and navigation framework. In response to these challenges, this study introduces a novel GPS-denied, adaptive, vision-based navigation framework tailored specifically for AUV inspection tasks. Different from conventional approaches involving manual parameter tuning, this framework dynamically adjusts contrast enhancement and edge detection functions based on incoming frame data. Fuzzy inference systems (FIS) have been harnessed within both image processing and the navigation algorithm, strengthening the overall robustness of the system. The verification of the proposed framework took place within a simulation environment. Through the implemented algorithm, the AUV adeptly identified, approached, and traversed the pipeline. Additionally, the framework distinctly showcased its capacity to dynamically adjust parameters, reduce processing time, and uphold consistency amid diverse illuminations and levels of noise.
{"title":"An Adaptive Image Thresholding Algorithm Using Fuzzy Logic for Autonomous Underwater Vehicle Navigation","authors":"I-Chen Sang;William R. Norris","doi":"10.1109/JSTSP.2024.3426484","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3426484","url":null,"abstract":"Breakthroughs in autonomous vehicle technology have ignited diverse topics within engineering research. Among these, the focus on conducting inspections through autonomous underwater vehicles (AUVs) stands out as particularly influential, owing to the substantial investments directed towards offshore infrastructures. Leveraging the capabilities of onboard sensors, AUVs hold the potential to adeptly trace and examine pipelines with high levels of accuracy. However, the complicated and varying underwater environment presents a formidable challenge to ensuring the robustness of the localization and navigation framework. In response to these challenges, this study introduces a novel GPS-denied, adaptive, vision-based navigation framework tailored specifically for AUV inspection tasks. Different from conventional approaches involving manual parameter tuning, this framework dynamically adjusts contrast enhancement and edge detection functions based on incoming frame data. Fuzzy inference systems (FIS) have been harnessed within both image processing and the navigation algorithm, strengthening the overall robustness of the system. The verification of the proposed framework took place within a simulation environment. Through the implemented algorithm, the AUV adeptly identified, approached, and traversed the pipeline. Additionally, the framework distinctly showcased its capacity to dynamically adjust parameters, reduce processing time, and uphold consistency amid diverse illuminations and levels of noise.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 3","pages":"358-367"},"PeriodicalIF":8.7,"publicationDate":"2024-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10596073","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137552","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-10DOI: 10.1109/JSTSP.2024.3424310
Marc Andrew Valdez;Jacob D. Rezac;Michael B. Wakin;Joshua A. Gordon
We propose compressive sensing approaches for broadband spherical near-field measurements that reduce measurement demands beyond what is achievable using conventional single-frequency compressive sensing. Our approaches use two different compressive signal models—sparsity-based and low-rank-based—whose viability we establish using a simulated standard gain horn antenna. Under mild assumptions on the device being tested, we prove that sparsity-based broadband compressive sensing provides significant measurement number reductions over single-frequency compressive sensing. We find that our proposed low-rank model also provides an effective means of achieving broadband compressive sensing, using numerical experiments, with performance on par with the best broadband sparsity-based method. Exemplifying these best-case results, even in the presence of measurement noise, the methods we propose can achieve relative errors of −40 dB using about 1/4 of the measurements required for conventional sampling. This is equivalent to about 1/2 sample per unknown, whereas traditional spherical near-field measurements require a minimum of roughly 2 measurements per unknown.
我们提出了用于宽带球形近场测量的压缩传感方法，该方法可降低测量要求，超越传统单频压缩传感方法。我们的方法使用了两种不同的压缩信号模型--基于稀疏性的模型和基于低秩的模型，我们使用模拟的标准增益喇叭天线建立了这两种模型的可行性。在测试设备的温和假设条件下，我们证明基于稀疏性的宽带压缩传感比单频压缩传感能显著减少测量次数。通过数值实验，我们发现我们提出的低秩模型也是实现宽带压缩传感的有效方法，其性能与基于稀疏性的最佳宽带方法相当。作为这些最佳结果的例证，即使存在测量噪声，我们提出的方法也能使用传统采样所需的约 1/4 测量值，实现 -40 dB 的相对误差。这相当于对每个未知数进行约 1/2 次采样，而传统的球面近场测量对每个未知数至少需要约 2 次测量。
{"title":"Multi-Frequency Spherical Near-Field Antenna Measurements Using Compressive Sensing","authors":"Marc Andrew Valdez;Jacob D. Rezac;Michael B. Wakin;Joshua A. Gordon","doi":"10.1109/JSTSP.2024.3424310","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3424310","url":null,"abstract":"We propose compressive sensing approaches for broadband spherical near-field measurements that reduce measurement demands beyond what is achievable using conventional single-frequency compressive sensing. Our approaches use two different compressive signal models—sparsity-based and low-rank-based—whose viability we establish using a simulated standard gain horn antenna. Under mild assumptions on the device being tested, we prove that sparsity-based broadband compressive sensing provides significant measurement number reductions over single-frequency compressive sensing. We find that our proposed low-rank model also provides an effective means of achieving broadband compressive sensing, using numerical experiments, with performance on par with the best broadband sparsity-based method. Exemplifying these best-case results, even in the presence of measurement noise, the methods we propose can achieve relative errors of −40 dB using about 1/4 of the measurements required for conventional sampling. This is equivalent to about 1/2 sample per unknown, whereas traditional spherical near-field measurements require a minimum of roughly 2 measurements per unknown.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 4","pages":"572-586"},"PeriodicalIF":8.7,"publicationDate":"2024-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142587618","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-09DOI: 10.1109/JSTSP.2024.3425052
Yi Xia;Jun Du;Zekai Zhang;Ziyuan Wang;Jingzehua Xu;Weishi Mi
Maintaining rapid and prolonged standoff target tracking for networked unmanned aerial vehicles (UAVs) is challenging, as existing methods fail to improve tracking performance while simultaneously reducing energy consumption. This paper proposes a deep reinforcement learning (DRL)-based tracking scheme for UAVs to approximate an escape target, effectively addressing time constraints and guaranteeing low energy expenditure. In the first phase, a coordinated target tracking protocol and a target position estimator are developed using only bearing measurements, which enable the deployment of UAVs along a standoff circle centered at the target with an expected angular spacing. Additionally, an unknown system dynamics estimator (USDE) is devised based on concise filtering operations to mitigate adverse disturbances. In the second phase, multi-agent deep deterministic policy gradient (MADDPG) is employed to strike an optimal balance between tracking accuracy and energy consumption by encoding time limitations as skilled barrier functions. Simulation results demonstrate that the proposed method outperforms benchmarks in terms of tracking accuracy and control cost.
{"title":"Standoff Target Tracking for Networked UAVs With Specified Performance via Deep Reinforcement Learning","authors":"Yi Xia;Jun Du;Zekai Zhang;Ziyuan Wang;Jingzehua Xu;Weishi Mi","doi":"10.1109/JSTSP.2024.3425052","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3425052","url":null,"abstract":"Maintaining rapid and prolonged standoff target tracking for networked unmanned aerial vehicles (UAVs) is challenging, as existing methods fail to improve tracking performance while simultaneously reducing energy consumption. This paper proposes a deep reinforcement learning (DRL)-based tracking scheme for UAVs to approximate an escape target, effectively addressing time constraints and guaranteeing low energy expenditure. In the first phase, a coordinated target tracking protocol and a target position estimator are developed using only bearing measurements, which enable the deployment of UAVs along a standoff circle centered at the target with an expected angular spacing. Additionally, an unknown system dynamics estimator (USDE) is devised based on concise filtering operations to mitigate adverse disturbances. In the second phase, multi-agent deep deterministic policy gradient (MADDPG) is employed to strike an optimal balance between tracking accuracy and energy consumption by encoding time limitations as skilled barrier functions. Simulation results demonstrate that the proposed method outperforms benchmarks in terms of tracking accuracy and control cost.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 3","pages":"516-528"},"PeriodicalIF":8.7,"publicationDate":"2024-07-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137556","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-03DOI: 10.1109/JSTSP.2024.3422823
R. Gnana Praveen;Jahangir Alam
Multimodal emotion recognition has immense potential for the comprehensive assessment of human emotions, utilizing multiple modalities that often exhibit complementary relationships. In video-based emotion recognition, audio and visual modalities have emerged as prominent contact-free channels, widely explored in existing literature. Current approaches typically employ cross-modal attention mechanisms between audio and visual modalities, assuming a constant state of complementarity. However, this assumption may not always hold true, as non-complementary relationships can also manifest, undermining the efficacy of cross-modal feature integration and thereby diminishing the quality of audio-visual feature representations. To tackle this problem, we introduce a novel Incongruity-Aware Cross-Attention (IACA) model, capable of harnessing the benefits of robust complementary relationships while efficiently managing non-complementary scenarios. Specifically, our approach incorporates a two-stage gating mechanism designed to adaptively select semantic features, thereby effectively capturing the inter-modal associations. Additionally, the proposed model demonstrates an ability to mitigate the adverse effects of severely corrupted or missing modalities. We rigorously evaluate the performance of the proposed model through extensive experiments conducted on the challenging RECOLA and Aff-Wild2 datasets. The results underscore the efficacy of our approach, as it outperforms state-of-the-art methods by adeptly capturing inter-modal relationships and minimizing the influence of missing or heavily corrupted modalities. Furthermore, we show that the proposed model is compatible with various cross-modal attention variants, consistently improving performance on both datasets.
{"title":"Incongruity-Aware Cross-Modal Attention for Audio-Visual Fusion in Dimensional Emotion Recognition","authors":"R. Gnana Praveen;Jahangir Alam","doi":"10.1109/JSTSP.2024.3422823","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3422823","url":null,"abstract":"Multimodal emotion recognition has immense potential for the comprehensive assessment of human emotions, utilizing multiple modalities that often exhibit complementary relationships. In video-based emotion recognition, audio and visual modalities have emerged as prominent contact-free channels, widely explored in existing literature. Current approaches typically employ cross-modal attention mechanisms between audio and visual modalities, assuming a constant state of complementarity. However, this assumption may not always hold true, as non-complementary relationships can also manifest, undermining the efficacy of cross-modal feature integration and thereby diminishing the quality of audio-visual feature representations. To tackle this problem, we introduce a novel Incongruity-Aware Cross-Attention (IACA) model, capable of harnessing the benefits of robust complementary relationships while efficiently managing non-complementary scenarios. Specifically, our approach incorporates a two-stage gating mechanism designed to adaptively select semantic features, thereby effectively capturing the inter-modal associations. Additionally, the proposed model demonstrates an ability to mitigate the adverse effects of severely corrupted or missing modalities. We rigorously evaluate the performance of the proposed model through extensive experiments conducted on the challenging RECOLA and Aff-Wild2 datasets. The results underscore the efficacy of our approach, as it outperforms state-of-the-art methods by adeptly capturing inter-modal relationships and minimizing the influence of missing or heavily corrupted modalities. Furthermore, we show that the proposed model is compatible with various cross-modal attention variants, consistently improving performance on both datasets.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 3","pages":"444-458"},"PeriodicalIF":8.7,"publicationDate":"2024-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137574","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-02DOI: 10.1109/JSTSP.2024.3421898
Zitong Wang;Yushan Li;Xiaoming Duan;Jianping He
The interaction topology plays a significant role in the distributed motion coordination of multi-robot systems (MRSs) for its noticeable impact on the information flow between robots. However, recent research has revealed that in adversarial environments, the topology can be inferred by external adversaries equipped with advanced sensors, posing severe security risks to MRSs. Therefore, it is of utmost importance to preserve the interaction topology from inference attacks while ensuring the coordination performance. To this end, we propose a topology-preserving motion coordination (TPMC) algorithm that strategically introduces perturbation signals during the coordination process with a compensation design. The major novelty is threefold: i) We focus on the second-order motion coordination model and tackle the coupling issue of the perturbation signals with the unstable state updating process; ii) We develop a general framework for distributed compensation of perturbation signals, strategically addressing the challenge of perturbation accumulation while ensuring precise motion coordination; iii) We derive the convergence conditions and rate characterization to achieve the motion coordination under the TPMC algorithm. Extensive simulations and real-world experiments are conducted to verify the performance of the proposed method.
{"title":"Topology-Preserving Motion Coordination for Multi-Robot Systems in Adversarial Environments","authors":"Zitong Wang;Yushan Li;Xiaoming Duan;Jianping He","doi":"10.1109/JSTSP.2024.3421898","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3421898","url":null,"abstract":"The interaction topology plays a significant role in the distributed motion coordination of multi-robot systems (MRSs) for its noticeable impact on the information flow between robots. However, recent research has revealed that in adversarial environments, the topology can be inferred by external adversaries equipped with advanced sensors, posing severe security risks to MRSs. Therefore, it is of utmost importance to preserve the interaction topology from inference attacks while ensuring the coordination performance. To this end, we propose a topology-preserving motion coordination (TPMC) algorithm that strategically introduces perturbation signals during the coordination process with a compensation design. The major novelty is threefold: i) We focus on the second-order motion coordination model and tackle the coupling issue of the perturbation signals with the unstable state updating process; ii) We develop a general framework for distributed compensation of perturbation signals, strategically addressing the challenge of perturbation accumulation while ensuring precise motion coordination; iii) We derive the convergence conditions and rate characterization to achieve the motion coordination under the TPMC algorithm. Extensive simulations and real-world experiments are conducted to verify the performance of the proposed method.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 3","pages":"473-486"},"PeriodicalIF":8.7,"publicationDate":"2024-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137600","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-28DOI: 10.1109/JSTSP.2024.3420234
Hongyu Chen;Yimin Liu;Yuwei Cheng
The robust perception of environments is crucial for mobile robots to operate autonomously in complex environments. Over the years, mobile robots mainly rely on optical sensors for perception, which degrade severely in adverse weather conditions. Recently, single-chip millimeter-wave (mmWave) radars have been widely used for mobile perception, owing to their robustness to all-weather conditions, lightweight design, and low cost. However, existing research based on mmWave radars primarily focuses on single radar and single task. Due to the limited field of view and sparse observation, perception based on a single radar may not ensure the required robustness in complex environments. To address this challenge, we propose a novel robust perception framework for robots in complex environments based on multiple mmWave radars, named MMR-PFR. The framework integrates three critical tasks for robots, including ego-motion estimation, multi-radar fusion mapping, and dynamic target state estimation. Multiple tasks collaborate and facilitate each other to improve overall performance. In the framework, we propose a new multi-radar point cloud fusion method to generate a more accurate environmental map. In addition, we propose a new online calibration algorithm for multiple radars to ensure the long-term reliability of the system. To evaluate MMR-PRF, we build a prototype and carry out experiments in real-world scenarios. The evaluation results show the effectiveness and superiority of the proposed framework.
{"title":"A Robust Robot Perception Framework for Complex Environments Using Multiple mmWave Radars","authors":"Hongyu Chen;Yimin Liu;Yuwei Cheng","doi":"10.1109/JSTSP.2024.3420234","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3420234","url":null,"abstract":"The robust perception of environments is crucial for mobile robots to operate autonomously in complex environments. Over the years, mobile robots mainly rely on optical sensors for perception, which degrade severely in adverse weather conditions. Recently, single-chip millimeter-wave (mmWave) radars have been widely used for mobile perception, owing to their robustness to all-weather conditions, lightweight design, and low cost. However, existing research based on mmWave radars primarily focuses on single radar and single task. Due to the limited field of view and sparse observation, perception based on a single radar may not ensure the required robustness in complex environments. To address this challenge, we propose a novel robust perception framework for robots in complex environments based on multiple mmWave radars, named MMR-PFR. The framework integrates three critical tasks for robots, including ego-motion estimation, multi-radar fusion mapping, and dynamic target state estimation. Multiple tasks collaborate and facilitate each other to improve overall performance. In the framework, we propose a new multi-radar point cloud fusion method to generate a more accurate environmental map. In addition, we propose a new online calibration algorithm for multiple radars to ensure the long-term reliability of the system. To evaluate MMR-PRF, we build a prototype and carry out experiments in real-world scenarios. The evaluation results show the effectiveness and superiority of the proposed framework.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 3","pages":"380-395"},"PeriodicalIF":8.7,"publicationDate":"2024-06-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137512","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-20DOI: 10.1109/JSTSP.2024.3417356
G. Torcolacci;A. Guerra;H. Zhang;F. Guidi;Q. Yang;Y. C. Eldar;D. Dardari
This paper addresses a near-field imaging problem utilizing extremely large-scale multiple-input multiple-output (XL-MIMO) antennas and reconfigurable intelligent surfaces (RISs) already in place for wireless communications. To this end, we consider a system with a fixed transmitting antenna array illuminating a region of interest (ROI) and a fixed receiving antenna array inferring the ROI's scattering coefficients. Leveraging XL-MIMO and high frequencies, the ROI is situated in the radiative near-field region of both antenna arrays, thus enhancing the degrees of freedom (DoF) (i.e., the channel matrix rank) of the illuminating and sensing channels available for imaging, here referred to as holographic imaging . To further boost the imaging performance, we optimize the illuminating waveform by solving a min-max optimization problem having the upper bound of the mean squared error (MSE) of the image estimate as the objective function. Additionally, we address the challenge of non-line-of-sight (NLOS) scenarios by considering the presence of a RIS and deriving its optimal reflection coefficients. Numerical results investigate the interplay between illumination optimization, geometric configuration (monostatic and bistatic), the DoF of the illuminating and sensing channels, image estimation accuracy, and image complexity.
{"title":"Holographic Imaging With XL-MIMO and RIS: Illumination and Reflection Design","authors":"G. Torcolacci;A. Guerra;H. Zhang;F. Guidi;Q. Yang;Y. C. Eldar;D. Dardari","doi":"10.1109/JSTSP.2024.3417356","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3417356","url":null,"abstract":"This paper addresses a near-field imaging problem utilizing extremely large-scale multiple-input multiple-output (XL-MIMO) antennas and reconfigurable intelligent surfaces (RISs) already in place for wireless communications. To this end, we consider a system with a fixed transmitting antenna array illuminating a region of interest (ROI) and a fixed receiving antenna array inferring the ROI's scattering coefficients. Leveraging XL-MIMO and high frequencies, the ROI is situated in the radiative near-field region of both antenna arrays, thus enhancing the degrees of freedom (DoF) (i.e., the channel matrix rank) of the illuminating and sensing channels available for imaging, here referred to as \u0000<italic>holographic imaging</i>\u0000. To further boost the imaging performance, we optimize the illuminating waveform by solving a min-max optimization problem having the upper bound of the mean squared error (MSE) of the image estimate as the objective function. Additionally, we address the challenge of non-line-of-sight (NLOS) scenarios by considering the presence of a RIS and deriving its optimal reflection coefficients. Numerical results investigate the interplay between illumination optimization, geometric configuration (monostatic and bistatic), the DoF of the illuminating and sensing channels, image estimation accuracy, and image complexity.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 4","pages":"587-602"},"PeriodicalIF":8.7,"publicationDate":"2024-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142587620","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Tactile sensors, which provide information about the physical properties of objects, are an essential component of robotic systems. The visuotactile sensing technology with the merits of high resolution and low cost has facilitated the development of robotics from environment exploration to dexterous operation. Over the years, several reviews on visuotactile sensors for robots have been presented, but few of them discussed the significance of signal processing methods to visuotactile sensors. Apart from ingenious hardware design, the full potential of the sensory system toward designated tasks can only be released with the appropriate signal processing methods. Therefore, this paper provides a comprehensive review of visuotactile sensors from the perspective of signal processing methods and outlooks possible future research directions for visuotactile sensors.
{"title":"When Vision Meets Touch: A Contemporary Review for Visuotactile Sensors From the Signal Processing Perspective","authors":"Shoujie Li;Zihan Wang;Changsheng Wu;Xiang Li;Shan Luo;Bin Fang;Fuchun Sun;Xiao-Ping Zhang;Wenbo Ding","doi":"10.1109/JSTSP.2024.3416841","DOIUrl":"https://doi.org/10.1109/JSTSP.2024.3416841","url":null,"abstract":"Tactile sensors, which provide information about the physical properties of objects, are an essential component of robotic systems. The visuotactile sensing technology with the merits of high resolution and low cost has facilitated the development of robotics from environment exploration to dexterous operation. Over the years, several reviews on visuotactile sensors for robots have been presented, but few of them discussed the significance of signal processing methods to visuotactile sensors. Apart from ingenious hardware design, the full potential of the sensory system toward designated tasks can only be released with the appropriate signal processing methods. Therefore, this paper provides a comprehensive review of visuotactile sensors from the perspective of signal processing methods and outlooks possible future research directions for visuotactile sensors.","PeriodicalId":13038,"journal":{"name":"IEEE Journal of Selected Topics in Signal Processing","volume":"18 3","pages":"267-287"},"PeriodicalIF":8.7,"publicationDate":"2024-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137511","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：