首页 > 最新文献

IEEE Robotics and Automation Letters最新文献

英文 中文
AirBender: Adaptive Transportation of Bendable Objects Using Dual UAVs
IF 4.6 2区 计算机科学 Q2 ROBOTICS Pub Date : 2025-01-29 DOI: 10.1109/LRA.2025.3536276
Jiawei Xu;Longsen Gao;Rafael Fierro;David Saldaña
The interaction of robots with bendable objects in midair presents significant challenges in control, often resulting in performance degradation and potential crashes, especially for aerial robots due to their limited actuation capabilities and constant need to remain airborne. This letter presents an adaptive controller that enables two aerial vehicles to collaboratively follow a trajectory while transporting a bendable object without relying on explicit elasticity models. Our method allows on-the-fly adaptation to the object's unknown deformable properties, ensuring stability and performance in trajectory-tracking tasks. We use Lyapunov analysis to demonstrate that our adaptive controller is asymptotically stable. Our method is evaluated through hardware experiments in various scenarios, demonstrating the capabilities of using multirotor aerial vehicles to handle bendable objects.
{"title":"AirBender: Adaptive Transportation of Bendable Objects Using Dual UAVs","authors":"Jiawei Xu;Longsen Gao;Rafael Fierro;David Saldaña","doi":"10.1109/LRA.2025.3536276","DOIUrl":"https://doi.org/10.1109/LRA.2025.3536276","url":null,"abstract":"The interaction of robots with bendable objects in midair presents significant challenges in control, often resulting in performance degradation and potential crashes, especially for aerial robots due to their limited actuation capabilities and constant need to remain airborne. This letter presents an adaptive controller that enables two aerial vehicles to collaboratively follow a trajectory while transporting a bendable object without relying on explicit elasticity models. Our method allows on-the-fly adaptation to the object's unknown deformable properties, ensuring stability and performance in trajectory-tracking tasks. We use Lyapunov analysis to demonstrate that our adaptive controller is asymptotically stable. Our method is evaluated through hardware experiments in various scenarios, demonstrating the capabilities of using multirotor aerial vehicles to handle bendable objects.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 3","pages":"2790-2797"},"PeriodicalIF":4.6,"publicationDate":"2025-01-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143379541","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Sampling-Based Model Predictive Control Leveraging Parallelizable Physics Simulations
IF 4.6 2区 计算机科学 Q2 ROBOTICS Pub Date : 2025-01-28 DOI: 10.1109/LRA.2025.3535185
Corrado Pezzato;Chadi Salmi;Elia Trevisan;Max Spahn;Javier Alonso-Mora;Carlos Hernández Corbato
We present a sampling-based model predictive control method that uses a generic physics simulator as the dynamical model. In particular, we propose a Model Predictive Path Integral controller (MPPI) that employs the GPU-parallelizable IsaacGym simulator to compute the forward dynamics of the robot and environment. Since the simulator implicitly defines the dynamic model, our method is readily extendable to different objects and robots, allowing one to solve complex navigation and contact-rich tasks. We demonstrate the effectiveness of this method in several simulated and real-world settings, including mobile navigation with collision avoidance, non-prehensile manipulation, and whole-body control for high-dimensional configuration spaces. This is a powerful and accessible open-source tool to solve many contact-rich motion planning tasks.
{"title":"Sampling-Based Model Predictive Control Leveraging Parallelizable Physics Simulations","authors":"Corrado Pezzato;Chadi Salmi;Elia Trevisan;Max Spahn;Javier Alonso-Mora;Carlos Hernández Corbato","doi":"10.1109/LRA.2025.3535185","DOIUrl":"https://doi.org/10.1109/LRA.2025.3535185","url":null,"abstract":"We present a sampling-based model predictive control method that uses a generic physics simulator as the dynamical model. In particular, we propose a Model Predictive Path Integral controller (MPPI) that employs the GPU-parallelizable IsaacGym simulator to compute the forward dynamics of the robot and environment. Since the simulator implicitly defines the dynamic model, our method is readily extendable to different objects and robots, allowing one to solve complex navigation and contact-rich tasks. We demonstrate the effectiveness of this method in several simulated and real-world settings, including mobile navigation with collision avoidance, non-prehensile manipulation, and whole-body control for high-dimensional configuration spaces. This is a powerful and accessible open-source tool to solve many contact-rich motion planning tasks.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 3","pages":"2750-2757"},"PeriodicalIF":4.6,"publicationDate":"2025-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143379603","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
IEEE Robotics and Automation Society Information
IF 4.6 2区 计算机科学 Q2 ROBOTICS Pub Date : 2025-01-28 DOI: 10.1109/LRA.2025.3532379
{"title":"IEEE Robotics and Automation Society Information","authors":"","doi":"10.1109/LRA.2025.3532379","DOIUrl":"https://doi.org/10.1109/LRA.2025.3532379","url":null,"abstract":"","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"C2-C2"},"PeriodicalIF":4.6,"publicationDate":"2025-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10856565","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143184120","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
IEEE Robotics and Automation Letters Information for Authors
IF 4.6 2区 计算机科学 Q2 ROBOTICS Pub Date : 2025-01-28 DOI: 10.1109/LRA.2025.3532383
{"title":"IEEE Robotics and Automation Letters Information for Authors","authors":"","doi":"10.1109/LRA.2025.3532383","DOIUrl":"https://doi.org/10.1109/LRA.2025.3532383","url":null,"abstract":"","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"C4-C4"},"PeriodicalIF":4.6,"publicationDate":"2025-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10856566","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143184117","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
IEEE Robotics and Automation Society Information
IF 4.6 2区 计算机科学 Q2 ROBOTICS Pub Date : 2025-01-28 DOI: 10.1109/LRA.2025.3532381
{"title":"IEEE Robotics and Automation Society Information","authors":"","doi":"10.1109/LRA.2025.3532381","DOIUrl":"https://doi.org/10.1109/LRA.2025.3532381","url":null,"abstract":"","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"C3-C3"},"PeriodicalIF":4.6,"publicationDate":"2025-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10856532","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143184118","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Reinforcement Learning of Flexible Policies for Symbolic Instructions With Adjustable Mapping Specifications
IF 4.6 2区 计算机科学 Q2 ROBOTICS Pub Date : 2025-01-28 DOI: 10.1109/LRA.2025.3535187
Wataru Hatanaka;Ryota Yamashina;Takamitsu Matsubara
Symbolic task representation is a powerful tool for encoding human instructions and domain knowledge. Such instructions guide robots to accomplish diverse objectives and meet constraints through reinforcement learning (RL). Most existing methods are based on fixed mappings from environmental states to symbols. However, in inspection tasks, where equipment conditions must be evaluated from multiple perspectives to avoid errors of oversight, robots must fulfill the same symbol from different states. To help robots respond to flexible symbol mapping, we propose representing symbols and their mapping specifications separately within an RL policy. This approach imposes on RL policy to learn combinations of symbolic instructions and mapping specifications, requiring an efficient learning framework. To cope with this issue, we introduce an approach for learning flexible policies called Symbolic Instructions with Adjustable Mapping Specifications (SIAMS). This paper represents symbolic instructions using linear temporal logic (LTL), a formal language that can be easily integrated into RL. Our method addresses the diversified completion patterns of instructions by (1) a specification-aware state modulation, which embeds differences in mapping specifications in state features, and (2) a symbol-number-based task curriculum, which gradually provides tasks according to the learning's progress. Evaluations in 3D simulations with discrete and continuous action spaces demonstrate that our method outperforms context-aware multi-task RL comparisons.
{"title":"Reinforcement Learning of Flexible Policies for Symbolic Instructions With Adjustable Mapping Specifications","authors":"Wataru Hatanaka;Ryota Yamashina;Takamitsu Matsubara","doi":"10.1109/LRA.2025.3535187","DOIUrl":"https://doi.org/10.1109/LRA.2025.3535187","url":null,"abstract":"Symbolic task representation is a powerful tool for encoding human instructions and domain knowledge. Such instructions guide robots to accomplish diverse objectives and meet constraints through reinforcement learning (RL). Most existing methods are based on fixed mappings from environmental states to symbols. However, in inspection tasks, where equipment conditions must be evaluated from multiple perspectives to avoid errors of oversight, robots must fulfill the same symbol from different states. To help robots respond to flexible symbol mapping, we propose representing symbols and their mapping specifications separately within an RL policy. This approach imposes on RL policy to learn combinations of symbolic instructions and mapping specifications, requiring an efficient learning framework. To cope with this issue, we introduce an approach for learning flexible policies called Symbolic Instructions with Adjustable Mapping Specifications (SIAMS). This paper represents symbolic instructions using linear temporal logic (LTL), a formal language that can be easily integrated into RL. Our method addresses the diversified completion patterns of instructions by (1) a specification-aware state modulation, which embeds differences in mapping specifications in state features, and (2) a symbol-number-based task curriculum, which gradually provides tasks according to the learning's progress. Evaluations in 3D simulations with discrete and continuous action spaces demonstrate that our method outperforms context-aware multi-task RL comparisons.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 3","pages":"2614-2621"},"PeriodicalIF":4.6,"publicationDate":"2025-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143361293","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A Magnetically-Actuated Ultrasound Capsule Endoscope (MUSCE) for Endoluminal Imaging in Tubular Environments
IF 4.6 2区 计算机科学 Q2 ROBOTICS Pub Date : 2025-01-28 DOI: 10.1109/LRA.2025.3534691
Zhengxin Yang;Lihao Liu;Zhangjian Li;Yang Jiao;Li Zhang;Yaoyao Cui
Endoscopic ultrasound (EUS) has the ability to image tissue in and beyond the wall of the gastrointestinal (GI) tract, assisting in the early diagnosis of digestive diseases. However, traditional EUS based on flexible endoscopes could make the operation procedure traumatic and intolerable to patients; moreover, certain areas such as most of the small intestine are generally hard to image due to the limited accessibility. To perform endoluminal US imaging in hard-to-reach GI tract, this study proposes a novel magnetically-actuated ultrasound capsule endoscope (MUSCE). Firstly, the MUSCE consisting of a rotatory magnetic-acoustic head and a stable soft tether is designed, featuring small dimensions, low cost, and active locomotion. Next, the magnetic force-modulated torque actuation principle and the corresponding control strategy and US image reconstruction algorithm are analyzed, enabling simultaneous active motion and B-scan ultrasonic imaging. Finally, the effectiveness of the MUSCE is validated by locomotion and imaging tests. Results demonstrate that the designed MUSCE could perform active motion (maximum speed of $sim$ 13.8 mm/s) and B-scan US imaging (resolution of $sim$84.2 $mu$m) in tubular environments with mitigated discomfort, providing a potential solution for clinical applications.
{"title":"A Magnetically-Actuated Ultrasound Capsule Endoscope (MUSCE) for Endoluminal Imaging in Tubular Environments","authors":"Zhengxin Yang;Lihao Liu;Zhangjian Li;Yang Jiao;Li Zhang;Yaoyao Cui","doi":"10.1109/LRA.2025.3534691","DOIUrl":"https://doi.org/10.1109/LRA.2025.3534691","url":null,"abstract":"Endoscopic ultrasound (EUS) has the ability to image tissue in and beyond the wall of the gastrointestinal (GI) tract, assisting in the early diagnosis of digestive diseases. However, traditional EUS based on flexible endoscopes could make the operation procedure traumatic and intolerable to patients; moreover, certain areas such as most of the small intestine are generally hard to image due to the limited accessibility. To perform endoluminal US imaging in hard-to-reach GI tract, this study proposes a novel magnetically-actuated ultrasound capsule endoscope (MUSCE). Firstly, the MUSCE consisting of a rotatory magnetic-acoustic head and a stable soft tether is designed, featuring small dimensions, low cost, and active locomotion. Next, the magnetic force-modulated torque actuation principle and the corresponding control strategy and US image reconstruction algorithm are analyzed, enabling simultaneous active motion and B-scan ultrasonic imaging. Finally, the effectiveness of the MUSCE is validated by locomotion and imaging tests. Results demonstrate that the designed MUSCE could perform active motion (maximum speed of <inline-formula><tex-math>$sim$</tex-math></inline-formula> 13.8 mm/s) and B-scan US imaging (resolution of <inline-formula><tex-math>$sim$</tex-math></inline-formula>84.2 <inline-formula><tex-math>$mu$</tex-math></inline-formula>m) in tubular environments with mitigated discomfort, providing a potential solution for clinical applications.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 3","pages":"2590-2597"},"PeriodicalIF":4.6,"publicationDate":"2025-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143361467","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Event-Frame-Inertial Odometry Using Point and Line Features Based on Coarse-to-Fine Motion Compensation
IF 4.6 2区 计算机科学 Q2 ROBOTICS Pub Date : 2025-01-27 DOI: 10.1109/LRA.2025.3535188
Byeongpil Choi;Hanyeol Lee;Chan Gook Park
An event camera is a vision sensor that captures pixel-level brightness changes and outputs this information as asynchronous events. These events are primarily generated from geometric structures such as edges, which are sensitive to variations in brightness. In this letter, we aim to leverage this line structure information alongside point features to enhance the robustness and accuracy of localization in indoor or human-made environments. To obtain precise line measurements from events, we propose a novel line detection method that incorporates a coarse-to-fine motion compensation scheme, which generates highly sharp event frames. The extracted line features are paired with point features, eliminating the need for traditional line descriptors. Finally, the event features are effectively fused with frame-based point features within a multi-state constraint Kalman filter-based backend, fully exploiting the complementary advantages of both sensors. The performance of the proposed method is verified through an author-constructed experiment and two public datasets, demonstrating improved accuracy in line detection and pose estimation.
{"title":"Event-Frame-Inertial Odometry Using Point and Line Features Based on Coarse-to-Fine Motion Compensation","authors":"Byeongpil Choi;Hanyeol Lee;Chan Gook Park","doi":"10.1109/LRA.2025.3535188","DOIUrl":"https://doi.org/10.1109/LRA.2025.3535188","url":null,"abstract":"An event camera is a vision sensor that captures pixel-level brightness changes and outputs this information as asynchronous events. These events are primarily generated from geometric structures such as edges, which are sensitive to variations in brightness. In this letter, we aim to leverage this line structure information alongside point features to enhance the robustness and accuracy of localization in indoor or human-made environments. To obtain precise line measurements from events, we propose a novel line detection method that incorporates a coarse-to-fine motion compensation scheme, which generates highly sharp event frames. The extracted line features are paired with point features, eliminating the need for traditional line descriptors. Finally, the event features are effectively fused with frame-based point features within a multi-state constraint Kalman filter-based backend, fully exploiting the complementary advantages of both sensors. The performance of the proposed method is verified through an author-constructed experiment and two public datasets, demonstrating improved accuracy in line detection and pose estimation.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 3","pages":"2622-2629"},"PeriodicalIF":4.6,"publicationDate":"2025-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143361294","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Soft Gripper With Movable Variable Stiffness Mechanism and Embedded Soft Sensors for Adaptive Grasping Strategies
IF 4.6 2区 计算机科学 Q2 ROBOTICS Pub Date : 2025-01-27 DOI: 10.1109/LRA.2025.3534058
Beom Geun Ki;Yong-Jai Park
Soft grippers have gained significant attention in robotics for their ability to adapt to diverse object geometries, comply with variable surfaces, and handle fragile items safely. Incorporating variable stiffness mechanisms has enhanced their performance by enabling adjustments in gripping force and structural rigidity. However, many existing designs primarily focus on stiffness modulation from the perspective of gripping force, limiting their ability to achieve localized control or handle objects with varying sizes, shapes, and weights effectively. This letter introduces a novel soft gripper integrating a movable variable stiffness mechanism and embedded soft sensors to overcome these challenges. The gripper locally modulates stiffness, allowing control over gripping force, contact area, and load distribution. Embedded soft sensors provide real-time feedback on gripping status, enabling stiffness adjustment and secure object handling. Fabricated as a monolithic structure through multi-material 3D printing, the gripper integrates functional components into a single manufacturing process with minimal post-processing. Experimental results validate the gripper's adaptability and versatility in handling delicate, irregular, and heavy objects, highlighting its potential for advanced robotic applications.
{"title":"Soft Gripper With Movable Variable Stiffness Mechanism and Embedded Soft Sensors for Adaptive Grasping Strategies","authors":"Beom Geun Ki;Yong-Jai Park","doi":"10.1109/LRA.2025.3534058","DOIUrl":"https://doi.org/10.1109/LRA.2025.3534058","url":null,"abstract":"Soft grippers have gained significant attention in robotics for their ability to adapt to diverse object geometries, comply with variable surfaces, and handle fragile items safely. Incorporating variable stiffness mechanisms has enhanced their performance by enabling adjustments in gripping force and structural rigidity. However, many existing designs primarily focus on stiffness modulation from the perspective of gripping force, limiting their ability to achieve localized control or handle objects with varying sizes, shapes, and weights effectively. This letter introduces a novel soft gripper integrating a movable variable stiffness mechanism and embedded soft sensors to overcome these challenges. The gripper locally modulates stiffness, allowing control over gripping force, contact area, and load distribution. Embedded soft sensors provide real-time feedback on gripping status, enabling stiffness adjustment and secure object handling. Fabricated as a monolithic structure through multi-material 3D printing, the gripper integrates functional components into a single manufacturing process with minimal post-processing. Experimental results validate the gripper's adaptability and versatility in handling delicate, irregular, and heavy objects, highlighting its potential for advanced robotic applications.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 3","pages":"2710-2717"},"PeriodicalIF":4.6,"publicationDate":"2025-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10854649","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143361461","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
MVF-GNN: Multi-View Fusion With GNN for 3D Semantic Segmentation
IF 4.6 2区 计算机科学 Q2 ROBOTICS Pub Date : 2025-01-27 DOI: 10.1109/LRA.2025.3534693
Zhenxiang Du;Minglun Ren;Wei Chu;Nengying Chen
Due to the high cost of obtaining 3D annotations and the accumulation of many 2D datasets with 2D semantic labels, deploying multi-view 2D images for 3D semantic segmentation has attracted widespread attention. Fusion of multi-view information requires establishing local-to-local as well as local-to-global dependencies among multiple views. However, previous methods that are based on 2D annotations supervision cannot model local-to-local and local-to-global dependencies simultaneously. In this letter, we propose a novel multi-view fusion framework with graph neural networks (MVF-GNN) for multi-view interaction and integration. First, a multi-view graph based on the associated pixels in multiple views is constructed. Then, a multi-scale multi-view graph attention network (MSMVGAT) module is introduced to perform graph reasoning on multi-view graphs at different scales. Finally, an attention multi-view graph aggregation (AMVGA) module is introduced to learn the importance of different views and integrate multi-view features. Experiments on the ScanNetv2 benchmark dataset show that our method outperforms state-of-the-art 2D/3D semantic segmentation methods based on 2D annotations supervision.
{"title":"MVF-GNN: Multi-View Fusion With GNN for 3D Semantic Segmentation","authors":"Zhenxiang Du;Minglun Ren;Wei Chu;Nengying Chen","doi":"10.1109/LRA.2025.3534693","DOIUrl":"https://doi.org/10.1109/LRA.2025.3534693","url":null,"abstract":"Due to the high cost of obtaining 3D annotations and the accumulation of many 2D datasets with 2D semantic labels, deploying multi-view 2D images for 3D semantic segmentation has attracted widespread attention. Fusion of multi-view information requires establishing local-to-local as well as local-to-global dependencies among multiple views. However, previous methods that are based on 2D annotations supervision cannot model local-to-local and local-to-global dependencies simultaneously. In this letter, we propose a novel multi-view fusion framework with graph neural networks (MVF-GNN) for multi-view interaction and integration. First, a multi-view graph based on the associated pixels in multiple views is constructed. Then, a multi-scale multi-view graph attention network (MSMVGAT) module is introduced to perform graph reasoning on multi-view graphs at different scales. Finally, an attention multi-view graph aggregation (AMVGA) module is introduced to learn the importance of different views and integrate multi-view features. Experiments on the ScanNetv2 benchmark dataset show that our method outperforms state-of-the-art 2D/3D semantic segmentation methods based on 2D annotations supervision.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 4","pages":"3262-3269"},"PeriodicalIF":4.6,"publicationDate":"2025-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143480856","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
IEEE Robotics and Automation Letters
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1