Advancements in 4D mmWave radar with multi-sensor fusion have significantly enhanced the robustness of autonomous driving systems. In the context of “Mining 5.0” based on parallel intelligence theory, autonomous haulage need to achieve full autonomy in open-pit mines. Current systems use 3D mmWave radar, LiDAR, and cameras but have limited automation progress. This perspective discusses the limitations of these systems and how integrating 4D mmWave radar can improve mining autonomy. This perspective results from discussions at several recent Distributed/Decentralized Hybrid Workshops on Autonomous Mining (DHW-AM) and aims at enhancing the intelligence of future mining operations.
{"title":"Enhancement Technology for Perception in Smart Mining Vehicles: 4D Millimeter-Wave Radar and Multi-Sensor Fusion","authors":"Jianjian Yang;Tianmu Gui;Yuyuan Zhang;Shirong Ge;Qiankun Huang;Guanghui Zhao","doi":"10.1109/TIV.2024.3427718","DOIUrl":"https://doi.org/10.1109/TIV.2024.3427718","url":null,"abstract":"Advancements in 4D mmWave radar with multi-sensor fusion have significantly enhanced the robustness of autonomous driving systems. In the context of “Mining 5.0” based on parallel intelligence theory, autonomous haulage need to achieve full autonomy in open-pit mines. Current systems use 3D mmWave radar, LiDAR, and cameras but have limited automation progress. This perspective discusses the limitations of these systems and how integrating 4D mmWave radar can improve mining autonomy. This perspective results from discussions at several recent Distributed/Decentralized Hybrid Workshops on Autonomous Mining (DHW-AM) and aims at enhancing the intelligence of future mining operations.","PeriodicalId":36532,"journal":{"name":"IEEE Transactions on Intelligent Vehicles","volume":"9 6","pages":"5009-5013"},"PeriodicalIF":14.0,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141965412","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-01DOI: 10.1109/TIV.2024.3437221
{"title":"TechRxiv: Share Your Preprint Research with the World!","authors":"","doi":"10.1109/TIV.2024.3437221","DOIUrl":"https://doi.org/10.1109/TIV.2024.3437221","url":null,"abstract":"","PeriodicalId":36532,"journal":{"name":"IEEE Transactions on Intelligent Vehicles","volume":"9 6","pages":"5118-5118"},"PeriodicalIF":14.0,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10631816","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141965416","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-29DOI: 10.1109/TIV.2024.3393638
Fei-Yue Wang
The current issue includes 3 perspectives, 2 letters and 17 regular papers. These perspectives explore critical issues within the field of IVs and propose prospective research directions based on the evolution of foundation models. After Scanning the Issue , I would like to share insights on how Sora-based imaginative intelligence could propel the future development of IVs.
{"title":"Imaginative Intelligence for Intelligent Vehicles: Sora Inspired New Directions for New Mobility and Vehicle Intelligence","authors":"Fei-Yue Wang","doi":"10.1109/TIV.2024.3393638","DOIUrl":"https://doi.org/10.1109/TIV.2024.3393638","url":null,"abstract":"The current issue includes 3 perspectives, 2 letters and 17 regular papers. These perspectives explore critical issues within the field of IVs and propose prospective research directions based on the evolution of foundation models. After \u0000<bold>Scanning the Issue</b>\u0000, I would like to share insights on how Sora-based imaginative intelligence could propel the future development of IVs.","PeriodicalId":36532,"journal":{"name":"IEEE Transactions on Intelligent Vehicles","volume":"9 4","pages":"4557-4562"},"PeriodicalIF":8.2,"publicationDate":"2024-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141315144","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This letter summarizes discussions from IEEE TIV's Autonomous Mining Workshop, emphasizing the potential of video generation models in advancing smart mining.
{"title":"Sora for Smart Mining: Towards Sustainability With Imaginative Intelligence and Parallel Intelligence","authors":"Yuting Xie;Cong Wang;Kunhua Liu;Zhe Xuanyuan;Yuhang He;Hui Cheng;Andreas Nüchter;Lingxi Li;Rouxing Huai;Shuming Tang;Siji Ma;Long Chen","doi":"10.1109/TIV.2024.3394520","DOIUrl":"https://doi.org/10.1109/TIV.2024.3394520","url":null,"abstract":"This letter summarizes discussions from IEEE TIV's Autonomous Mining Workshop, emphasizing the potential of video generation models in advancing smart mining.","PeriodicalId":36532,"journal":{"name":"IEEE Transactions on Intelligent Vehicles","volume":"9 4","pages":"4577-4578"},"PeriodicalIF":8.2,"publicationDate":"2024-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141315192","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-25DOI: 10.1109/TIV.2024.3393749
Christian Creß;Walter Zimmer;Nils Purschke;Bach Ngoc Doan;Sven Kirchner;Venkatnarayanan Lakshminarasimhan;Leah Strand;Alois C. Knoll
Event-based cameras are predestined for Intelligent Transportation Systems (ITS). They provide very high temporal resolution and dynamic range, which can eliminate motion blur and improve detection performance at night. However, event-based images lack color and texture compared to images from a conventional RGB camera. Considering that, data fusion between event-based and conventional cameras can combine the strengths of both modalities. For this purpose, extrinsic calibration is necessary. To the best of our knowledge, no targetless calibration between event-based and RGB cameras can handle multiple moving objects, nor does data fusion optimized for the domain of roadside ITS exist. Furthermore, synchronized event-based and RGB camera datasets considering roadside perspective are not yet published. To fill these research gaps, based on our previous work, we extended our targetless calibration approach with clustering methods to handle multiple moving objects. Furthermore, we developed an Early Fusion, Simple Late Fusion, and a novel Spatiotemporal Late Fusion method. Lastly, we published the TUMTraf Event Dataset, which contains more than 4,111 synchronized event-based and RGB images with 50,496 labeled 2D boxes. During our extensive experiments, we verified the effectiveness of our calibration method with multiple moving objects. Furthermore, compared to a single RGB camera, we increased the detection performance of up to +9% mAP in the day and up to +13% mAP during the challenging night with our presented event-based sensor fusion methods.
{"title":"TUMTraf Event: Calibration and Fusion Resulting in a Dataset for Roadside Event-Based and RGB Cameras","authors":"Christian Creß;Walter Zimmer;Nils Purschke;Bach Ngoc Doan;Sven Kirchner;Venkatnarayanan Lakshminarasimhan;Leah Strand;Alois C. Knoll","doi":"10.1109/TIV.2024.3393749","DOIUrl":"https://doi.org/10.1109/TIV.2024.3393749","url":null,"abstract":"Event-based cameras are predestined for Intelligent Transportation Systems (ITS). They provide very high temporal resolution and dynamic range, which can eliminate motion blur and improve detection performance at night. However, event-based images lack color and texture compared to images from a conventional RGB camera. Considering that, data fusion between event-based and conventional cameras can combine the strengths of both modalities. For this purpose, extrinsic calibration is necessary. To the best of our knowledge, no targetless calibration between event-based and RGB cameras can handle multiple moving objects, nor does data fusion optimized for the domain of roadside ITS exist. Furthermore, synchronized event-based and RGB camera datasets considering roadside perspective are not yet published. To fill these research gaps, based on our previous work, we extended our targetless calibration approach with clustering methods to handle multiple moving objects. Furthermore, we developed an Early Fusion, Simple Late Fusion, and a novel Spatiotemporal Late Fusion method. Lastly, we published the TUMTraf Event Dataset, which contains more than 4,111 synchronized event-based and RGB images with 50,496 labeled 2D boxes. During our extensive experiments, we verified the effectiveness of our calibration method with multiple moving objects. Furthermore, compared to a single RGB camera, we increased the detection performance of up to +9% mAP in the day and up to +13% mAP during the challenging night with our presented event-based sensor fusion methods.","PeriodicalId":36532,"journal":{"name":"IEEE Transactions on Intelligent Vehicles","volume":"9 7","pages":"5186-5203"},"PeriodicalIF":14.0,"publicationDate":"2024-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10508494","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142320476","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-23DOI: 10.1109/TIV.2024.3392753
Jingjing Fan;Lili Fan;Qinghua Ni;Junhao Wang;Yi Liu;Ren Li;Yutong Wang;Sanjin Wang
In extreme off-road scenarios, autonomous driving technology holds strategic significance for enhancing emergency rescue capabilities, reducing labor intensity, and mitigating safety risks. Challenges such as adverse conditions, complex terrains, unstable satellite signals, and lack of roads pose serious safety challenges for autonomous driving. This perspective first delves into a Bird's Eye View (BEV)-based perception-planning framework, aiming to enhance the adaptability of intelligent vehicles to their environment. Subsequently, this perspective further discusses key issues such as Cyber-Physical-Social Systems (CPSS), foundation models, and technologies like Sora for off-road scenario generation, vehicle cognitive enhancement, and autonomous decision-making. Ultimately, the discussed framework is poised to endow intelligent vehicles with the capability to perform challenging tasks in complex off-road scenarios, realizing a more efficient, safer, and sustainable transportation system, thereby providing better support for the low-altitude economy.
在极端越野场景中，自动驾驶技术对于提高应急救援能力、降低劳动强度和减少安全风险具有重要的战略意义。恶劣的条件、复杂的地形、不稳定的卫星信号、缺乏道路等挑战给自动驾驶带来了严峻的安全挑战。本视角首先深入探讨了基于鸟瞰（BEV）的感知规划框架，旨在增强智能车辆对环境的适应性。随后，本视角进一步讨论了一些关键问题，如网络-物理-社会系统（CPSS）、基础模型以及用于越野场景生成、车辆认知增强和自主决策的 Sora 等技术。最终，所讨论的框架将赋予智能车辆在复杂越野场景中执行挑战性任务的能力，实现更高效、更安全和可持续的交通系统，从而为低空经济提供更好的支持。
{"title":"Perception and Planning of Intelligent Vehicles Based on BEV in Extreme Off-Road Scenarios","authors":"Jingjing Fan;Lili Fan;Qinghua Ni;Junhao Wang;Yi Liu;Ren Li;Yutong Wang;Sanjin Wang","doi":"10.1109/TIV.2024.3392753","DOIUrl":"https://doi.org/10.1109/TIV.2024.3392753","url":null,"abstract":"In extreme off-road scenarios, autonomous driving technology holds strategic significance for enhancing emergency rescue capabilities, reducing labor intensity, and mitigating safety risks. Challenges such as adverse conditions, complex terrains, unstable satellite signals, and lack of roads pose serious safety challenges for autonomous driving. This perspective first delves into a Bird's Eye View (BEV)-based perception-planning framework, aiming to enhance the adaptability of intelligent vehicles to their environment. Subsequently, this perspective further discusses key issues such as Cyber-Physical-Social Systems (CPSS), foundation models, and technologies like Sora for off-road scenario generation, vehicle cognitive enhancement, and autonomous decision-making. Ultimately, the discussed framework is poised to endow intelligent vehicles with the capability to perform challenging tasks in complex off-road scenarios, realizing a more efficient, safer, and sustainable transportation system, thereby providing better support for the low-altitude economy.","PeriodicalId":36532,"journal":{"name":"IEEE Transactions on Intelligent Vehicles","volume":"9 4","pages":"4568-4572"},"PeriodicalIF":8.2,"publicationDate":"2024-04-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141315236","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-17DOI: 10.1109/TIV.2024.3388726
Jiahang Li;Yikang Zhang;Peng Yun;Guangliang Zhou;Qijun Chen;Rui Fan
The recent advancements in deep convolutional neural networks have shown significant promise in the domain of road scene parsing. Nevertheless, the existing works focus primarily on freespace detection, with little attention given to hazardous road defects that could compromise both driving safety and comfort. In this article, we introduce RoadFormer, a novel Transformer-based data-fusion network developed for road scene parsing. RoadFormer utilizes a duplex encoder architecture to extract heterogeneous features from both RGB images and surface normal information. The encoded features are subsequently fed into a novel heterogeneous feature synergy block for effective feature fusion and recalibration. The pixel decoder then learns multi-scale long-range dependencies from the fused and recalibrated heterogeneous features, which are subsequently processed by a Transformer decoder to produce the final semantic prediction. Additionally, we release SYN-UDTIRI, the first large-scale road scene parsing dataset that contains over 10,407 RGB images, dense depth images, and the corresponding pixel-level annotations for both freespace and road defects of different shapes and sizes. Extensive experimental evaluations conducted on our SYN-UDTIRI dataset, as well as on three public datasets, including KITTI road, CityScapes, and ORFD, demonstrate that RoadFormer outperforms all other state-of-the-art networks for road scene parsing. Specifically, RoadFormer ranks first on the KITTI road benchmark.
{"title":"RoadFormer: Duplex Transformer for RGB-Normal Semantic Road Scene Parsing","authors":"Jiahang Li;Yikang Zhang;Peng Yun;Guangliang Zhou;Qijun Chen;Rui Fan","doi":"10.1109/TIV.2024.3388726","DOIUrl":"https://doi.org/10.1109/TIV.2024.3388726","url":null,"abstract":"The recent advancements in deep convolutional neural networks have shown significant promise in the domain of road scene parsing. Nevertheless, the existing works focus primarily on freespace detection, with little attention given to hazardous road defects that could compromise both driving safety and comfort. In this article, we introduce RoadFormer, a novel Transformer-based data-fusion network developed for road scene parsing. RoadFormer utilizes a duplex encoder architecture to extract heterogeneous features from both RGB images and surface normal information. The encoded features are subsequently fed into a novel heterogeneous feature synergy block for effective feature fusion and recalibration. The pixel decoder then learns multi-scale long-range dependencies from the fused and recalibrated heterogeneous features, which are subsequently processed by a Transformer decoder to produce the final semantic prediction. Additionally, we release SYN-UDTIRI, the first large-scale road scene parsing dataset that contains over 10,407 RGB images, dense depth images, and the corresponding pixel-level annotations for both freespace and road defects of different shapes and sizes. Extensive experimental evaluations conducted on our SYN-UDTIRI dataset, as well as on three public datasets, including KITTI road, CityScapes, and ORFD, demonstrate that RoadFormer outperforms all other state-of-the-art networks for road scene parsing. Specifically, RoadFormer ranks first on the KITTI road benchmark.","PeriodicalId":36532,"journal":{"name":"IEEE Transactions on Intelligent Vehicles","volume":"9 7","pages":"5163-5172"},"PeriodicalIF":14.0,"publicationDate":"2024-04-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142320486","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-11DOI: 10.1109/TIV.2024.3387405
Jiangcheng Su;Hailong Huang;Hong Zhang;Yutong Wang;Fei-Yue Wang
Electric Vertical Takeoff and Landing (eVTOL) aircraft has gained significant attention as a basic element of urban air mobility (UAM), a potential solution for urban transportation challenges using low-altitude urban airspace. Ensuring the safe operation of eVTOL is crucial for UAM applications, which are related to various professional fields such as aerodynamics, control, structures, and power systems. This article systematically analyzes the characteristics of different design configurations, including multi-rotor, lift+cruise, and tilt-rotor types of eVTOL. The advantages and limitations of each type of eVTOL are analyzed. After that, the overall design problems are analyzed, and challenges of eVTOL control system design are discussed from aspects of overall control structure and subsystems, such as controller, sensors, actuators, and command generator. This article tries to fill the gap in the eVTOL design from a control perspective and provides some resolutions for the eVTOL application.
{"title":"eVTOL Performance Analysis: A Review From Control Perspectives","authors":"Jiangcheng Su;Hailong Huang;Hong Zhang;Yutong Wang;Fei-Yue Wang","doi":"10.1109/TIV.2024.3387405","DOIUrl":"https://doi.org/10.1109/TIV.2024.3387405","url":null,"abstract":"Electric Vertical Takeoff and Landing (eVTOL) aircraft has gained significant attention as a basic element of urban air mobility (UAM), a potential solution for urban transportation challenges using low-altitude urban airspace. Ensuring the safe operation of eVTOL is crucial for UAM applications, which are related to various professional fields such as aerodynamics, control, structures, and power systems. This article systematically analyzes the characteristics of different design configurations, including multi-rotor, lift+cruise, and tilt-rotor types of eVTOL. The advantages and limitations of each type of eVTOL are analyzed. After that, the overall design problems are analyzed, and challenges of eVTOL control system design are discussed from aspects of overall control structure and subsystems, such as controller, sensors, actuators, and command generator. This article tries to fill the gap in the eVTOL design from a control perspective and provides some resolutions for the eVTOL application.","PeriodicalId":36532,"journal":{"name":"IEEE Transactions on Intelligent Vehicles","volume":"9 5","pages":"4877-4889"},"PeriodicalIF":14.0,"publicationDate":"2024-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141964752","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：