Mohit K. Sharma;Ibrahim Farhat;Chen-Feng Liu;Nassim Sehad;Wassim Hamidouche;Mérouane Debbah
{"title":"实时沉浸式空中视频流:全面调查、基准测试和公开挑战","authors":"Mohit K. Sharma;Ibrahim Farhat;Chen-Feng Liu;Nassim Sehad;Wassim Hamidouche;Mérouane Debbah","doi":"10.1109/OJCOMS.2024.3455763","DOIUrl":null,"url":null,"abstract":"Over the past decade, the use of Unmanned Aerial Vehicles (UAVs) has grown significantly due to their agility, maneuverability, and rapid deployability. An important application is the use of UAV-mounted 360-degree cameras for real-time streaming of Omnidirectional Videos (ODVs), enabling immersive experiences with up to six Degrees-of-freedom (6DoF) for applications like remote surveillance and gaming. However, streaming high-resolution ODVs with low latency (below 1 second) over an air-to-ground (A2G) wireless channel faces challenges due to its inherent non-stationarity, impacting the Quality-of-experience (QoE). Limited onboard energy availability and energy consumption variability based on flight parameters add to the complexity. This paper conducts a thorough survey of challenges and research efforts in UAV-based immersive video streaming. First, we outline the end-to-end 360-degree video transmission pipeline, covering coding, packaging, and streaming with a focus on standardization for device and service interoperability. Next, we review the research on optimizing video streaming over UAV-to-ground wireless channels, and present a real testbed demonstrating 360-degree video streaming from a UAV with remote control over a 5G network. To assess performance, a high-resolution 360-degree video dataset captured from UAVs under different conditions is introduced. Encoding schemes like AVC/H.264, HEVC/H.265, VVC/H.266, VP9, and AV1 are evaluated for encoding latency and QoE. Results show that HEVC‘s hardware implementation achieves a good QoE-latency trade-off, while AV1’s software implementation provides superior QoE. The paper concludes with discussions on open challenges and future directions for efficient and low-latency immersive video streaming via UAVs.","PeriodicalId":33803,"journal":{"name":"IEEE Open Journal of the Communications Society","volume":"5 ","pages":"5680-5705"},"PeriodicalIF":6.3000,"publicationDate":"2024-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10668820","citationCount":"0","resultStr":"{\"title\":\"Real-Time Immersive Aerial Video Streaming: A Comprehensive Survey, Benchmarking, and Open Challenges\",\"authors\":\"Mohit K. Sharma;Ibrahim Farhat;Chen-Feng Liu;Nassim Sehad;Wassim Hamidouche;Mérouane Debbah\",\"doi\":\"10.1109/OJCOMS.2024.3455763\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Over the past decade, the use of Unmanned Aerial Vehicles (UAVs) has grown significantly due to their agility, maneuverability, and rapid deployability. An important application is the use of UAV-mounted 360-degree cameras for real-time streaming of Omnidirectional Videos (ODVs), enabling immersive experiences with up to six Degrees-of-freedom (6DoF) for applications like remote surveillance and gaming. However, streaming high-resolution ODVs with low latency (below 1 second) over an air-to-ground (A2G) wireless channel faces challenges due to its inherent non-stationarity, impacting the Quality-of-experience (QoE). Limited onboard energy availability and energy consumption variability based on flight parameters add to the complexity. This paper conducts a thorough survey of challenges and research efforts in UAV-based immersive video streaming. First, we outline the end-to-end 360-degree video transmission pipeline, covering coding, packaging, and streaming with a focus on standardization for device and service interoperability. Next, we review the research on optimizing video streaming over UAV-to-ground wireless channels, and present a real testbed demonstrating 360-degree video streaming from a UAV with remote control over a 5G network. To assess performance, a high-resolution 360-degree video dataset captured from UAVs under different conditions is introduced. Encoding schemes like AVC/H.264, HEVC/H.265, VVC/H.266, VP9, and AV1 are evaluated for encoding latency and QoE. Results show that HEVC‘s hardware implementation achieves a good QoE-latency trade-off, while AV1’s software implementation provides superior QoE. The paper concludes with discussions on open challenges and future directions for efficient and low-latency immersive video streaming via UAVs.\",\"PeriodicalId\":33803,\"journal\":{\"name\":\"IEEE Open Journal of the Communications Society\",\"volume\":\"5 \",\"pages\":\"5680-5705\"},\"PeriodicalIF\":6.3000,\"publicationDate\":\"2024-09-06\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10668820\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Open Journal of the Communications Society\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10668820/\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ENGINEERING, ELECTRICAL & ELECTRONIC\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Open Journal of the Communications Society","FirstCategoryId":"1085","ListUrlMain":"https://ieeexplore.ieee.org/document/10668820/","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
Real-Time Immersive Aerial Video Streaming: A Comprehensive Survey, Benchmarking, and Open Challenges
Over the past decade, the use of Unmanned Aerial Vehicles (UAVs) has grown significantly due to their agility, maneuverability, and rapid deployability. An important application is the use of UAV-mounted 360-degree cameras for real-time streaming of Omnidirectional Videos (ODVs), enabling immersive experiences with up to six Degrees-of-freedom (6DoF) for applications like remote surveillance and gaming. However, streaming high-resolution ODVs with low latency (below 1 second) over an air-to-ground (A2G) wireless channel faces challenges due to its inherent non-stationarity, impacting the Quality-of-experience (QoE). Limited onboard energy availability and energy consumption variability based on flight parameters add to the complexity. This paper conducts a thorough survey of challenges and research efforts in UAV-based immersive video streaming. First, we outline the end-to-end 360-degree video transmission pipeline, covering coding, packaging, and streaming with a focus on standardization for device and service interoperability. Next, we review the research on optimizing video streaming over UAV-to-ground wireless channels, and present a real testbed demonstrating 360-degree video streaming from a UAV with remote control over a 5G network. To assess performance, a high-resolution 360-degree video dataset captured from UAVs under different conditions is introduced. Encoding schemes like AVC/H.264, HEVC/H.265, VVC/H.266, VP9, and AV1 are evaluated for encoding latency and QoE. Results show that HEVC‘s hardware implementation achieves a good QoE-latency trade-off, while AV1’s software implementation provides superior QoE. The paper concludes with discussions on open challenges and future directions for efficient and low-latency immersive video streaming via UAVs.
期刊介绍:
The IEEE Open Journal of the Communications Society (OJ-COMS) is an open access, all-electronic journal that publishes original high-quality manuscripts on advances in the state of the art of telecommunications systems and networks. The papers in IEEE OJ-COMS are included in Scopus. Submissions reporting new theoretical findings (including novel methods, concepts, and studies) and practical contributions (including experiments and development of prototypes) are welcome. Additionally, survey and tutorial articles are considered. The IEEE OJCOMS received its debut impact factor of 7.9 according to the Journal Citation Reports (JCR) 2023.
The IEEE Open Journal of the Communications Society covers science, technology, applications and standards for information organization, collection and transfer using electronic, optical and wireless channels and networks. Some specific areas covered include:
Systems and network architecture, control and management
Protocols, software, and middleware
Quality of service, reliability, and security
Modulation, detection, coding, and signaling
Switching and routing
Mobile and portable communications
Terminals and other end-user devices
Networks for content distribution and distributed computing
Communications-based distributed resources control.