实时沉浸式空中视频流:全面调查、基准测试和公开挑战

IF 6.3 Q1 ENGINEERING, ELECTRICAL & ELECTRONIC IEEE Open Journal of the Communications Society Pub Date : 2024-09-06 DOI:10.1109/OJCOMS.2024.3455763
Mohit K. Sharma;Ibrahim Farhat;Chen-Feng Liu;Nassim Sehad;Wassim Hamidouche;Mérouane Debbah
{"title":"实时沉浸式空中视频流:全面调查、基准测试和公开挑战","authors":"Mohit K. Sharma;Ibrahim Farhat;Chen-Feng Liu;Nassim Sehad;Wassim Hamidouche;Mérouane Debbah","doi":"10.1109/OJCOMS.2024.3455763","DOIUrl":null,"url":null,"abstract":"Over the past decade, the use of Unmanned Aerial Vehicles (UAVs) has grown significantly due to their agility, maneuverability, and rapid deployability. An important application is the use of UAV-mounted 360-degree cameras for real-time streaming of Omnidirectional Videos (ODVs), enabling immersive experiences with up to six Degrees-of-freedom (6DoF) for applications like remote surveillance and gaming. However, streaming high-resolution ODVs with low latency (below 1 second) over an air-to-ground (A2G) wireless channel faces challenges due to its inherent non-stationarity, impacting the Quality-of-experience (QoE). Limited onboard energy availability and energy consumption variability based on flight parameters add to the complexity. This paper conducts a thorough survey of challenges and research efforts in UAV-based immersive video streaming. First, we outline the end-to-end 360-degree video transmission pipeline, covering coding, packaging, and streaming with a focus on standardization for device and service interoperability. Next, we review the research on optimizing video streaming over UAV-to-ground wireless channels, and present a real testbed demonstrating 360-degree video streaming from a UAV with remote control over a 5G network. To assess performance, a high-resolution 360-degree video dataset captured from UAVs under different conditions is introduced. Encoding schemes like AVC/H.264, HEVC/H.265, VVC/H.266, VP9, and AV1 are evaluated for encoding latency and QoE. Results show that HEVC‘s hardware implementation achieves a good QoE-latency trade-off, while AV1’s software implementation provides superior QoE. The paper concludes with discussions on open challenges and future directions for efficient and low-latency immersive video streaming via UAVs.","PeriodicalId":33803,"journal":{"name":"IEEE Open Journal of the Communications Society","volume":"5 ","pages":"5680-5705"},"PeriodicalIF":6.3000,"publicationDate":"2024-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10668820","citationCount":"0","resultStr":"{\"title\":\"Real-Time Immersive Aerial Video Streaming: A Comprehensive Survey, Benchmarking, and Open Challenges\",\"authors\":\"Mohit K. Sharma;Ibrahim Farhat;Chen-Feng Liu;Nassim Sehad;Wassim Hamidouche;Mérouane Debbah\",\"doi\":\"10.1109/OJCOMS.2024.3455763\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Over the past decade, the use of Unmanned Aerial Vehicles (UAVs) has grown significantly due to their agility, maneuverability, and rapid deployability. An important application is the use of UAV-mounted 360-degree cameras for real-time streaming of Omnidirectional Videos (ODVs), enabling immersive experiences with up to six Degrees-of-freedom (6DoF) for applications like remote surveillance and gaming. However, streaming high-resolution ODVs with low latency (below 1 second) over an air-to-ground (A2G) wireless channel faces challenges due to its inherent non-stationarity, impacting the Quality-of-experience (QoE). Limited onboard energy availability and energy consumption variability based on flight parameters add to the complexity. This paper conducts a thorough survey of challenges and research efforts in UAV-based immersive video streaming. First, we outline the end-to-end 360-degree video transmission pipeline, covering coding, packaging, and streaming with a focus on standardization for device and service interoperability. Next, we review the research on optimizing video streaming over UAV-to-ground wireless channels, and present a real testbed demonstrating 360-degree video streaming from a UAV with remote control over a 5G network. To assess performance, a high-resolution 360-degree video dataset captured from UAVs under different conditions is introduced. Encoding schemes like AVC/H.264, HEVC/H.265, VVC/H.266, VP9, and AV1 are evaluated for encoding latency and QoE. Results show that HEVC‘s hardware implementation achieves a good QoE-latency trade-off, while AV1’s software implementation provides superior QoE. The paper concludes with discussions on open challenges and future directions for efficient and low-latency immersive video streaming via UAVs.\",\"PeriodicalId\":33803,\"journal\":{\"name\":\"IEEE Open Journal of the Communications Society\",\"volume\":\"5 \",\"pages\":\"5680-5705\"},\"PeriodicalIF\":6.3000,\"publicationDate\":\"2024-09-06\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10668820\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Open Journal of the Communications Society\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10668820/\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ENGINEERING, ELECTRICAL & ELECTRONIC\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Open Journal of the Communications Society","FirstCategoryId":"1085","ListUrlMain":"https://ieeexplore.ieee.org/document/10668820/","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
引用次数: 0

摘要

在过去的十年中,无人飞行器(UAV)因其灵活性、机动性和快速部署性而得到了广泛应用。一个重要的应用是使用安装在无人机上的 360 度摄像头实时流式传输全方位视频(ODV),为远程监控和游戏等应用提供高达六自由度(6DoF)的身临其境体验。然而,在空对地(A2G)无线信道上以低延迟(低于 1 秒)流式传输高分辨率 ODV 面临着挑战,因为其固有的非稳定性会影响体验质量(QoE)。有限的机载能源可用性和基于飞行参数的能源消耗可变性增加了问题的复杂性。本文对基于无人机的沉浸式视频流所面临的挑战和研究工作进行了深入调查。首先,我们概述了端到端的 360 度视频传输管道,包括编码、包装和流媒体,重点是设备和服务互操作性的标准化。接下来,我们回顾了关于优化无人机到地面无线信道视频流的研究,并展示了一个真实的测试平台,演示了通过 5G 网络从无人机远程控制 360 度视频流。为了评估性能,介绍了在不同条件下从无人机捕获的高分辨率 360 度视频数据集。对 AVC/H.264、HEVC/H.265、VVC/H.266、VP9 和 AV1 等编码方案的编码延迟和 QoE 进行了评估。结果表明,HEVC 的硬件实现在 QoE 和延迟之间实现了良好的权衡,而 AV1 的软件实现则提供了更优越的 QoE。论文最后讨论了通过无人机进行高效、低延迟沉浸式视频流传输所面临的挑战和未来发展方向。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Real-Time Immersive Aerial Video Streaming: A Comprehensive Survey, Benchmarking, and Open Challenges
Over the past decade, the use of Unmanned Aerial Vehicles (UAVs) has grown significantly due to their agility, maneuverability, and rapid deployability. An important application is the use of UAV-mounted 360-degree cameras for real-time streaming of Omnidirectional Videos (ODVs), enabling immersive experiences with up to six Degrees-of-freedom (6DoF) for applications like remote surveillance and gaming. However, streaming high-resolution ODVs with low latency (below 1 second) over an air-to-ground (A2G) wireless channel faces challenges due to its inherent non-stationarity, impacting the Quality-of-experience (QoE). Limited onboard energy availability and energy consumption variability based on flight parameters add to the complexity. This paper conducts a thorough survey of challenges and research efforts in UAV-based immersive video streaming. First, we outline the end-to-end 360-degree video transmission pipeline, covering coding, packaging, and streaming with a focus on standardization for device and service interoperability. Next, we review the research on optimizing video streaming over UAV-to-ground wireless channels, and present a real testbed demonstrating 360-degree video streaming from a UAV with remote control over a 5G network. To assess performance, a high-resolution 360-degree video dataset captured from UAVs under different conditions is introduced. Encoding schemes like AVC/H.264, HEVC/H.265, VVC/H.266, VP9, and AV1 are evaluated for encoding latency and QoE. Results show that HEVC‘s hardware implementation achieves a good QoE-latency trade-off, while AV1’s software implementation provides superior QoE. The paper concludes with discussions on open challenges and future directions for efficient and low-latency immersive video streaming via UAVs.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
CiteScore
13.70
自引率
3.80%
发文量
94
审稿时长
10 weeks
期刊介绍: The IEEE Open Journal of the Communications Society (OJ-COMS) is an open access, all-electronic journal that publishes original high-quality manuscripts on advances in the state of the art of telecommunications systems and networks. The papers in IEEE OJ-COMS are included in Scopus. Submissions reporting new theoretical findings (including novel methods, concepts, and studies) and practical contributions (including experiments and development of prototypes) are welcome. Additionally, survey and tutorial articles are considered. The IEEE OJCOMS received its debut impact factor of 7.9 according to the Journal Citation Reports (JCR) 2023. The IEEE Open Journal of the Communications Society covers science, technology, applications and standards for information organization, collection and transfer using electronic, optical and wireless channels and networks. Some specific areas covered include: Systems and network architecture, control and management Protocols, software, and middleware Quality of service, reliability, and security Modulation, detection, coding, and signaling Switching and routing Mobile and portable communications Terminals and other end-user devices Networks for content distribution and distributed computing Communications-based distributed resources control.
期刊最新文献
vFFR: A Very Fast Failure Recovery Strategy Implemented in Devices With Programmable Data Plane Scalable High-Throughput and Low-Latency DVB-S2(x) LDPC Decoders on SIMD Devices Service Continuity in Edge Computing Through Edge Proxies and HTTP Alternative Services Delay Guarantees for a Swarm of Mobile Sensors in Safety-Critical Applications Distributed Massive MIMO for Wireless Power Transfer in the Industrial Internet of Things
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1