We propose a VR video conferencing system over named data networks (NDN). The system is designed to support real-time, multi-party streaming and playback of 360 degree video on a web player. A centralized architecture is used, with a signaling server to coordinate multiple participants. To ensure real-time requirement, a protocol featuring prefetching is used for producer-consumer communication. Along with the native support of multicast in NDN, this design is expected to better support large amount of data streaming between multiple users. As a proof of concept, a protoype of the system is implemented with one-way real-time 360 video streaming. Experiments show that seamless streaming and interactive playback of 360 video can be achieved with low latency. Therefore, the proposed system has the potential to provide immersive VR experience for real-time multi-party video conferencing.
{"title":"VR Video Conferencing over Named Data Networks","authors":"Liyang Zhang, S. O. Amin, C. Westphal","doi":"10.1145/3097895.3097897","DOIUrl":"https://doi.org/10.1145/3097895.3097897","url":null,"abstract":"We propose a VR video conferencing system over named data networks (NDN). The system is designed to support real-time, multi-party streaming and playback of 360 degree video on a web player. A centralized architecture is used, with a signaling server to coordinate multiple participants. To ensure real-time requirement, a protocol featuring prefetching is used for producer-consumer communication. Along with the native support of multicast in NDN, this design is expected to better support large amount of data streaming between multiple users. As a proof of concept, a protoype of the system is implemented with one-way real-time 360 video streaming. Experiments show that seamless streaming and interactive playback of 360 video can be achieved with low latency. Therefore, the proposed system has the potential to provide immersive VR experience for real-time multi-party video conferencing.","PeriodicalId":270981,"journal":{"name":"Proceedings of the Workshop on Virtual Reality and Augmented Reality Network","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125485746","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Simone Mangiante, G. Klas, Amit Navon, G. Zhuang, Ran Ju, M. F. Silva
VR/AR is rapidly progressing towards enterprise and end customers with the promise of bringing immersive experience to numerous applications. Soon it will target smartphones from the cloud and 360° video delivery will need unprecedented requirements for ultra-low latency and ultra-high throughput to mobile networks. Latest developments in NFV and Mobile Edge Computing reveal already the potential to enable VR streaming in cellular networks and to pave the way towards 5G and next stages in VR technology. In this paper we present a Field Of View (FOV) rendering solution at the edge of a mobile network, designed to optimize the bandwidth and latency required by VR 360° video streaming. Preliminary test results show the immediate benefits in bandwidth saving this approach can provide and generate new directions for VR/AR network research.
{"title":"VR is on the Edge: How to Deliver 360° Videos in Mobile Networks","authors":"Simone Mangiante, G. Klas, Amit Navon, G. Zhuang, Ran Ju, M. F. Silva","doi":"10.1145/3097895.3097901","DOIUrl":"https://doi.org/10.1145/3097895.3097901","url":null,"abstract":"VR/AR is rapidly progressing towards enterprise and end customers with the promise of bringing immersive experience to numerous applications. Soon it will target smartphones from the cloud and 360° video delivery will need unprecedented requirements for ultra-low latency and ultra-high throughput to mobile networks. Latest developments in NFV and Mobile Edge Computing reveal already the potential to enable VR streaming in cellular networks and to pave the way towards 5G and next stages in VR technology. In this paper we present a Field Of View (FOV) rendering solution at the edge of a mobile network, designed to optimize the bandwidth and latency required by VR 360° video streaming. Preliminary test results show the immediate benefits in bandwidth saving this approach can provide and generate new directions for VR/AR network research.","PeriodicalId":270981,"journal":{"name":"Proceedings of the Workshop on Virtual Reality and Augmented Reality Network","volume":"165 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116430617","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Fanyi Duanmu, Eymen Kurdoglu, S. A. Hosseini, Yong Liu, Yao Wang
360 degree video compression and streaming is one of the key components of Virtual Reality (VR) applications. In 360 video streaming, a user may freely navigate through the captured 3D environment by changing her desired viewing direction. Only a small portion of the entire 360 degree video is watched at any time. Streaming the entire 360 degree raw video is therefore unnecessary and bandwidth-consuming. One the other hand, only streaming the video in the predicted user's view direction will introduce streaming discontinuity whenever the the prediction is wrong. In this work, a two-tier 360 video streaming framework with prioritized buffer control is proposed to effectively accommodate the dynamics in both network bandwidth and viewing direction. Through simulations driven by real network bandwidth and viewing direction traces, we demonstrate that the proposed framework can significantly outperform the conventional 360 video streaming solutions.
{"title":"Prioritized Buffer Control in Two-tier 360 Video Streaming","authors":"Fanyi Duanmu, Eymen Kurdoglu, S. A. Hosseini, Yong Liu, Yao Wang","doi":"10.1145/3097895.3097898","DOIUrl":"https://doi.org/10.1145/3097895.3097898","url":null,"abstract":"360 degree video compression and streaming is one of the key components of Virtual Reality (VR) applications. In 360 video streaming, a user may freely navigate through the captured 3D environment by changing her desired viewing direction. Only a small portion of the entire 360 degree video is watched at any time. Streaming the entire 360 degree raw video is therefore unnecessary and bandwidth-consuming. One the other hand, only streaming the video in the predicted user's view direction will introduce streaming discontinuity whenever the the prediction is wrong. In this work, a two-tier 360 video streaming framework with prioritized buffer control is proposed to effectively accommodate the dynamics in both network bandwidth and viewing direction. Through simulations driven by real network bandwidth and viewing direction traces, we demonstrate that the proposed framework can significantly outperform the conventional 360 video streaming solutions.","PeriodicalId":270981,"journal":{"name":"Proceedings of the Workshop on Virtual Reality and Augmented Reality Network","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123824033","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper, we conduct a reality check for Augmented Reality (AR) on mobile devices. We dissect and measure the cloud-offloading feature for computation-intensive visual tasks of two popular commercial AR systems. Our key finding is that their cloud-based recognition is still not mature and not optimized for latency, data usage and energy consumption. In order to identify the opportunities for further improving the Quality of Experience (QoE) for mobile AR, we break down the end-to-end latency of the pipeline for typical cloud-based mobile AR and pinpoint the dominating components in the critical path.
{"title":"On the Networking Challenges of Mobile Augmented Reality","authors":"Wenxiao Zhang, B. Han, P. Hui","doi":"10.1145/3097895.3097900","DOIUrl":"https://doi.org/10.1145/3097895.3097900","url":null,"abstract":"In this paper, we conduct a reality check for Augmented Reality (AR) on mobile devices. We dissect and measure the cloud-offloading feature for computation-intensive visual tasks of two popular commercial AR systems. Our key finding is that their cloud-based recognition is still not mature and not optimized for latency, data usage and energy consumption. In order to identify the opportunities for further improving the Quality of Experience (QoE) for mobile AR, we break down the end-to-end latency of the pipeline for typical cloud-based mobile AR and pinpoint the dominating components in the critical path.","PeriodicalId":270981,"journal":{"name":"Proceedings of the Workshop on Virtual Reality and Augmented Reality Network","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114253682","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ran Ju, Jun He, Fengxin Sun, Jin Li, Feng Li, Jirong Zhu, Lei Han
Online VR streaming faces great challenges such as the high throughput and real time interaction requirement. In this paper, we propose a novel ultra wide view based method to stream high quality VR on Internet at low bandwidth and little computation cost. First, we only transmit the region where user is looking at instead of full 360° view to save bandwidth. To achieve this goal, we split the source VR into small grid videos in advance. The grid videos are able to reconstruct any view flexibly in user end. Second, according to the fact that users generally interact at low speed, we expand the view that user requested to meet the real time interaction requirement. Besides, a low resolution full view stream is supplied to handle exceptional cases such as high speed view change. We test our solution in an experimental network. The results show remarkable bandwidth saving of over 60% in average at little computation cost while supplying the same quality of experience as local VR.
{"title":"Ultra Wide View Based Panoramic VR Streaming","authors":"Ran Ju, Jun He, Fengxin Sun, Jin Li, Feng Li, Jirong Zhu, Lei Han","doi":"10.1145/3097895.3097899","DOIUrl":"https://doi.org/10.1145/3097895.3097899","url":null,"abstract":"Online VR streaming faces great challenges such as the high throughput and real time interaction requirement. In this paper, we propose a novel ultra wide view based method to stream high quality VR on Internet at low bandwidth and little computation cost. First, we only transmit the region where user is looking at instead of full 360° view to save bandwidth. To achieve this goal, we split the source VR into small grid videos in advance. The grid videos are able to reconstruct any view flexibly in user end. Second, according to the fact that users generally interact at low speed, we expand the view that user requested to meet the real time interaction requirement. Besides, a low resolution full view stream is supplied to handle exceptional cases such as high speed view change. We test our solution in an experimental network. The results show remarkable bandwidth saving of over 60% in average at little computation cost while supplying the same quality of experience as local VR.","PeriodicalId":270981,"journal":{"name":"Proceedings of the Workshop on Virtual Reality and Augmented Reality Network","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121975449","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Deep learning has the potential to make Augmented Reality (AR) devices smarter, but few AR apps use such technology today because it is compute-intensive, and front-end devices cannot deliver sufficient compute power. We propose a distributed framework that ties together front-end devices with more powerful back-end "helpers" that allow deep learning to be executed locally or to be offloaded. This framework should be able to intelligently use current estimates of network conditions and back-end server loads, in conjunction with the application's requirements, to determine an optimal strategy. This work reports our preliminary investigation in implementing such a framework, in which the front-end is assumed to be smartphones. Our specific contributions include: (1) development of an Android application that performs real-time object detection, either locally on the smartphone or remotely on a server; and (2) characterization of the tradeoffs between object detection accuracy, latency, and battery drain, based on the system parameters of video resolution, deep learning model size, and offloading decision.
{"title":"Delivering Deep Learning to Mobile Devices via Offloading","authors":"Xukan Ran, Haoliang Chen, Zhenming Liu, Jiasi Chen","doi":"10.1145/3097895.3097903","DOIUrl":"https://doi.org/10.1145/3097895.3097903","url":null,"abstract":"Deep learning has the potential to make Augmented Reality (AR) devices smarter, but few AR apps use such technology today because it is compute-intensive, and front-end devices cannot deliver sufficient compute power. We propose a distributed framework that ties together front-end devices with more powerful back-end \"helpers\" that allow deep learning to be executed locally or to be offloaded. This framework should be able to intelligently use current estimates of network conditions and back-end server loads, in conjunction with the application's requirements, to determine an optimal strategy. This work reports our preliminary investigation in implementing such a framework, in which the front-end is assumed to be smartphones. Our specific contributions include: (1) development of an Android application that performs real-time object detection, either locally on the smartphone or remotely on a server; and (2) characterization of the tradeoffs between object detection accuracy, latency, and battery drain, based on the system parameters of video resolution, deep learning model size, and offloading decision.","PeriodicalId":270981,"journal":{"name":"Proceedings of the Workshop on Virtual Reality and Augmented Reality Network","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114643661","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Online streaming of Virtual Reality and 360° videos is rapidly growing, as more and more major content providers and news outlets adopt the format to enrich the user experience. We characterize 360° videos by examining several thousand YouTube videos across more than a dozen categories. 360° videos, at first sight, seem to pose a challenge for the network to stream because of their substantially higher bit rates and larger number of resolutions. However, a careful examination of video characteristics reveals that there are significant opportunities for reducing the actual bit rate delivered to client devices based on the user's field of view. We study the bit rate and the motion in 360° videos, and compare them against regular videos by investigating several important metrics. We find that 360° videos are less variable in terms of bit rate, and have less motion than regular videos. Our expectation is that variability in the bit rates due to the motion of the camera in regular videos (or switching between cameras) is now translated to responsiveness requirements for end to end 360° streaming architectures.
{"title":"Characterization of 360-degree Videos","authors":"Shahryar Afzal, Jiasi Chen, K. Ramakrishnan","doi":"10.1145/3097895.3097896","DOIUrl":"https://doi.org/10.1145/3097895.3097896","url":null,"abstract":"Online streaming of Virtual Reality and 360° videos is rapidly growing, as more and more major content providers and news outlets adopt the format to enrich the user experience. We characterize 360° videos by examining several thousand YouTube videos across more than a dozen categories. 360° videos, at first sight, seem to pose a challenge for the network to stream because of their substantially higher bit rates and larger number of resolutions. However, a careful examination of video characteristics reveals that there are significant opportunities for reducing the actual bit rate delivered to client devices based on the user's field of view. We study the bit rate and the motion in 360° videos, and compare them against regular videos by investigating several important metrics. We find that 360° videos are less variable in terms of bit rate, and have less motion than regular videos. Our expectation is that variability in the bit rates due to the motion of the camera in regular videos (or switching between cameras) is now translated to responsiveness requirements for end to end 360° streaming architectures.","PeriodicalId":270981,"journal":{"name":"Proceedings of the Workshop on Virtual Reality and Augmented Reality Network","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127976334","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We study the delivery of 360°-navigable videos to 5G VR/AR wireless clients in future cooperative multi-cellular systems. A collection of small-cell base stations interconnected via back-haul links are sharing their caching and computing resources to maximize the aggregate reward they earn by serving 360° videos requested by VR/AR wireless clients. We design an efficient representation method to construct the 360° videos such that they only deliver the remote scene viewpoint content genuinely needed by the VR/AR user, thereby overcoming the present highly inefficient approach of sending a bulky 360° video, whose major part comprises scene information never accessed by the user. Moreover, we design an optimization framework that allows the base stations to select cooperative caching/rendering/streaming strategies that maximize the aggregate reward they earn when serving the users, for the given caching/computational resources at each base station. We formulate the problem of interest as integer programming, show its NP-hardness, and derive a fully-polynomial-time approximation solution with strong performance guarantees. Our advances demonstrate orders of magnitude operational efficiency gains over state-of-the-art caching and 360° video representation mechanisms and are very promising. This is a first-of-its-kind study to explore fundamental trade-offs between caching, computing, and communication for emerging VR/AR applications of broad societal impact.
{"title":"VR/AR Immersive Communication: Caching, Edge Computing, and Transmission Trade-Offs","authors":"Jacob Chakareski","doi":"10.1145/3097895.3097902","DOIUrl":"https://doi.org/10.1145/3097895.3097902","url":null,"abstract":"We study the delivery of 360°-navigable videos to 5G VR/AR wireless clients in future cooperative multi-cellular systems. A collection of small-cell base stations interconnected via back-haul links are sharing their caching and computing resources to maximize the aggregate reward they earn by serving 360° videos requested by VR/AR wireless clients. We design an efficient representation method to construct the 360° videos such that they only deliver the remote scene viewpoint content genuinely needed by the VR/AR user, thereby overcoming the present highly inefficient approach of sending a bulky 360° video, whose major part comprises scene information never accessed by the user. Moreover, we design an optimization framework that allows the base stations to select cooperative caching/rendering/streaming strategies that maximize the aggregate reward they earn when serving the users, for the given caching/computational resources at each base station. We formulate the problem of interest as integer programming, show its NP-hardness, and derive a fully-polynomial-time approximation solution with strong performance guarantees. Our advances demonstrate orders of magnitude operational efficiency gains over state-of-the-art caching and 360° video representation mechanisms and are very promising. This is a first-of-its-kind study to explore fundamental trade-offs between caching, computing, and communication for emerging VR/AR applications of broad societal impact.","PeriodicalId":270981,"journal":{"name":"Proceedings of the Workshop on Virtual Reality and Augmented Reality Network","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131503575","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Proceedings of the Workshop on Virtual Reality and Augmented Reality Network","authors":"","doi":"10.1145/3097895","DOIUrl":"https://doi.org/10.1145/3097895","url":null,"abstract":"","PeriodicalId":270981,"journal":{"name":"Proceedings of the Workshop on Virtual Reality and Augmented Reality Network","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123780366","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：