Gain Without Pain: Enabling Real-time Environmental Perception on 2x Mobile Devices in Multiplayer Augmented Reality

Abstract

Mobile multiplayer augmented reality(AR) emerges in various applications including games, education, training, etc. Edge computing technique enables real-time environmental perception ability(e.g. object detection and segmentation) of devices by offloading complex computation to the nearby edge server. However, with more players involved and offloading video streams, bandwidth competition intensifies and lengthens the transmission latency, which severely impairs the accuracy of environmental perception in multiplayer AR applications. We realize staggering offloading period of each device can reduce transmission latency and maintain real-time environmental perception when more players involved. We propose Bonus containing two key techniques: collaborative offloading scheduler to eliminate bandwidth competition among multiple devices, which improve the performance of the overall system to achieve “Pareto Efficiency”; time-aware video reshaper based on mainstream 802.11 protocol to enable Bonus compatible with most wireless scenarios. we evaluate Bonus and four SOTA solutions across 24 videos on different environmental perception tasks. Results demonstrate that Bonus achieves 119.5% accuracy and 1.9x player capacity compared to the closest baseline under wireless environment.