Distributed edge computing for cooperative augmented reality: enhancing mobile sensing capabilities

Abstract

Cooperative Augmented Reality (AR) can provide real-time, immersive, and context-aware situational awareness while enhancing mobile sensing capabilities and benefiting various applications. Distributed edge computing has emerged as an essential paradigm to facilitate cooperative AR. We designed and implemented a distributed system to enable fast, reliable, and scalable cooperative AR. In this paper, we present a novel approach and architecture that integrates advanced sensing, communications, and processing techniques to create such a cooperative AR system and demonstrate its capability with HoloLens and edge servers connected over a wireless network. Our research addresses the challenges of implementing a distributed cooperative AR system capable of capturing data from a multitude of sensors on HoloLens, performing fusion and accurate object recognition, and seamlessly projecting the reconstructed 3D model into the wearer’s field of view. The paper delves into the intricate architecture of the proposed cooperative AR system, detailing its distributed sensing and edge computing components, and the Apache Storm-integrated platform. The implementation encompasses data collection, aggregation, analysis, object recognition, and rendering of 3D models on the HoloLens, all in real-time. The proposed system enhances the AR experience while showcasing the vast potential of distributed edge computing. Our findings illustrate the feasibility and advantages of merging distributed cooperative sensing and edge computing to offer dynamic, immersive AR experiences, paving the way for new applications.