Accelerate Multi-view Inference with End-edge Collaborative Computing

Abstract

Multi-view inference can utilize visual information from several views like a human being and significantly improve accuracy in some scenes, but it inevitably incurs more computing overhead than traditional DNN inference. To meet the requirement of low latency in typical scenarios, we consider utilizing model partition technique of edge computing to speed up multi-view inference, and design a multi-view end-edge co-inference execution framework (MV-IEF) which can make use of both end and edge resources for multi-view inference tasks. However, when employing the framework simply, the efficiency of multi-view inference will be constrained by network dynamics and heterogeneity of devices corresponding to multiple views. To break this constraint, we establish an optimization model based on the framework to minimize the multi-view inference time and solve it on the basis of game theory. And meanwhile, we propose a joint optimization algorithm for multi-view resource allocation and model partition (MV-JRAMP), which can make remarkable decisions of resource allocation and model partiton according to network status and computing capabilities of devices. Finally, we build a prototype and evaluate the performance of MV-JRAMP. Experiments show that MV-JRAMP can accelerate multi-view inference by up to 3.71×.