Precog: prefetching for image recognition applications at the edge

Image recognition applications are on the rise. Increasingly, applications on edge devices such as mobile smartphones, drones and cars, are relying on recognition techniques to provide interactive and intelligent functionality. Given the complexity of these techniques, and resource constrained nature of edge devices, applications rely on offloading compute intensive recognition tasks to the cloud. This has also lead to the rise of cloud-based recognition services. This involves sending captured images to remote servers across the Internet, which leads to slower responses. With the rising numbers of edge devices, both, the network and such centralized cloud-based solutions, are likely to be under stress, and lead to further slower responses. To reduce the recognition latency, and provide better scalability to the cloud-based solutions, we propose Precog. Precog employs selective computation on the devices to reduce the need to offload images to the cloud. In coordination with edge servers, it uses prediction to prefetch parts of the trained classifiers used for recognition onto the devices, and uses these smaller models to accelerate recognition on devices. Our evaluation shows that Precog can reduce latency by up to 5×, better utilize edge and cloud resources and also increase accuracy. We believe that Precog is the first system to use devices and edge servers collaboratively to enable prefetching and caching on the devices, and drive down recognition latency for mobile applications.