Towards Executing Computer Vision Functionality on Programmable Network Devices

By offering the possibility to already perform processing as packets traverse the network, programmable data planes open up new perspectives for applications suffering from strict latency and high bandwidth requirements. Real-time Computer Vision (CV), with its high data rates and often mission- and safety-critical roles in the control of autonomous vehicles and industrial machinery, could particularly benefit from executing parts of its logic within network elements. In this paper, we thus explore what it takes to bring CV to the network. We present our work-in-progress efforts of implementing a line-following algorithm based on convolution filters on a P4-programmable NIC. We find that by appropriately identifying regions of interest in the image data and by diligently distributing the necessary calculations among the various match/action stages of the ingress- and egress pipelines of the NIC, our prototypical implementation can achieve over 19 decisions per second on 640x480 px grayscale images with filters large enough to guide a small autonomous car through various courses.