The Embedded VGG-Net Video Stream Processing Framework for Real-Time Classification of Cutting Volume at Shale Shaker

Abstract

A deep learning framework is proposed and implemented for monitoring the cutting volumes of a shaker in real-time on a deep-water drilling rig. The framework aims at performing classification and quantification with a live video streaming. Compared to the traditional video analytics method that is time-consuming, the proposed framework is more efficient and can be implemented for a real-time video analysis application. The real-time deep learning video analysis model consists of two parts for processing. The first part is a multi-thread video processing engine. A modularized service named Rig-Site Virtual Presence (RSVP) provides real-time video streaming from the rig. The multi-thread video processing engine implements real-time decoding, preprocessing and encoding of the video stream. The second part is a customized deep classification model. Based on the deep neural network (DNN), we implement the following adaptations: 1) Applied whitening and instance normalization to video frames; 2) Optimized the number of convolutional layers and the number of nodes in fully-connected layers; 3) Applied L2-norm regularization. The customized model is embedded in the multi-thread video processing engine, which ensures the capability for the real-time inference. The deep learning model categorizes every video frame into "ExtraHeavy", "Heavy", "Light" or "None". The model also outputs the corresponding numerical probabilities of each outcome. The training of the model is accomplished on a Nvidia GeForce 1070 GPU using the video stream with 137Kbps bitrate, 5.84 frames/s, and a frame size of 720×486. With only a common CPU support, the inference of the pre-trained model can be conducted in real-time. Both labeled frames and numerical results will be saved for later examination. Compared to the manual labeling results, the proposed deep learning framework achieves very promising results for analyzing video streaming in real-time.