Fast hardware assisted online learning using unsupervised deep learning structure for anomaly detection

Real-time scenarios of deep learning algorithms are challenged by two less frequently addressed issues. The first is data inefficiency i.e., the model requires several epochs of trial and error to converge which makes it impractical to be applied to real-time applications. The Second is the high precision computation load of the deep learning algorithms to achieve high accuracy during training and inference. In this paper, we address these two issues and apply our model to the task of online anomaly detection using FPGA. To address the first issue, we propose a compressed training model for the contrastive divergence algorithm (CD) in the Deep Belief Network (DBN). The goal is to dynamically adjust the training vector according to the feedback from the free energy and the reconstruction error, which allows for better generalization. To address the second issue, we propose a Hybrid-Stochastic-Dynamic-Fixed-Point (HSDFP) method, which provides training environment with high reduction in calculation, area, and power in FPGA. Our framework enables the DBN structure to take actions and detect attacks online. Thus, the network can collect efficient number of training samples and avoid overfitting. We show that (1) our proposed method converges faster than the state-of-the-art deep learning methods, (2) FPGA implementation achieves accelerated inference speed of 0.008ms and a high power efficiency of 37 G-ops/s/W compared to CPU, GPU, and 16-bit fixed-point arithmetic (3) FPGA also achieves minimal degradation in accuracy of 95%, 95.4%, and 97.9% on the benchmark datasets: MNIST, NSL-KDD, and Kyoto datasets.