Fault-tolerant deep learning inference on CPU-GPU integrated edge devices with TEEs

Abstract

CPU-GPU integrated edge devices and deep learning algorithms have received significant progress in recent years, leading to increasingly widespread application of edge intelligence. However, deep learning inference on these edge devices is vulnerable to Fault Injection Attacks (FIAs) that can modify device memory or execute instructions with errors. We propose DarkneTF, a Fault-Tolerant (FT) deep learning inference framework for CPU-GPU integrated edge devices, to ensure the correctness of model inference results by detecting the threat of FIAs. DarkneTF introduces algorithm-based verification to implement the FT deep learning inference. The verification process involves verifying the integrity of model weights and validating the correctness of time-intensive calculations, such as convolutions. We improve the Freivalds algorithm to enhance the ability to detect tiny perturbations by strengthening randomization. As the verification process is also susceptible to FIAs, DarkneTF offloads the verification process into Trusted Execution Environments (TEEs). This scheme ensures the verification process’s security and allows for accelerated model inference using the integrated GPUs. Experimental results show that GPU-accelerated FT inference on HiKey 960 achieves notable speedups ranging from 3.46x to 5.57x compared to FT inference on a standalone CPU. The extra memory overhead incurred FT inference remains at an exceedingly low level, with a range of 0.46% to 10.22%. The round-off error of the improved Freivalds algorithm is below $2.50\times 10^{-4}$, and the accuracy of detecting FIAs is above 92.73%.