Reachability analysis of recurrent neural networks

Abstract

The paper proposes a new sparse star set representation and extends the recent star reachability method to verify the robustness of vanilla, long short-term memory (LSTM), and gated recurrent units (GRU) recurrent neural networks (RNNs) for safety-critical applications. RNNs are a popular machine learning method for various applications, but they are vulnerable to adversarial attacks, where slightly perturbing the input sequence can lead to an unexpected result. Recent notable techniques for verifying RNNs include unrolling and invariant inference approaches. The first method has scaling issues since unrolling an RNN creates a large feedforward neural network. The second method, using invariant sets, has better scalability but can produce unknown results due to the accumulation of over-approximation errors over time. This paper introduces a complementary verification method for RNNs that is both sound and complete. A relaxation parameter can be used to convert the method into a fast over-approximation method that still provides soundness guarantees. The vanilla RNN verification method is designed to be used with NNV, a tool for verifying deep neural networks and learning-enabled cyber–physical systems, while the verification approach of LSTM and GRU RNNs are implemented on StarV. Compared to state-of-the-art methods for verifying a vanilla RNN, the extended exact reachability method is $10\times$ faster, and the over-approximation method is $100\times$ to $5000\times$ faster. Although the sparse star set is slow compared to state-of-the-art methods, it was able to verify more robust cases in general than them.