Quantum theory-inspired inter-sentence semantic interaction model for textual adversarial defense

Abstract

Deep neural networks have a recognized susceptibility to diverse forms of adversarial attacks in the field of natural language processing and such a security issue poses substantial security risks and erodes trust in artificial intelligence applications among people who use them. Meanwhile, quantum theory-inspired models that represent word composition as a quantum mixture of words have modeled the non-linear semantic interaction. However, modeling without considering the non-linear semantic interaction between sentences in the current literature does not exploit the potential of the quantum probabilistic description for improving the robustness in adversarial settings. In the present study, a novel quantum theory-inspired inter-sentence semantic interaction model is proposed for enhancing adversarial robustness via fusing contextual semantics. More specifically, it is analyzed why humans are able to understand textual adversarial examples, and a crucial point is observed that humans are adept at associating information from the context to comprehend a paragraph. Guided by this insight, the input text is segmented into subsentences, with the model simulating contextual comprehension by representing each subsentence as a particle within a mixture system, utilizing a density matrix to model inter-sentence interactions. A loss function integrating cross-entropy and orthogonality losses is employed to encourage the orthogonality of measurement states. Comprehensive experiments are conducted to validate the efficacy of proposed methodology, and the results underscore its superiority over baseline models even commercial applications based on large language models in terms of accuracy across diverse adversarial attack scenarios, showing the potential of proposed approach in enhancing the robustness of neural networks under adversarial attacks.