Text Detoxification System in Dialogue Conversations

Abstract

The work is aimed at improving the cultural level of correspondence in dialog systems. The key feature of the work is its focus on real–time use and ensuring sustainable detoxification, taking into account the specifics of dialog communication (typos, noise symbols, transliteration, etc.). The solution offers the use of a neural network approach and software processing to obtain embeds of tokens and the subsequent solution of the classification problem. Unlike traditional message filters, the task is to preserve the meaning of the source text by clearing it of toxic content. The operability of the system can be checked on the basis of the Telegram messenger, in which the model is presented in the form of a bot. The system itself is deployed on the basis of Serverless technology from a cloud provider, which allows it to adapt to peak loads and at the same time be easy to maintain.