Persona Knowledge Extraction from Dialog Data in Russian Language

Abstract

The article deals with the joint application of linguistic rules and machine learning models to solve the problem of knowledge extraction from dialog data in Russian. Linguistic rules based on morphological, syntactic and grammatical features are used for automatic markup of the training dataset. The neural network model based on the T5 architecture was trained in multitasking mode, which implied solving the following tasks: a) answer generation based on the dialog history and the facts about the agent's persona found relevant to this history; b) extraction of facts about the persona using the generation method based on the last replica of the agent. The Toloka Persona Chat Rus dataset was used for the experiments. The metrics of both approaches show their applicability to the Russian language, for which no studies have been conducted before.