Open-domain Conversational Agent based on Pre-trained Transformers for Human-Robot Interaction

Abstract

: Generative pre-trained transformers belong to the breakthroughs in Natural Language Processing (NLP), allowing Human-Robot Interactions ( e.g. the creation of an open-domain chatbot). However, a substantial amount of research and available data are in English, causing low-resourced languages to be overlooked. This work addresses this problem for European Portuguese with two options: (i) Translation of the sentences before and after using the model fine-tuned on an English-based dataset, (ii) Translation of the English-based dataset to Portuguese and then fine-tune this model on it. We rely on the DialoGPT (dialogue generative pre-trained transformer), a tunable neural conversational answer generation model that learns the basic skills to conduct a dialogue. We use two sources of evaluation: (i) Metrics for text generation based on uncertainty ( i.e. perplexity), and similarity between sentences ( i.e. BLEU, METEOR and ROUGE) and (ii) Human-based evaluation of the sentences. The translation of sentences before and after of the modified DialoGPT model, using the Daily Dialogue dataset led to the best results.