Extending Output Attentions in Recurrent Neural Networks for Dialog Generation

Abstract

Attention mechanism in recurrent neural networks has been widely used in natural language processing. In this paper, the research team explore a new mechanism of extending output attention in recurrent neural networks for dialog systems. The new attention method was compared with the current method in generating dialog sentence using a real dataset. Our architecture exhibits several attractive properties such as better handle long sequences and, it could generate more reasonable replies in many cases.