Chinese Story Generation with FastText Transformer Network

Abstract

The sequence transformer models are based on complex recurrent neural network or convolutional networks that include an encoder and a decoder. High-accuracy models are usually represented by used connect the encoder and decoder through an attention mechanism. Story generation is an important thing. If we can let computers learn the ability of story-telling, computers can help people do more things. Actually, the squence2squence model combine attention mechanism is being used to Chinese poetry generation. However, it difficult to apply in Chinese story generation, because there are some rules in Chinese poetry generation. Therefore, we trying to use 1372 human-labeled summarization of paragraphs from a classic novel named “Demi-Gods and Semi-Devils” (天龍八部) to train the transformer network. In our experiment, we use FastText to combine Demi-Gods and Semi-Devils Dataset and A Large Scale Chinese Short Text Summarization Dataset to be input data. In addition, we got a lower loss rate by using two layer of self-attention mechanism.