Dynamic Span Selection for Mandarin Articles Using Contextual Relations and Orthography

Abstract

Span selection is an important prerequisite for many natural language processing tasks. Existing methods usually generate phrase-like spans from entire articles without leveraging the topics or the key points within each paragraph that usually lie behind sentence generation during the writing processes. This study looks at multi-sentence span selection for generating multiple, independent, key-point spans with complete endings for news articles. The proposed span selection model consists of a context relation model and an end span model that merge context-related sentences within a span. The context relation model captures the topics shared between sentences, and the end span model utilizes the embeddings of Zhuyin, the orthography of Mandarin, and the cross attention between words and Zhuyin to effectively capture the end positions of the spans. To evaluate the proposed framework, we construct a news report dataset in Mandarin. Experimental results show that the proposed model not only improves performance, but is also better than previous approaches and close to human span production. The proposed Zhuyin embeddings and cross-attention also improve on BERT’s end sentence detection performance in Mandarin.