Estimating Semantic Relationships between Sentences Using Word Embedding with BERT

Abstract

In this study, we focus on conjunctions between sentences to estimate the semantic relations between sentences. As a method for estimating the types of hidden conjunctions, we propose a method using a word embedding with bidirectional encoder representations from the transformer (BERT), which has shown high accuracy in various natural language processing tasks. By using Japanese newspaper articles, we have confirmed the effectiveness of the proposed method in estimating the presence or absence of conjunctions and the types of conjunctions. There was a difference in the accuracy by changing the estimator used to input word embedding. The result varied greatly depending on the conjunction.