A homogeneous multimodality sentence representation for relation extraction

Deep neural networks enable a sentence to be transformed into different multimodalities such as a token sequence representation (a one-dimensional semantic representation) or a semantic plane (a two-dimensional semantic representation). Sequence representation has the advantage of learning sequential dependencies of a sentence. Semantic plane is built by organizing all spans of a sentence, which is effective in resolving complicated sentence semantic structures. The two representations are derived from a homogeneous resource (the same sentence), but they are separately used in related works. In this paper, a homogeneous multimodality sentence representation is proposed to make full use of semantic information in a sentence. We construct a homomodality model, which is composed of three components: a sequential encoder to generate sequential modality, a plane encoder to build plane modality, and a multimodality fusion component aligning homogeneous multimodalities for learning a multi-granularity semantic representation. Our model is evaluated on four public datasets to support the relation extraction task. Compared with related works, it achieves state-of-the-art performance on all datasets. Analytical experiments show that fusing homogeneous multimodalities is effective in making full use of sentence information for advancing the discriminability of a deep neural network.