Incorporating Scenario Knowledge into A Unified Fine-tuning Architecture for Event Representation

Abstract

Given an occurred event, human can easily predict the next event or reason the preceding event, yet which is difficult for machine to perform such event reasoning. Event representation bridges the connection and targets to model the process of event reasoning as a machine-readable format, which then can support a wide range of applications in information retrieval, e.g., question answering and information extraction. Existing work mainly resorts to a joint training to integrate all levels of training loss in event chains by a simple loss summation, which is easily trapped into a local optimum. In addition, the scenario knowledge in event chains is not well investigated for event representation. In this paper, we propose a unified fine-tuning architecture, incorporated with scenario knowledge for event representation, i.e., UniFA-S, which mainly consists of a unified fine-tuning architecture (UniFA) and a scenario-level variational auto-encoder (S-VAE). In detail, UniFA employs a multi-step fine-tuning to integrate all levels of training and S-VAE applies a stochastic variable to implicitly represent the scenario-level knowledge. We evaluate our proposal from two aspects, i.e., the representation and inference abilities. For the representation ability, our ensemble model UniFA-S can beat state-of-the-art baselines for two similarity tasks. For the inference ability, UniFA-S can outperform the best baseline, achieving 4.1%-8.2% improvements in terms of accuracy for various inference tasks.