Zero-Shot Multilingual Sentiment Analysis using Hierarchical Attentive Network and BERT

Abstract

Sentiment analysis is considered an important downstream task in language modelling. We propose Hierarchical Attentive Network using BERT for document sentiment classification. We further showed that importing representation from Multiplicative LSTM model in our architecture results in faster convergence. We then propose a method to build a sentiment classifier for a language in which we have no labelled sentiment data. We exploit the possible semantic invariance across languages in the context of sentiment to achieve this.