Classification of Judicial Documents Based on Improved Bert Model

Abstract

Bert model is the most comprehensive language model proposed in the past two years. It uses large-scale unlabeled prediction to train the representation of text with rich semantic information, that is, the semantic representation of text. Then, the semantic representation of the text is slightly tuned in a specific NLP task. Finally, it is applied to the NLP task, and has good performance in various natural language processing tasks Performance. For the classification of judicial documents, proposed a based on the improved Bert model way. The word vector is provided through the pre training model Bert. The semantic information is embedded into the LSTM model through the context of the Bert model. LSTM model is which is solves the problems of gradient disappearance and gradient explosion in long sequence Through the self attention mechanism, the important information in the text is given higher weight, and LSTM is used to encode the sequence and feature fusion contain category information. Through the Bert model and Bert-LSTM model, we can see that in accuracy improved 4.55% of the Bert-LSTM model.