A BERT-Based Pre-Training Model for Solving Math Application Problems

Solving the math application problem is hot research in intelligence education. An increasing number of research scholars are using pre-trained models to tackle machine solution problems. Noteworthily, the semantic relationships required in the machine solution task are for describing math problems, while those of the BERT model with pre-training weights are of general significance, which will cause a mismatched word vector representation. To solve this problem, we proposed a self-supervised pre-training method based on loss priority. We use the input data from the downstream task datasets to fine-tune the existing BERT model so that the dynamic word vector it obtained can better match the downstream tasks. And the size of the loss value of each data batch in each round of training will be recorded to decide which data should be trained in the next round, so that the model has a faster convergence speed. Furthermore, considering that in large-scale mathematics application problems, some problems have almost the same forms of solution. We proposed a machine solution model training algorithm based on the analogy of the same problem type. Extensive experiments on two well-known datasets show the superiority of our proposed algorithms compared to other state-of-the-art algorithms.