A robust framework for mathematical formula detection

Abstract

Mathematical formulas identification is a crucial step in the pipeline of many tasks such as mathematical information retrieval, storing digital science documents, etc. For basic mathematical formulas recognition, all these tasks need to detect the bounding boxes of mathematical expression as a prerequisite step. Currently, deep learning-based object detection methods work well for mathematical formula detection (MFD). These methods are divided into two categories: anchor self-study and anchor not self-study. The anchor self-study method is efficient with large quantity labels but not so well with small quantities, whereas the second type of method works better with small quantities. Therefore, we proposed an algorithm that keeps the good prediction of each type and then merges both into final results. To demonstrate the hypothesis, we select two typical object detection methods: YOLOv5 and Faster RCNN as the representation of two kind approaches to building an MFD framework. Our experiment results on ICDAR2021-MFD1 achieved the F1 score of the whole system is 89.3 while the single detector just reached 74.2, 88.9 (Faster RCNN and YOLOv5 respectively) that proving the effectiveness of the proposal.