Few-shot machine reading comprehension for bridge inspection via domain-specific and task-aware pre-tuning approach

With the wide application of information technologies in the field of bridge engineering, many electronic bridge inspection reports have been generated. However, due to insufficient research on machine reading comprehension (MRC) in this field, a lot of bridge inspection information, e.g., structural basic data, inspected defects, and maintenance suggestions, has not been fully used. Especially, it is time-consuming and labor-intensive to pre-train a domain-specific language model from scratch or annotate large-scale question answering corpora, which also brings challenges to the MRC research in this field. To tackle the problems, this paper proposes a novel few-shot MRC approach for bridge inspection based on the idea of data augmentation. The proposed model uses a pre-trained model as backbone, along with introducing a pre-tuning stage to bridge the gaps between general-purpose pre-training and domain-specific MRC tasks. In order to reduce the workload of manual annotation, we present a novel pre-tuning data generation algorithm which is based on the domain-specific question classification and answer prediction neural models. After pre-tuning and fine-tuning, the proposed model achieves efficient bridge inspection MRC. The experimental results show that the proposed model outperforms the mainstream fine-tuning-based approaches and few-shot MRC baseline models in various settings. With 1024 fine-tuning samples, the F1 value and Exact Match (EM) value are 86.42%, 74.65%, respectively. Our research work can serve as a foundation for the construction of automatic question answering systems for intelligent bridge management and maintenance.