Structured reasoning and answer verification: Enhancing question answering system accuracy and explainability

The performance of question-answering (QA) models has significantly advanced, yet challenges remain in verifying the accuracy of generated answers and providing clear explanations of the reasoning behind them. In response, this study introduces a novel answer verification model that detects inaccuracies in QA system outputs and offers structured, multi-step explanations to enhance both understanding and reliability. We built an answer verification system consisting of a stepwise prover and two types of verifiers and tested the proposed system on the EntailmentBank dataset as well as the ARC, AQUA-RAT, and AR-LSAT datasets from the STREET benchmark. By correcting the answers generated by the T5-large and GPT-3.5 QA models and comparing the results before and after correction, we observed notable improvements in answer accuracy and explanation clarity. Specifically, the proposed model increased the exact match score of the T5-large model by 1.76% and that of GPT-3.5 by 3.53% on the EntailmentBank dataset. Additionally, to address potential data scarcity, the study proposes a data augmentation technique that employs large language models and multi-hop datasets to generate reasoning chains, thereby enriching the training data. Although the augmented data did not match the quality of the gold data, which is manually curated and verified by humans, our experiments demonstrated that combining gold data with augmented data resulted in better performance than using only a subset of the gold data.