Handling language prior and compositional reasoning issues in Visual Question Answering system

Abstract

Visual Question Answering (VQA) models often suffer from language bias, favoring common but incorrect answers, and struggle with compositional reasoning in complex queries. This paper proposes a unified approach using a multimodal large language model enhanced with adaptive prompts designed for specific tasks. Our method directly addresses these issues by reducing language bias and improving compositional reasoning. Extensive evaluations on benchmark datasets, including VQA v2.0, VQACP, TDIUC, GQA, Visual7 W, TextVQA, and STVQA show that our approach outperforms state-of-the-art models, achieving accuracy improvements of 8% to 9%. These results demonstrate the effectiveness of our method in enhancing VQA accuracy, making it a significant advancement for more reliable and robust applications in real-world scenarios.