Multimodal fine-grained reasoning for post quality evaluation

Abstract

Accurate assessment of post quality frequently necessitates complex relational reasoning skills that emulate human cognitive processes, thereby requiring the modeling of nuanced relationships. However, existing research on post-quality assessment suffers from the following problems: (1) They are often categorization tasks that rely solely on unimodal data, which inadequately captures information in multimodal contexts and fails to differentiate the quality of students’ posts finely. (2) They ignore the noise in the multimodal deep fusion between posts and topics, which may produce misleading information for the model. (3) They do not adequately capture the complex and fine-grained relationships between post and topic, resulting in an inaccurate evaluation, such as relevance and comprehensiveness. Based on the above challenges, the Multimodal Fine-grained Topic-post Relational Reasoning(MFTRR) framework is proposed for modeling fine-grained cues by simulating the human thinking process. It consists of the local–global semantic correlation reasoning module and the multi-level evidential relational reasoning module. Specifically, MFTRR addresses the challenge of unimodal and categorization task limitations by framing post-quality assessment as a ranking task and integrating multimodal data to more effectively distinguish quality differences. To capture the most relevant semantic relationships, the Local–Global Semantic Correlation Reasoning Module enables deep interactions between posts and topics at both local and global scales. It is complemented by a topic-based maximum information fusion mechanism to filter out noise. Furthermore, to model complex and subtle relational reasoning, the Multi-Level Evidential Relational Reasoning Module analyzes topic-post relationships at both macro and micro levels by identifying critical cues and delving into granular relational cues. MFTRR is evaluated using three newly curated multimodal topic-post datasets, in addition to the publicly available Lazada-Home dataset. Experimental results indicate that MFTRR outperforms state-of-the-art baselines, achieving a 9.52% improvement in the NDCG@3 metric compared to the best text-only method on the Art History course dataset.