DPN: Dynamics Priori Networks for Radiology Report Generation

Abstract

Radiology report generation is of significant importance. Unlike standard image captioning tasks, radiology report generation faces more pronounced visual and textual biases due to constrained data availability, making it increasingly reliant on prior knowledge in this context. In this paper, we introduce a radiology report generation network termed Dynamics Priori Networks (DPN), which leverages a dynamic knowledge graph and prior knowledge. Concretely, we establish an adaptable graph network and harness both medical domain knowledge and expert insights to enhance the model's intelligence. Notably, we introduce an image-text contrastive module and an image-text matching module to enhance the quality of the generated results. Our method is evaluated on two widely available datasets: X-ray collection from Indiana University (IU X-ray) and Medical Information Mart for Intensive Care, Chest X-Ray (MIMIC-CXR), where it demonstrates superior performance, particularly excelling in critical metrics.