Memory-Guided Transformer with group attention for knee MRI diagnosis

Abstract

Magnetic resonance imaging (MRI) plays an important role in the diagnosis of knee injuries, due to its detailed information, which greatly enhances physicians’ diagnostic accuracy. However, the complex image information also makes it difficult for physicians to interpret MRI. There is an urgent need for a computer-assisted method to help physicians extract key information from MRIs and make diagnostic decisions. Knee MRI includes features across three different levels: anatomical plane-level, dataset-level, and case-level. In this paper, we approach the intelligent diagnosis of knee injuries as an interpretable MRI classification task, using a three-stage Memory-Guided Transformer (MGT) for implementation. The first stage focuses on extracting anatomical plane-level and dataset-level features through group attention and cross-attention, which are then stored in the memory matrix. In the second stage, the trained memory matrix guides the extraction of case-level features from different anatomical planes for each case. Finally, the probability of knee injury is determined using linear regression. The MGT was trained with the publicly available MRNet dataset. Compared with the original optimal model PERMIT, it shows a 5.7% improvement in the Youden index. A high level of consistency was observed between the physician-labeled diagnostic regions and the regions identified by group attention. Visualization of the trained memory revealed specific patterns, with column 62 corresponding to healthy subjects and column 81 to patients. These results demonstrate that MGT can effectively assist physicians in diagnosing knee injuries while offering excellent interpretability.