{"title":"ResMT: A hybrid CNN-transformer framework for glioma grading with 3D MRI","authors":"","doi":"10.1016/j.compeleceng.2024.109745","DOIUrl":null,"url":null,"abstract":"<div><div>Accurate grading of gliomas is crucial for treatment strategies and prognosis. While convolutional neural networks (CNNs) have proven effective in classifying medical images, they struggle with capturing long-range dependencies among pixels. Transformer-based networks can address this issue, but CNN-based methods often perform better when trained on small datasets. Additionally, tumor segmentation is essential for classification models, but training an additional segmentation model significantly increases workload. To address these challenges, we propose ResMT, which combines CNN and transformer architectures for glioma grading, extracting both local and global features efficiently. Specifically, we designed a spatial residual module (SRM) where a 3D CNN captures glioma's volumetric complexity, and Swin UNETR, a pre-trained segmentation model, enhances the network without extra training. Our model also includes a multi-plane channel and spatial attention module (MCSA) to refine the analysis by focusing on critical features across multiple planes (axial, coronal, and sagittal). Transformer blocks establish long-range relationships among planes and slices. We evaluated ResMT on the BraTs19 dataset, comparing it with baselines and state-of-the-art models. Results demonstrate that ResMT achieves the highest prediction performance with an AUC of 0.9953, highlighting hybrid CNN-transformer models' potential for 3D MRI classification.</div></div>","PeriodicalId":50630,"journal":{"name":"Computers & Electrical Engineering","volume":null,"pages":null},"PeriodicalIF":4.0000,"publicationDate":"2024-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computers & Electrical Engineering","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0045790624006724","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, HARDWARE & ARCHITECTURE","Score":null,"Total":0}
引用次数: 0
Abstract
Accurate grading of gliomas is crucial for treatment strategies and prognosis. While convolutional neural networks (CNNs) have proven effective in classifying medical images, they struggle with capturing long-range dependencies among pixels. Transformer-based networks can address this issue, but CNN-based methods often perform better when trained on small datasets. Additionally, tumor segmentation is essential for classification models, but training an additional segmentation model significantly increases workload. To address these challenges, we propose ResMT, which combines CNN and transformer architectures for glioma grading, extracting both local and global features efficiently. Specifically, we designed a spatial residual module (SRM) where a 3D CNN captures glioma's volumetric complexity, and Swin UNETR, a pre-trained segmentation model, enhances the network without extra training. Our model also includes a multi-plane channel and spatial attention module (MCSA) to refine the analysis by focusing on critical features across multiple planes (axial, coronal, and sagittal). Transformer blocks establish long-range relationships among planes and slices. We evaluated ResMT on the BraTs19 dataset, comparing it with baselines and state-of-the-art models. Results demonstrate that ResMT achieves the highest prediction performance with an AUC of 0.9953, highlighting hybrid CNN-transformer models' potential for 3D MRI classification.
期刊介绍:
The impact of computers has nowhere been more revolutionary than in electrical engineering. The design, analysis, and operation of electrical and electronic systems are now dominated by computers, a transformation that has been motivated by the natural ease of interface between computers and electrical systems, and the promise of spectacular improvements in speed and efficiency.
Published since 1973, Computers & Electrical Engineering provides rapid publication of topical research into the integration of computer technology and computational techniques with electrical and electronic systems. The journal publishes papers featuring novel implementations of computers and computational techniques in areas like signal and image processing, high-performance computing, parallel processing, and communications. Special attention will be paid to papers describing innovative architectures, algorithms, and software tools.