Multimodal medical image fusion based on interval gradients and convolutional neural networks.

IF 2.9 3区 医学 Q2 RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING BMC Medical Imaging Pub Date : 2024-09-05 DOI:10.1186/s12880-024-01418-x
Xiaolong Gu, Ying Xia, Jie Zhang
{"title":"Multimodal medical image fusion based on interval gradients and convolutional neural networks.","authors":"Xiaolong Gu, Ying Xia, Jie Zhang","doi":"10.1186/s12880-024-01418-x","DOIUrl":null,"url":null,"abstract":"<p><p>Many image fusion methods have been proposed to leverage the advantages of functional and anatomical images while compensating for their shortcomings. These methods integrate functional and anatomical images while presenting physiological and metabolic organ information, making their diagnostic efficiency far greater than that of single-modal images. Currently, most existing multimodal medical imaging fusion methods are based on multiscale transformation, which involves obtaining pyramid features through multiscale transformation. Low-resolution images are used to analyse approximate image features, and high-resolution images are used to analyse detailed image features. Different fusion rules are applied to achieve feature fusion at different scales. Although these fusion methods based on multiscale transformation can effectively achieve multimodal medical image fusion, much detailed information is lost during multiscale and inverse transformation, resulting in blurred edges and a loss of detail in the fusion images. A multimodal medical image fusion method based on interval gradients and convolutional neural networks is proposed to overcome this problem. First, this method uses interval gradients for image decomposition to obtain structure and texture images. Second, deep neural networks are used to extract perception images. Three methods are used to fuse structure, texture, and perception images. Last, the images are combined to obtain the final fusion image after colour transformation. Compared with the reference algorithms, the proposed method performs better in multiple objective indicators of <math><msub><mi>Q</mi> <mrow><mi>EN</mi></mrow> </msub> </math> , <math><msub><mi>Q</mi> <mrow><mi>NIQE</mi></mrow> </msub> </math> , <math><msub><mi>Q</mi> <mrow><mi>SD</mi></mrow> </msub> </math> , <math><msub><mi>Q</mi> <mrow><mi>SSEQ</mi></mrow> </msub> </math> and <math><msub><mi>Q</mi> <mrow><mi>TMQI</mi></mrow> </msub> </math> .</p>","PeriodicalId":9020,"journal":{"name":"BMC Medical Imaging","volume":"24 1","pages":"232"},"PeriodicalIF":2.9000,"publicationDate":"2024-09-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11375917/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"BMC Medical Imaging","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1186/s12880-024-01418-x","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING","Score":null,"Total":0}
引用次数: 0

Abstract

Many image fusion methods have been proposed to leverage the advantages of functional and anatomical images while compensating for their shortcomings. These methods integrate functional and anatomical images while presenting physiological and metabolic organ information, making their diagnostic efficiency far greater than that of single-modal images. Currently, most existing multimodal medical imaging fusion methods are based on multiscale transformation, which involves obtaining pyramid features through multiscale transformation. Low-resolution images are used to analyse approximate image features, and high-resolution images are used to analyse detailed image features. Different fusion rules are applied to achieve feature fusion at different scales. Although these fusion methods based on multiscale transformation can effectively achieve multimodal medical image fusion, much detailed information is lost during multiscale and inverse transformation, resulting in blurred edges and a loss of detail in the fusion images. A multimodal medical image fusion method based on interval gradients and convolutional neural networks is proposed to overcome this problem. First, this method uses interval gradients for image decomposition to obtain structure and texture images. Second, deep neural networks are used to extract perception images. Three methods are used to fuse structure, texture, and perception images. Last, the images are combined to obtain the final fusion image after colour transformation. Compared with the reference algorithms, the proposed method performs better in multiple objective indicators of Q EN , Q NIQE , Q SD , Q SSEQ and Q TMQI .

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
基于区间梯度和卷积神经网络的多模态医学图像融合。
人们提出了许多图像融合方法,以充分利用功能图像和解剖图像的优势,同时弥补它们的不足。这些方法整合了功能和解剖图像,同时呈现了生理和代谢器官信息,使其诊断效率远远高于单模态图像。目前,现有的多模态医学成像融合方法大多基于多尺度变换,即通过多尺度变换获得金字塔特征。低分辨率图像用于分析近似图像特征,高分辨率图像用于分析详细图像特征。不同的融合规则用于实现不同尺度的特征融合。虽然这些基于多尺度变换的融合方法能有效实现多模态医学图像融合,但在多尺度变换和反变换过程中会丢失很多细节信息,导致融合图像的边缘模糊和细节丢失。为了克服这一问题,本文提出了一种基于区间梯度和卷积神经网络的多模态医学图像融合方法。首先,该方法使用区间梯度进行图像分解,以获得结构和纹理图像。其次,利用深度神经网络提取感知图像。使用三种方法融合结构、纹理和感知图像。最后,图像经过色彩转换后得到最终的融合图像。与参考算法相比,所提出的方法在 Q EN、Q NIQE、Q SD、Q SSEQ 和 Q TMQI 等多个客观指标上表现更好。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
BMC Medical Imaging
BMC Medical Imaging RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING-
CiteScore
4.60
自引率
3.70%
发文量
198
审稿时长
27 weeks
期刊介绍: BMC Medical Imaging is an open access journal publishing original peer-reviewed research articles in the development, evaluation, and use of imaging techniques and image processing tools to diagnose and manage disease.
期刊最新文献
In vitro detection of cancer cells using a novel fluorescent choline derivative. Prediction of esophageal fistula in radiotherapy/chemoradiotherapy for patients with advanced esophageal cancer by a clinical-deep learning radiomics model : Prediction of esophageal fistula in radiotherapy/chemoradiotherapy patients. Prior information guided deep-learning model for tumor bed segmentation in breast cancer radiotherapy. The predictive value of nomogram for adnexal cystic-solid masses based on O-RADS US, clinical and laboratory indicators. The study on ultrasound image classification using a dual-branch model based on Resnet50 guided by U-net segmentation results.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1