Pub Date : 2025-12-19DOI: 10.1109/MSP.2025.3604570
Chuang Liang;Rogers F. Silva;Tulay Adali;Rongtao Jiang;Daoqiang Zhang;Shile Qi;Vince D. Calhoun
Multimodal fusion provides significant benefits over single-modality analysis by leveraging both shared and complementary information across diverse data sources. In this article, we systematically review methods for the fusion of heterogonous multimodal biomedical data of varying dimensionality (including neuroimaging, biomics, clinical phenotypes, and text), with a focus on neuroscience. We discuss the strengths and limitations of these strategies based on a survey of 302 research articles. Next, we examine the applications of these methods to a variety of scenarios spanning a continuum from scientific research to clinical practice. Finally, an in-depth discussion of common challenges and promising directions for future development of multimodal biomedical data fusion are provided. Overall, multimodal fusion shows substantial benefits and transformative potential in the field of neuroscience. Future research should prioritize improving model generalization, enhancing interpretability, addressing inherent data limitations, and developing unified platforms alongside multimodal foundational models to bridge the gaps among fusion techniques, research, and application to various domains.
{"title":"Multimodal Data Fusion in Neuroscience: Promises, challenges, and future directions [Special Issue on Accelerating Brain Discovery Through Data Science and Neurotechnology]","authors":"Chuang Liang;Rogers F. Silva;Tulay Adali;Rongtao Jiang;Daoqiang Zhang;Shile Qi;Vince D. Calhoun","doi":"10.1109/MSP.2025.3604570","DOIUrl":"10.1109/MSP.2025.3604570","url":null,"abstract":"Multimodal fusion provides significant benefits over single-modality analysis by leveraging both shared and complementary information across diverse data sources. In this article, we systematically review methods for the fusion of heterogonous multimodal biomedical data of varying dimensionality (including neuroimaging, biomics, clinical phenotypes, and text), with a focus on neuroscience. We discuss the strengths and limitations of these strategies based on a survey of 302 research articles. Next, we examine the applications of these methods to a variety of scenarios spanning a continuum from scientific research to clinical practice. Finally, an in-depth discussion of common challenges and promising directions for future development of multimodal biomedical data fusion are provided. Overall, multimodal fusion shows substantial benefits and transformative potential in the field of neuroscience. Future research should prioritize improving model generalization, enhancing interpretability, addressing inherent data limitations, and developing unified platforms alongside multimodal foundational models to bridge the gaps among fusion techniques, research, and application to various domains.","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"42 5","pages":"8-21"},"PeriodicalIF":9.6,"publicationDate":"2025-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145786129","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-12-19DOI: 10.1109/MSP.2025.3640746
{"title":"From Signals to Causes: Methodological Advances in Causal Inference [Call for Papers]","authors":"","doi":"10.1109/MSP.2025.3640746","DOIUrl":"10.1109/MSP.2025.3640746","url":null,"abstract":"","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"42 5","pages":"C2-C2"},"PeriodicalIF":9.6,"publicationDate":"2025-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11306224","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145785568","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-12-19DOI: 10.1109/MSP.2025.3592771
Vincenzo Catrambone;Gaetano Valenza
The brain and heart have traditionally been investigated as distinct systems, with ad hoc signal processing methodologies tailored to their specific dynamics at the cortical, subcortical, and peripheral levels. However, increasing evidence highlights the fundamental role of brain–heart interplay (BHI), which can generate dynamics that neither system can produce in isolation. Through this interplay, impairments in one system can profoundly influence the other via complex neural, mechanical, and biochemical pathways. Consequently, there is a growing scientific interest in quantitatively characterizing BHI to better understand its functional dynamics and potential clinical implications. Focusing on the neural brain–heart axis as monitored through electroencephalographic and electrocardiographic signals, this study aims to systematically categorize existing signal processing methods for functional BHI assessment, thereby providing a comprehensive taxonomy from a methodological point of view. We show that BHI has been quantified using diverse analytical frameworks that leverage physiological specificity, mathematical modeling, and the ability to capture directional and time-varying interactions. Furthermore, we present a tutorial-like description on a physiologically inspired modeling approach that enables the estimation of BHI with high temporal resolution while preserving directional information. This study fosters the development of integrated approaches for BHI quantification, calling for collaboration among signal processing developers, neuroscientists, cardiologists, and computational physiologists.
{"title":"Methodological Taxonomy for Functional Brain–Heart Interplay Assessment: Creating a comprehensive taxonomy","authors":"Vincenzo Catrambone;Gaetano Valenza","doi":"10.1109/MSP.2025.3592771","DOIUrl":"10.1109/MSP.2025.3592771","url":null,"abstract":"The brain and heart have traditionally been investigated as distinct systems, with ad hoc signal processing methodologies tailored to their specific dynamics at the cortical, subcortical, and peripheral levels. However, increasing evidence highlights the fundamental role of brain–heart interplay (BHI), which can generate dynamics that neither system can produce in isolation. Through this interplay, impairments in one system can profoundly influence the other via complex neural, mechanical, and biochemical pathways. Consequently, there is a growing scientific interest in quantitatively characterizing BHI to better understand its functional dynamics and potential clinical implications. Focusing on the neural brain–heart axis as monitored through electroencephalographic and electrocardiographic signals, this study aims to systematically categorize existing signal processing methods for functional BHI assessment, thereby providing a comprehensive taxonomy from a methodological point of view. We show that BHI has been quantified using diverse analytical frameworks that leverage physiological specificity, mathematical modeling, and the ability to capture directional and time-varying interactions. Furthermore, we present a tutorial-like description on a physiologically inspired modeling approach that enables the estimation of BHI with high temporal resolution while preserving directional information. This study fosters the development of integrated approaches for BHI quantification, calling for collaboration among signal processing developers, neuroscientists, cardiologists, and computational physiologists.","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"42 5","pages":"71-79"},"PeriodicalIF":9.6,"publicationDate":"2025-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145785572","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-12-19DOI: 10.1109/MSP.2025.3610335
Harry J. Davies;Giorgos Iacovides;Wuyang Zhou;Anthony G. Constantinides;Danilo P. Mandic
{"title":"Demystifying Encoder–Decoder Neural Networks: Correncoder for Regression via Latent Spaces [Lecture Notes]","authors":"Harry J. Davies;Giorgos Iacovides;Wuyang Zhou;Anthony G. Constantinides;Danilo P. Mandic","doi":"10.1109/MSP.2025.3610335","DOIUrl":"10.1109/MSP.2025.3610335","url":null,"abstract":"","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"42 5","pages":"98-110"},"PeriodicalIF":9.6,"publicationDate":"2025-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11306199","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145785575","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-12-19DOI: 10.1109/MSP.2025.3619327
Vince D. Calhoun;Damien Coyle;Javier Escudero;Borbala Hunyadi;Jing Sui
The study of the brain remains one of the most intricate and compelling frontiers of scientific research. With the advent of advanced neurotechnology and data science, we stand on the brink of unprecedented discoveries that could help unravel the complexities of brain function and dysfunction. This special issue is in part motivated by discussions arising from the highly successful IEEE Brain Discovery and Neurotechnology Workshop, a satellite event to the Society for Neuroscience. As an IEEE-wide effort, the IEEE Brain Technical Community brings together engineers, computer scientists, and neuroscientists to advance neurotechnology through collaboration, research, and standardization. Its purpose is to accelerate ethical and responsible development of brain-related technologies, foster new initiatives, and connect academia, industry, and government.
{"title":"Editorial for Special Issue on Accelerating Brain Discovery Through Data Science and Neurotechnology: Part 2 [From the Guest Editors]","authors":"Vince D. Calhoun;Damien Coyle;Javier Escudero;Borbala Hunyadi;Jing Sui","doi":"10.1109/MSP.2025.3619327","DOIUrl":"10.1109/MSP.2025.3619327","url":null,"abstract":"The study of the brain remains one of the most intricate and compelling frontiers of scientific research. With the advent of advanced neurotechnology and data science, we stand on the brink of unprecedented discoveries that could help unravel the complexities of brain function and dysfunction. This special issue is in part motivated by discussions arising from the highly successful IEEE Brain Discovery and Neurotechnology Workshop, a satellite event to the Society for Neuroscience. As an IEEE-wide effort, the IEEE Brain Technical Community brings together engineers, computer scientists, and neuroscientists to advance neurotechnology through collaboration, research, and standardization. Its purpose is to accelerate ethical and responsible development of brain-related technologies, foster new initiatives, and connect academia, industry, and government.","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"42 5","pages":"5-7"},"PeriodicalIF":9.6,"publicationDate":"2025-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11306190","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145786128","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-12-19DOI: 10.1109/MSP.2025.3640720
{"title":"SPS Social Media","authors":"","doi":"10.1109/MSP.2025.3640720","DOIUrl":"10.1109/MSP.2025.3640720","url":null,"abstract":"","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"42 5","pages":"7-7"},"PeriodicalIF":9.6,"publicationDate":"2025-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11306192","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145785573","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Brain foundation models (BFMs) represent a transformative paradigm in computational neuroscience that leverages large-scale pretraining on diverse neural signals to achieve robust generalization across tasks, modalities, and experimental contexts. This survey article establishes the first comprehensive definition and framework for BFMs, systematically examining their construction, core methodologies, and applications. We present key approaches for data processing and training strategies alongside diverse applications spanning brain decoding and scientific discovery. Through critical analysis of recent methodological innovations, we identify fundamental challenges that must be addressed to realize the full potential of BFMs, including advancing data quality and standardization, optimizing model architectures, improving training efficiency, and enhancing interpretability. By bridging the gap between neuroscience and artificial intelligence (AI), BFMs present unprecedented opportunities to revolutionize brain research, clinical diagnostics, and therapeutic interventions. This article serves as a foundational reference for researchers and practitioners seeking to understand and advance this emerging field.
脑基础模型（Brain foundation models，简称BFMs）代表了计算神经科学的一种变革范式，它利用对不同神经信号的大规模预训练来实现跨任务、模式和实验环境的鲁棒泛化。这篇综述文章建立了bfm的第一个综合定义和框架，系统地检查了它们的构造、核心方法和应用。我们提出了数据处理和训练策略的关键方法，以及跨越大脑解码和科学发现的各种应用。通过对最近方法创新的批判性分析，我们确定了必须解决的基本挑战，以实现bfm的全部潜力，包括提高数据质量和标准化，优化模型架构，提高训练效率和增强可解释性。通过弥合神经科学和人工智能（AI）之间的差距，bfm为彻底改变大脑研究、临床诊断和治疗干预提供了前所未有的机会。本文为寻求理解和推进这一新兴领域的研究人员和实践者提供了基础参考。
{"title":"Brain Foundation Models: A survey on advancements in neural signal processing and brain discovery","authors":"Xinliang Zhou;Chenyu Liu;Zhisheng Chen;Kun Wang;Yi Ding;Ziyu Jia;Qingsong Wen","doi":"10.1109/MSP.2025.3592356","DOIUrl":"10.1109/MSP.2025.3592356","url":null,"abstract":"Brain foundation models (BFMs) represent a transformative paradigm in computational neuroscience that leverages large-scale pretraining on diverse neural signals to achieve robust generalization across tasks, modalities, and experimental contexts. This survey article establishes the first comprehensive definition and framework for BFMs, systematically examining their construction, core methodologies, and applications. We present key approaches for data processing and training strategies alongside diverse applications spanning brain decoding and scientific discovery. Through critical analysis of recent methodological innovations, we identify fundamental challenges that must be addressed to realize the full potential of BFMs, including advancing data quality and standardization, optimizing model architectures, improving training efficiency, and enhancing interpretability. By bridging the gap between neuroscience and artificial intelligence (AI), BFMs present unprecedented opportunities to revolutionize brain research, clinical diagnostics, and therapeutic interventions. This article serves as a foundational reference for researchers and practitioners seeking to understand and advance this emerging field.","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"42 5","pages":"22-35"},"PeriodicalIF":9.6,"publicationDate":"2025-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145785574","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：