IEEE Journal of Biomedical and Health Informatics最新文献

Hybrid Brain-Computer Interface (BCI) enhances accuracy and reliability by leveraging the complementary information provided by multi-modality signal fusion. EEG-fNIRS, a fusion of electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS), have emerged as the suitable techniques for real-world BCI applications due to their portability and economic viability. Existing methods typically focus on the high-level feature representation with late-fusion or early-fusion strategies during the recognition tasks. However, they usually overlook the joint feature extraction of both intra-modality and inter-modality, which is crucial for optimizing BCI performance. In this study, we introduce an Intra- and Inter-modality Correlation Network (IIMCNet) to integrate both the inherent features derived from individual modalities: EEG, deoxygenated hemoglobin (HbR), and oxygenated hemoglobin (HbO), as well as the cross-modality features between EEG-HbR, EEG-HbO, and HbR-HbO data. The intra-modality correlation features are generated using a late fusion method (Intra-net), which combines the uni-modality features extracted by E-Net and f-Net. Concurrently, the inter-modality correlation features are extracted using an early fusion method (Inter-net). Inter-net is consist of three dilated convolution-based C-Nets that focus on neurovascular coupling across modalities. Finally, three intra-modality features, three inter-modality features, and the concatenate hybrid feature are fed into deep supervision module to enhance robustness and accuracy. Experiment results demonstrate the IIMCNet exhibits superior performance compared to methods that rely solely on either intra-modality or inter-modality correlation networks. Furthermore, IIMCNet outperforms other state-of-the-art methods in motor imagery and mental arithmetic tasks, respectively.

Continuous monitoring of Hearth Rate (HR) based on photoplethysmography (PPG) sensors is an essential capability of nearly all wrist-worn devices. However, arm movements lead to the creation of Motion Artifacts (MA), affecting the accuracy of HR tracking using PPG sensors. This problem is commonly tackled by exploiting the recorded accelerometer data to correlate them with the PPG signal and eventually clean it. Thus, automatic fusion techniques based on Deep Learning (DL) algorithms have been proposed, but they are considered too large and complex to be deployed on wearable devices. The current work presents a novel and lightweight DL architecture, PULSE, improving sensor fusion by applying a multi-head cross-attention layer to the extracted temporal features. Moreover, we propose a relation-based knowledge distillation mechanism to pass PULSE's knowledge to a student network that uses modality-wise convolutions to replace the attention module and mimic the teacher's performance with 5× fewer parameters. The teacher and student are evaluated on two datasets: a) PPG-DaLiA the most extensive available dataset, with PULSE achieving close performance to the best state-of-the-art model, and b) WESAD with PULSE reducing the mean absolute error by 22.6%. The student model is further compressed using post-training quantization and deployed on two commercial-off-the-shelf microcontrollers, demonstrating its suitability for real-time execution, having a close-to-state-of-the-art MAE of 4.81 BPM (+0.40 BPM) on the PPG-DaLiA, but a 10.9× lower memory footprint of 37.9 kB, and consuming 45.9× lower energy (0.577 mJ).

Fully automated myocardial segmentation from cardiac magnetic resonance imaging (MRI) is vital for efficient diagnosis and treatment planning. Although numerous automated methods have been proposed, they typically focus on single MRI sequences and therefore have difficulties in generalizing across vendors and across cardiac MRI protocols. Simultaneous analysis of complementary cardiac MRI sequences, such as cine, T1 mapping, and late gadolinium enhancement (LGE) MRI, remains challenging due to their distinct image characteristics and scanner-specific variations. To address these issues, we propose an unsupervised domain adaptation approach that allows robust myocardial segmentation across multi-vendor cine, T1, and LGE MRI data. In particular, we introduce a class- imbalance self-training framework to transfer information learned from a source domain with labels to any unlabeled target domain, while maintaining consistent performance across different MRI sequences. Our framework iteratively refines segmentation accuracy by generating pseudo-labels for target data using a hardness-aware strategy, thus effectively addressing the problem of class imbalance in cardiac MRI segmentation. To mitigate data scarcity following pseudo-label selection, we employ a variance-guided vicinal feature extrapolation, which expands data points in the feature space into a probabilistic distribution. This, in turn, facilitates joint source-target training by generating a larger intersection in the feature space. Experimental results demonstrate that our framework outperforms existing methods when assessed using the Dice coefficient and Hausdorff distance. Our framework enables cardiac evaluation across MRI protocols without sequence-specific manual annotations.

Multimodal medical image fusion plays a crucial role in medical diagnosis by integrating complementary information from different modalities to enhance image readability and clinical applicability. However, existing methods mainly follow computer vision standards for feature extraction and fusion strategy formulation, overlooking the rich semantic information inherent in medical images. To address this limitation, we propose a novel semantic-guided medical image fusion approach that, for the first time, incorporates medical prior knowledge into the fusion process. Specifically, we construct a publicly available multimodal medical image-text dataset, upon which text descriptions generated by BiomedGPT are encoded and semantically aligned with image features in a high-dimensional space via a semantic interaction alignment module. During this process, a cross attention based linear transformation automatically maps the relationship between textual and visual features to facilitate comprehensive learning. The aligned features are then embedded into a text-injection module for further feature-level fusion. Unlike traditional methods, we further generate diagnostic reports from the fused images to assess the preservation of medical information. Additionally, we design a medical semantic loss function to enhance the retention of textual cues from the source images. Experimental results on test datasets demonstrate that the proposed method achieves superior performance in both qualitative and quantitative evaluations while preserving more critical medical information.

Over the past five years, artificial intelligence (AI) has introduced new models and methods for addressing the challenges associated with the broader adoption of AI models and systems in medicine. This paper reviews recent advances in AI for medical image and video analysis, outlines emerging paradigms, highlights pathways for successful clinical translation, and provides recommendations for future work. Hybrid Convolutional Neural Network (CNN) Transformer architectures now deliver state-of-the-art results in segmentation, classification, reconstruction, synthesis, and registration. Foundation and generative AI models enable the use of transfer learning to smaller datasets with limited ground truth. Federated learning supports privacy-preserving collaboration across institutions. Explainable and trustworthy AI approaches have become essential to foster clinician trust, ensure regulatory compliance, and facilitate ethical deployment. Together, these developments pave the way for integrating AI into radiology, pathology, and wider healthcare workflows.

Multimodal fusion is an effective solution for holistic brain tumor diagnosis; however, it faces challenges under missing modalities. Traditional multi-encoder architectures can easily capture modality-specific features, while single-encoder architectures readily obtain modality-shared features. The reverse, however, is challenging. In this paper, we propose a two-stage dual-view prototype learning framework to extract the modality-specific feature and the class-specific feature simultaneously. In the first stage, we utilize the Transformer decoder to learn the modality-prototypes that are used to optimize the modality reconstruction task. A masked autoencoder is introduced to generate shared features of incomplete modalities. The learned modality-prototypes that contain modality-specific features are blended with the modality-share features for the reconstruction process. In the second stage, we learn the class-prototypes through the Transformer decoder to generate a segmentation mask through voxel-to-prototype comparison. A masked modality strategy is introduced to handle random modality absence during training. Furthermore, modality-view and class-view contrastive learning strategies are developed to enhance prototype learning. We conduct experiments on BraTS2020 and BraTS2018; the experimental results demonstrate the superior performance of our model under various missing modality scenarios. On BraTS2020, our model achieves DSC improvements of 5.9% for ET, 0.5% for TC, and 0.2% for WT compared to state-of-the-art methods. Notably, in the challenging T1C modality missing scenario, our model achieved clinically significant gains of 9.5% for ET and 1.8% for TC. The code is available at https://github.com/Xiheran/MCPL.

Cancer is characterized by complex subtypes and pronounced heterogeneity, which pose significant challenges for accurate identification and effective treatment. In response, multi-omics clustering has emerged as a powerful approach for integrating heterogeneous biological data to identify cancer subtypes, thereby playing a crucial role in early diagnosis and precision medicine. Despite promising progress, existing multi-omics clustering methods face two key limitations. First, most methods focus on mining the common information across omics but neglect the unique heterogeneity features of each omics. Second, representation learning and clustering are often decoupled, preventing joint optimization of feature representations and the clustering affinity matrix, ultimately leading to suboptimal performance. To tackle these difficulties, we propose a novel Structure-Aware Consensus Representation Learning with Dual-Channel Attention for Multi-Omics Cancer Subtype Clustering(SACR-DCA). SACR-DCA integrates two pivotal modules: (1) The multi-omics specific feature extraction and common representation fusion module, which uniquely captures both omics-specific characteristics and their shared information via a dual-channel attention fusion framework; (2) The clustering-oriented structure-aware representation learning and consensus enhancement module, which enhances consensus representations through structure-aware learning to boost clustering efficacy, leveraging a Cauchy-Schwarz (CS) divergence constraint for clustering adaptability. Performance experiments on ten real-world datasets fully demonstrate that our method outperforms existing methods. The source code is available at https://github.com/liukun2000/SACR-DCA.

Circular RNAs (circRNAs) represent a distinctive class of non-coding RNAs with covalently closed loop structures that play crucial regulatory roles in drug response. While existing computational methods have achieved certain progress in prediction tasks, they primarily relied on circRNA genotypes and traditional molecular fingerprints, with limited utilization of multi-omics data and inadequate consideration of heterogeneous network topology. To address these limitations, this study proposed the CircRNA-Drug Bipartite Graph Transformer (CDBGT) framework to predict associations. Rather than limiting to associations between circRNA genotypes and drugs, this study integrated circRNA-drug response and target association information from multiple databases. CDBGT employed pre-trained models RNA-FM and ChemBERTa to extract features of sequence and molecular fingerprint and utilized multi-omics data to construct similarity matrices. The framework incorporated a bipartite graph transformer with topological positional encoding, comprehensively considering degree encoding, degree ranking encoding and spectral encoding to extract topological information from heterogeneous networks. Experimental results showed that CDBGT performed stably in 5-fold cross-validation. On the Response dataset, it achieved ROC-AUC of 0.9674 and PR-AUC of 0.9540, while on the Target dataset it reached ROC-AUC of 0.8621. Compared with existing methods, it showed an improvement of 3.20 to 26.87 percentage points in ROC-AUC. Ablation experiments demonstrated the necessity of each module. Through literature-supported case studies, this work suggested potential directions for circRNA-based therapeutic research.

Accurate polyp segmentation in colonoscopy images is essential for early colorectal cancer detection but remains a challenging problem due to the limitations in existing methods for optimizing boundary features and aligning cross-level representations. Specifically, the indistinct polyp boundaries and scale variations across different feature levels pose significant challenges for segmentation accuracy. To address these issues, we propose a dual domain collaborative network (DDCNet) that introduces two novel modules: a frequency context enhancement module (FCEM), which operates in the frequency domain to refine high- and low-frequency features, and a cross-level shift recalibrated fusion module (CSFM), which improves multi scale feature alignment in the spatial domain. The FCEM improves boundary precision by adaptively refining high frequency boundary features and enhancing low-frequency contextual information, while the CSFM mitigates cross level feature misalignment by dynamically recalibrating multi-scale features throughout the encoder-decoder architecture. Additionally, we design a hybrid loss function that integrates boundary, cross-entropy, and frequency consistency losses to further boost segmentation performance. Experimental results on three benchmark datasets (Kvasir SEG, CVC-ClinicDB, and CVC-ColonDB) demonstrate that DDCNet achieves state-of-the-art performance, with Dice coefficients of 0.9343, 0.9447, and 0.8155, respectively. These results represent improvements of 1.0%-1.5% over the best existing methods. Ablation studies further validate the individual contributions of FCEM, CSFM, and the hybrid loss function. Additionally, we compared the proposed loss function with three commonly used functions.

Accurate drug-drug interaction (DDI) prediction is crucial for optimizing the efficacy of combination therapies and minimizing adverse effects. Most existing methods rely on single features and struggle to integrate structural and sequential drug information. Additionally, prediction bias caused by class imbalance remains a significant challenge. To address these issues, this study proposes a multi-source example-driven learning framework for DDI (MEDL-DDI) that jointly models structural and sequential drug representations to achieve robust multimodal fusion and mitigate class imbalance. MEDL-DDI enriches SMILES with chemical knowledge, extracts global semantic features via a Transformer, and identifies key substructures through a graph information bottleneck. Moreover, an example-driven mechanism guided by example centers enhances the model's ability to recognize minority classes. Experimental results on three benchmark datasets validate that MEDL-DDI outperforms state-of-the-art methods. The case study on cardiovascular drug interactions further highlights MEDL-DDI's practical value and applicability.