Pub Date : 2026-07-10DOI: 10.1109/tmi.2026.3711975
Qi Zhang, Xia Li, Yibo Hu, Jianqi Sun
Unsupervised anomaly detection (UAD) in brain MRI is crucial for early diagnosis, yet generalizing existing methods across diverse diseases, sequences, and missing data scenarios remains a significant challenge. Current reconstruction-based methods often fail to detect subtle anomalies, while conventional translation methods lack flexibility regarding input sequences. To address these limitations, we propose UniTransAD, a unified translation-based anomaly detection framework. UniTransAD introduces three key innovations: (1) a unified cyclic-translation inference paradigm built upon content-style disentanglement, capable of processing diverse brain MRI inputs; (2) a Dynamic Style Prototype Memory (DSPM) that enables a flexible and robust cyclic-inference mechanism; and (3) a dual-level detection mechanism that combines pixel-level translation errors with feature-level dissimilarities to enhance detection specificity. Furthermore, to rigorously evaluate generalization beyond disease-specific datasets, we establish the Brain-OmniA evaluation dataset, aggregating seven public datasets covering distinct brain pathologies and sequences. Extensive experiments demonstrate that UniTransAD significantly outperforms state-of-the-art methods on Brain-OmniA with superior flexibility. In summary, UniTransAD offers a robust, flexible and generalizable solution for clinical anomaly detection in heterogeneous clinical environments. Our code, pre-trained models, and full dataset are available at: https://github.com/zhibaishouheilab/UniTransAD.
{"title":"UniTransAD: Unified Translation Framework for Anomaly Detection in Brain MRI","authors":"Qi Zhang, Xia Li, Yibo Hu, Jianqi Sun","doi":"10.1109/tmi.2026.3711975","DOIUrl":"https://doi.org/10.1109/tmi.2026.3711975","url":null,"abstract":"Unsupervised anomaly detection (UAD) in brain MRI is crucial for early diagnosis, yet generalizing existing methods across diverse diseases, sequences, and missing data scenarios remains a significant challenge. Current reconstruction-based methods often fail to detect subtle anomalies, while conventional translation methods lack flexibility regarding input sequences. To address these limitations, we propose UniTransAD, a unified translation-based anomaly detection framework. UniTransAD introduces three key innovations: (1) a unified cyclic-translation inference paradigm built upon content-style disentanglement, capable of processing diverse brain MRI inputs; (2) a Dynamic Style Prototype Memory (DSPM) that enables a flexible and robust cyclic-inference mechanism; and (3) a dual-level detection mechanism that combines pixel-level translation errors with feature-level dissimilarities to enhance detection specificity. Furthermore, to rigorously evaluate generalization beyond disease-specific datasets, we establish the Brain-OmniA evaluation dataset, aggregating seven public datasets covering distinct brain pathologies and sequences. Extensive experiments demonstrate that UniTransAD significantly outperforms state-of-the-art methods on Brain-OmniA with superior flexibility. In summary, UniTransAD offers a robust, flexible and generalizable solution for clinical anomaly detection in heterogeneous clinical environments. Our code, pre-trained models, and full dataset are available at: https://github.com/zhibaishouheilab/UniTransAD.","PeriodicalId":13418,"journal":{"name":"IEEE Transactions on Medical Imaging","volume":"80 1","pages":""},"PeriodicalIF":10.6,"publicationDate":"2026-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148416094","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-07-07DOI: 10.1109/tmi.2026.3710717
Qiushi Yang,Wuyang Li,Xiaoqing Guo,Maymay Cerys Harwood,Peter Y M Woo,Jingyang Zhang,Yang Chen,Ke Zhang,Yixuan Yuan
Automatic radiology report generation has gained increasing attention for its potential to assist in clinical reporting and reduce the workload of radiologists. Existing 3D radiology report generation methods employ multi-modal foundation model to encode volume-text inputs and produce diagnosis reports, while they ignore the characteristics of 3D volumes including much background regions and suffer from generating hallucinations, especially in medical domain that contains many uncommon professional terms. In this paper, we aim to efficiently adapt the pre-trained foundation model to specific 3D radiology report generation, and present a Prior-drIven Prompting with diagnosis-oriented retrieval-Augmentation (PIPA) framework. In PIPA, we design a Prior-drIven Prompting (PIP) strategy to exploit diagnostic knowledge from input volumes and a Diagnosis-oriented volume-report retrievalaugmentation Generation (DIG) module to explore beneficial knowledge from external database. Specifically, in PIP, to take full advantage of the patient's clinical information, e.g., age and symptoms, and the possible disease information, e.g., brain tumor, edema, we formulate them as the patient and disease priors to mine clinical relevant knowledge. Furthermore, we propose utilizing visual and textual embeddings as queries to retrieve similar external data by devising a diagnosis-oriented retrieval-augmentation scheme for leveraging more report resources as references for LLM to produce accuracy outcomes. With PIP and DIG, PIPA integrates clinical priors and external data to learn effective diagnostic representations for high-quality report generation. We evaluate the framework on both public and in-house 3D medical datasets with corresponding reports, demonstrating its strong performance in generating accurate diagnosis reports. Source codes have been published at https://github.com/CUHK-AIM-Group/PIPA/tree/ main.
{"title":"PIPA: Prior-Driven Prompting with Diagnosis-Oriented Retrieval-Augmentation for 3D Radiology Report Generation.","authors":"Qiushi Yang,Wuyang Li,Xiaoqing Guo,Maymay Cerys Harwood,Peter Y M Woo,Jingyang Zhang,Yang Chen,Ke Zhang,Yixuan Yuan","doi":"10.1109/tmi.2026.3710717","DOIUrl":"https://doi.org/10.1109/tmi.2026.3710717","url":null,"abstract":"Automatic radiology report generation has gained increasing attention for its potential to assist in clinical reporting and reduce the workload of radiologists. Existing 3D radiology report generation methods employ multi-modal foundation model to encode volume-text inputs and produce diagnosis reports, while they ignore the characteristics of 3D volumes including much background regions and suffer from generating hallucinations, especially in medical domain that contains many uncommon professional terms. In this paper, we aim to efficiently adapt the pre-trained foundation model to specific 3D radiology report generation, and present a Prior-drIven Prompting with diagnosis-oriented retrieval-Augmentation (PIPA) framework. In PIPA, we design a Prior-drIven Prompting (PIP) strategy to exploit diagnostic knowledge from input volumes and a Diagnosis-oriented volume-report retrievalaugmentation Generation (DIG) module to explore beneficial knowledge from external database. Specifically, in PIP, to take full advantage of the patient's clinical information, e.g., age and symptoms, and the possible disease information, e.g., brain tumor, edema, we formulate them as the patient and disease priors to mine clinical relevant knowledge. Furthermore, we propose utilizing visual and textual embeddings as queries to retrieve similar external data by devising a diagnosis-oriented retrieval-augmentation scheme for leveraging more report resources as references for LLM to produce accuracy outcomes. With PIP and DIG, PIPA integrates clinical priors and external data to learn effective diagnostic representations for high-quality report generation. We evaluate the framework on both public and in-house 3D medical datasets with corresponding reports, demonstrating its strong performance in generating accurate diagnosis reports. Source codes have been published at https://github.com/CUHK-AIM-Group/PIPA/tree/ main.","PeriodicalId":13418,"journal":{"name":"IEEE Transactions on Medical Imaging","volume":"25 1","pages":""},"PeriodicalIF":10.6,"publicationDate":"2026-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148400555","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-07-07DOI: 10.1109/tmi.2026.3710844
Qinkai Yu,He Zhao,Yanyu Xu,Meng Wang,Yitian Zhao,Huazhu Fu,Xujiong Ye,Aline Villavicencio,Gregory Y H Lip,Yalin Zheng,Yanda Meng
Ordinal regression is well-known for lever-aging the underlying inherent order between successive categories to obtain additional regularization beyond traditional probabilistic classification mechanism. However, there are challenges in real-world medical grading tasks: 1) The uneven distribution of disease severity levels, characterized by a long-tailed format, complicates the ordinal regression process. 2)The ambiguity in establishing disease severity thresholds introduces substantial challenges, rendering the ordinal regression framework susceptible to inter-class inconsistencies. To address the challenge, this work proposes DiffGeo-AOR, by introducing an autoregressive process to ordinal regression that operates directly on continuous global features, without any need for vector quantization. DiffGeo-AOR decomposes a K-class ordinal problem into K-1 conditional binary decision steps, enabling the model to explicitly infer whether the severity has crossed the next grade threshold at each step. We also introduces parameterized diffusion optimization to model conditional probability distributions, allowing continuous global features to be extracted and directly leveraged in the autoregressive process. In addition, we design a FiLM-gated Step-Aware Diffusion Conditioning Fusion that guides each step's decision based on both the current image representation and the previous soft prediction probabilities. Furthermore, we regularize the feature space with rank-anchored ordinal priors during training to facilitate stable convergence of the autoregressive module. DiffGeo-AOR consistently outperforms current state-of-the-art ordinal regression methods across both 2D and 3D medical grading tasks on three large-scale datasets. The implementation code is publicly available at https://github.com/Qinkaiyu/DiffGeo-AOR.
{"title":"DiffGeo-AOR: Diffusion-Optimized Medical Grading via Geometric Priors enhanced Autoregressive Ordinal Regression.","authors":"Qinkai Yu,He Zhao,Yanyu Xu,Meng Wang,Yitian Zhao,Huazhu Fu,Xujiong Ye,Aline Villavicencio,Gregory Y H Lip,Yalin Zheng,Yanda Meng","doi":"10.1109/tmi.2026.3710844","DOIUrl":"https://doi.org/10.1109/tmi.2026.3710844","url":null,"abstract":"Ordinal regression is well-known for lever-aging the underlying inherent order between successive categories to obtain additional regularization beyond traditional probabilistic classification mechanism. However, there are challenges in real-world medical grading tasks: 1) The uneven distribution of disease severity levels, characterized by a long-tailed format, complicates the ordinal regression process. 2)The ambiguity in establishing disease severity thresholds introduces substantial challenges, rendering the ordinal regression framework susceptible to inter-class inconsistencies. To address the challenge, this work proposes DiffGeo-AOR, by introducing an autoregressive process to ordinal regression that operates directly on continuous global features, without any need for vector quantization. DiffGeo-AOR decomposes a K-class ordinal problem into K-1 conditional binary decision steps, enabling the model to explicitly infer whether the severity has crossed the next grade threshold at each step. We also introduces parameterized diffusion optimization to model conditional probability distributions, allowing continuous global features to be extracted and directly leveraged in the autoregressive process. In addition, we design a FiLM-gated Step-Aware Diffusion Conditioning Fusion that guides each step's decision based on both the current image representation and the previous soft prediction probabilities. Furthermore, we regularize the feature space with rank-anchored ordinal priors during training to facilitate stable convergence of the autoregressive module. DiffGeo-AOR consistently outperforms current state-of-the-art ordinal regression methods across both 2D and 3D medical grading tasks on three large-scale datasets. The implementation code is publicly available at https://github.com/Qinkaiyu/DiffGeo-AOR.","PeriodicalId":13418,"journal":{"name":"IEEE Transactions on Medical Imaging","volume":"9 1","pages":""},"PeriodicalIF":10.6,"publicationDate":"2026-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148400557","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-07-06DOI: 10.1109/tmi.2026.3710244
Jian Zhong,Li Lin,Kenneth K Y Wong,Xiaoying Tang
Universal medical image segmentation aims to unify heterogeneous datasets or annotation protocols within a single adaptable framework. However, existing prompt-based universal models often overlook background context, neglect hierarchical task dependencies, and struggle to generalize to unseen annotation granularities. These challenges are particularly pronounced in OCT-based retinal layer segmentation, where annotation schemes differ significantly across studies. To this end, we here propose UniOCTSeg++, a universal OCT segmentation framework that (1) introduces a Refined Hierarchical Prompting Strategy (RHPS) to reconstruct task-aware prompts into foreground-background paired embeddings, explicitly encoding fine-to-coarse anatomical relationships; and (2) adopts a Bi-directional Progressive Consistency Learning (BPCL) scheme that enforces mutual constraints between fine- and coarse-grained predictions under a training schedule with gradually increasing task difficulty, improving stability and mitigating pseudo-label noise. Moreover, we construct the Hierarchical Retinal OCT Segmentation Benchmark (HROCT-Bench), comprising 4.86 million OCT B-scans collected from eleven public datasets across eight annotation granularities, providing a unified evaluation protocol for universal OCT segmentation. Extensive experiments demonstrate that UniOCTSeg++ achieves state-of-the-art adaptability, reaching 90.06% DSC/ 1.38 HD95 on internal datasets and 86.83% DSC / 2.00 HD95 on external datasets. We further demonstrate UniOCTSeg++'s strong label efficiency: when trained with only 30% labeled data and supplemented with large-scale unlabeled data, UniOCTSeg++ approaches the performance of its fully supervised counterpart, highlighting its practical value for real-world deployment. The benchmark and code will be released at https://github.com/Halcyon1010/UniOCTSeg++.
{"title":"UniOCTSeg++: Refined Hierarchical Prompt Strategy and Bi-directional Progressive Consistency Learning for Universal Retinal Layer Segmentation in OCT.","authors":"Jian Zhong,Li Lin,Kenneth K Y Wong,Xiaoying Tang","doi":"10.1109/tmi.2026.3710244","DOIUrl":"https://doi.org/10.1109/tmi.2026.3710244","url":null,"abstract":"Universal medical image segmentation aims to unify heterogeneous datasets or annotation protocols within a single adaptable framework. However, existing prompt-based universal models often overlook background context, neglect hierarchical task dependencies, and struggle to generalize to unseen annotation granularities. These challenges are particularly pronounced in OCT-based retinal layer segmentation, where annotation schemes differ significantly across studies. To this end, we here propose UniOCTSeg++, a universal OCT segmentation framework that (1) introduces a Refined Hierarchical Prompting Strategy (RHPS) to reconstruct task-aware prompts into foreground-background paired embeddings, explicitly encoding fine-to-coarse anatomical relationships; and (2) adopts a Bi-directional Progressive Consistency Learning (BPCL) scheme that enforces mutual constraints between fine- and coarse-grained predictions under a training schedule with gradually increasing task difficulty, improving stability and mitigating pseudo-label noise. Moreover, we construct the Hierarchical Retinal OCT Segmentation Benchmark (HROCT-Bench), comprising 4.86 million OCT B-scans collected from eleven public datasets across eight annotation granularities, providing a unified evaluation protocol for universal OCT segmentation. Extensive experiments demonstrate that UniOCTSeg++ achieves state-of-the-art adaptability, reaching 90.06% DSC/ 1.38 HD95 on internal datasets and 86.83% DSC / 2.00 HD95 on external datasets. We further demonstrate UniOCTSeg++'s strong label efficiency: when trained with only 30% labeled data and supplemented with large-scale unlabeled data, UniOCTSeg++ approaches the performance of its fully supervised counterpart, highlighting its practical value for real-world deployment. The benchmark and code will be released at https://github.com/Halcyon1010/UniOCTSeg++.","PeriodicalId":13418,"journal":{"name":"IEEE Transactions on Medical Imaging","volume":"190 1","pages":""},"PeriodicalIF":10.6,"publicationDate":"2026-07-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148393513","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-07-06DOI: 10.1109/tmi.2026.3710133
Nora E Fitzgerald,Gabriel Montaldo,Mathilda Froesel,Alan Urban,Wim Vanduffel
Linking circuit level activity to large scale functional organization requires imaging methods combining high spatial resolution, broad coverage, and single trial sensitivity. We present volumetric functional ultrasound imaging (3D-fUS) in behaving macaques, enabling imaging of ~1 cm³ cortical volumes at high spatiotemporal resolution (100 × 150 × 150 μm³ voxels, 1.67 Hz). Visually evoked responses were reliably detected at the level of single trials and single voxels, substantially reducing experimental time. To enable model-based analyses analogous to functional magnetic resonance imaging (fMRI), we estimated a fUS hemodynamic response function (fUS-HRF) that was consistent across subjects, cortical areas, and visual stimuli and was well approximated by a gamma function. Compared with fMRI HRFs, the fUS-HRF exhibited faster dynamics, enabling shorter and more closely spaced stimulus presentations. Together, these results establish 3D-fUS as a fast, volumetric, and circuit relevant imaging modality for efficient investigation of distributed cortical dynamics in primates.
{"title":"Volumetric Functional Ultrasound Imaging in Macaques.","authors":"Nora E Fitzgerald,Gabriel Montaldo,Mathilda Froesel,Alan Urban,Wim Vanduffel","doi":"10.1109/tmi.2026.3710133","DOIUrl":"https://doi.org/10.1109/tmi.2026.3710133","url":null,"abstract":"Linking circuit level activity to large scale functional organization requires imaging methods combining high spatial resolution, broad coverage, and single trial sensitivity. We present volumetric functional ultrasound imaging (3D-fUS) in behaving macaques, enabling imaging of ~1 cm³ cortical volumes at high spatiotemporal resolution (100 × 150 × 150 μm³ voxels, 1.67 Hz). Visually evoked responses were reliably detected at the level of single trials and single voxels, substantially reducing experimental time. To enable model-based analyses analogous to functional magnetic resonance imaging (fMRI), we estimated a fUS hemodynamic response function (fUS-HRF) that was consistent across subjects, cortical areas, and visual stimuli and was well approximated by a gamma function. Compared with fMRI HRFs, the fUS-HRF exhibited faster dynamics, enabling shorter and more closely spaced stimulus presentations. Together, these results establish 3D-fUS as a fast, volumetric, and circuit relevant imaging modality for efficient investigation of distributed cortical dynamics in primates.","PeriodicalId":13418,"journal":{"name":"IEEE Transactions on Medical Imaging","volume":"815 1","pages":""},"PeriodicalIF":10.6,"publicationDate":"2026-07-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148393514","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Multi-style virtual staining transforms histological images into multiple staining modalities, offering significant clinical value at reduced cost and time. However, a critical challenge impeding clinical adoption is incompletely paired training data-an inevitable consequence of tissue degradation and processing artifacts during sequential staining. Current methods assume perfectly paired datasets, severely limiting their clinical utility. We address this problem by introducing MUST (MUlti-style virtual STaining), which reformulates virtual staining as progressive cross-modality refinement under incomplete supervision. Our approach comprises two synergistic components: (1) Collaborative Denoising (CoDe) that uses cross-modality cross attention to condition a latent diffusion model, enabling effective information exchange across modalities with incomplete supervision, and (2) Semantic Preservation (SP) that further maintains cross-modal consistency through contrastive learning while generating reliable pseudo-supervision from confident model predictions in samples without ground truth. Extensive experiments across three histopathology datasets demonstrate that MUST significantly outperforms state-of-the-art methods, effectively mining cross-modality correlations while generating high-confidence pseudo-supervision from incomplete data. Code and trained models will be publicly released upon publication at https://github.com/JiaxinZhuang/MUST.
{"title":"MUST: Multi-style virtual staining with incomplete pairs.","authors":"Jiaxin Zhuang,Yao DU,Xiaoyu Zheng,Linshan Wu,Chao He,Lin Luo,Hao Chen","doi":"10.1109/tmi.2026.3709810","DOIUrl":"https://doi.org/10.1109/tmi.2026.3709810","url":null,"abstract":"Multi-style virtual staining transforms histological images into multiple staining modalities, offering significant clinical value at reduced cost and time. However, a critical challenge impeding clinical adoption is incompletely paired training data-an inevitable consequence of tissue degradation and processing artifacts during sequential staining. Current methods assume perfectly paired datasets, severely limiting their clinical utility. We address this problem by introducing MUST (MUlti-style virtual STaining), which reformulates virtual staining as progressive cross-modality refinement under incomplete supervision. Our approach comprises two synergistic components: (1) Collaborative Denoising (CoDe) that uses cross-modality cross attention to condition a latent diffusion model, enabling effective information exchange across modalities with incomplete supervision, and (2) Semantic Preservation (SP) that further maintains cross-modal consistency through contrastive learning while generating reliable pseudo-supervision from confident model predictions in samples without ground truth. Extensive experiments across three histopathology datasets demonstrate that MUST significantly outperforms state-of-the-art methods, effectively mining cross-modality correlations while generating high-confidence pseudo-supervision from incomplete data. Code and trained models will be publicly released upon publication at https://github.com/JiaxinZhuang/MUST.","PeriodicalId":13418,"journal":{"name":"IEEE Transactions on Medical Imaging","volume":"1 1","pages":""},"PeriodicalIF":10.6,"publicationDate":"2026-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148379857","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neuron segmentation in complex mouse brain images improves neuron reconstruction and supports studies of brain structure and function, while the existing deep learning-based methods do not sufficiently exploit prior information, including neuronal morphology and imaging mechanism. We propose NUNet-LLM, the first LLM-integrated framework for neuron segmentation and reconstruction. NUNet-LLM consists of an LLM-based text path and a 3D UNet-based image path to extract and fuse multi-modal features, guiding the deep model to better focus on slender nerve fibers. The text path leverages two pre-trained LLMs to generate dataset- and task-level textual descriptions and compute static textual features in advance, so no additional LLM inference is required during testing. The image path combines a 3D UNet with wavelet transform and an attention mechanism; its encoder extracts robust image features, and after fusion with textual features, its decoder predicts segmentation masks. To train NUNet-LLM, we constructed a mouse brain neuronal cube dataset (mNeuCuDa) from 18 manually annotated neurons in mouse brain images, and introduce a synthetic dataset (sNeuCuDa) to reduce interference from the unlabeled nerve fibers. In addition, we designed a topology structure loss by combining cross-entropy, structure loss, and edgeaware loss. After segmenting, an automatic algorithm was applied to reconstruct intertwined neurons in the neuronal images, and G-Cut was utilized to decouple them. Experiments on mouse brain neuronal images and BigNeuron demonstrate the effectiveness of NUNet-LLM for neuron segmentation and reconstruction.
{"title":"LLM-enhanced Neuron Segmentation and Reconstruction in Complex Mouse Brain Images.","authors":"Chengda Mo,Xinle Dai,Qiufu Li,Linlin Shen,Cheng Zhao","doi":"10.1109/tmi.2026.3709050","DOIUrl":"https://doi.org/10.1109/tmi.2026.3709050","url":null,"abstract":"Neuron segmentation in complex mouse brain images improves neuron reconstruction and supports studies of brain structure and function, while the existing deep learning-based methods do not sufficiently exploit prior information, including neuronal morphology and imaging mechanism. We propose NUNet-LLM, the first LLM-integrated framework for neuron segmentation and reconstruction. NUNet-LLM consists of an LLM-based text path and a 3D UNet-based image path to extract and fuse multi-modal features, guiding the deep model to better focus on slender nerve fibers. The text path leverages two pre-trained LLMs to generate dataset- and task-level textual descriptions and compute static textual features in advance, so no additional LLM inference is required during testing. The image path combines a 3D UNet with wavelet transform and an attention mechanism; its encoder extracts robust image features, and after fusion with textual features, its decoder predicts segmentation masks. To train NUNet-LLM, we constructed a mouse brain neuronal cube dataset (mNeuCuDa) from 18 manually annotated neurons in mouse brain images, and introduce a synthetic dataset (sNeuCuDa) to reduce interference from the unlabeled nerve fibers. In addition, we designed a topology structure loss by combining cross-entropy, structure loss, and edgeaware loss. After segmenting, an automatic algorithm was applied to reconstruct intertwined neurons in the neuronal images, and G-Cut was utilized to decouple them. Experiments on mouse brain neuronal images and BigNeuron demonstrate the effectiveness of NUNet-LLM for neuron segmentation and reconstruction.","PeriodicalId":13418,"journal":{"name":"IEEE Transactions on Medical Imaging","volume":"15 1","pages":""},"PeriodicalIF":10.6,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148365090","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2026-07-01DOI: 10.1109/tmi.2026.3709056
John M Drago,Georgy D Guryev,Nicolas Arango,Elfar Adalsteinsson,Bastien Guerin,Lawrence L Wald
High-field magnetic resonance imaging (MRI) suffers from pronounced magnetic field inhomogeneities and subject-specific field variations, motivating the inscanner design of individually tailored excitation pulses to exploit the full capabilities of high-field MRI. Contemporary methods may employ piecewise-constant (PWC) waveform parameterizations to design excitation pulses, which require many optimization variables and hinder rapid, inscanner customization. We represent the radiofrequency (RF) and gradient waveforms in a global waveform basis, demonstrated using a Chebyshev polynomial basis, to reduce problem dimensionality and accelerate convergence, while ensuring waveform smoothness. The adjoint method efficiently computes derivatives of the excitation objective function with respect to the basis coefficients, which each influence an entire waveform. GPU acceleration of derivative computation further reduces computation time, while system and safety constraints are enforced throughout the optimization. Using this global waveform basis yields an approximate five- to ten-fold speedup in subject-specific (tailored) pulse optimization for non-selective excitations and, in the best case, comparable gains for slice-selective designs, making real-time, subject-specific pulse optimization feasible even for advanced pulse types.
{"title":"The Ritz Adjoint Method for MRI Pulse Design.","authors":"John M Drago,Georgy D Guryev,Nicolas Arango,Elfar Adalsteinsson,Bastien Guerin,Lawrence L Wald","doi":"10.1109/tmi.2026.3709056","DOIUrl":"https://doi.org/10.1109/tmi.2026.3709056","url":null,"abstract":"High-field magnetic resonance imaging (MRI) suffers from pronounced magnetic field inhomogeneities and subject-specific field variations, motivating the inscanner design of individually tailored excitation pulses to exploit the full capabilities of high-field MRI. Contemporary methods may employ piecewise-constant (PWC) waveform parameterizations to design excitation pulses, which require many optimization variables and hinder rapid, inscanner customization. We represent the radiofrequency (RF) and gradient waveforms in a global waveform basis, demonstrated using a Chebyshev polynomial basis, to reduce problem dimensionality and accelerate convergence, while ensuring waveform smoothness. The adjoint method efficiently computes derivatives of the excitation objective function with respect to the basis coefficients, which each influence an entire waveform. GPU acceleration of derivative computation further reduces computation time, while system and safety constraints are enforced throughout the optimization. Using this global waveform basis yields an approximate five- to ten-fold speedup in subject-specific (tailored) pulse optimization for non-selective excitations and, in the best case, comparable gains for slice-selective designs, making real-time, subject-specific pulse optimization feasible even for advanced pulse types.","PeriodicalId":13418,"journal":{"name":"IEEE Transactions on Medical Imaging","volume":"136 1","pages":""},"PeriodicalIF":10.6,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148365089","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：