首页 > 最新文献

ArXiv最新文献

英文 中文
Fully 3D Unrolled Magnetic Resonance Fingerprinting Reconstruction via Staged Pretraining and Implicit Gridding. 全三维展开磁共振指纹重建通过阶段预训练和隐式网格。
Pub Date : 2026-02-12
Yonatan Urman, Mark Nishimura, Daniel Abraham, Xiaozhi Cao, Kawin Setsompop

Magnetic Resonance Fingerprinting (MRF) enables fast quantitative imaging, yet reconstructing high-resolution 3D data remains computationally demanding. Non-Cartesian reconstructions require repeated non-uniform FFTs, and the commonly used Locally Low Rank (LLR) prior adds computational overhead and becomes insufficient at high accelerations. Learned 3D priors could address these limitations, but training them at scale is challenging due to memory and runtime demands. We propose SPUR-iG, a fully 3D deep unrolled subspace reconstruction framework that integrates efficient data consistency with a progressive training strategy. Data consistency leverages implicit GROG, which grids non-Cartesian data onto a Cartesian grid with an implicitly learned kernel, enabling FFT-based updates with minimal artifacts. Training proceeds in three stages: (1) pretraining a denoiser with extensive data augmentation, (2) greedy per-iteration unrolled training, and (3) final fine-tuning with gradient checkpointing. Together, these stages make large-scale 3D unrolled learning feasible within a reasonable compute budget. On a large in vivo dataset with retrospective undersampling, SPUR-iG improves subspace coefficient maps quality and quantitative accuracy at 1-mm isotropic resolution compared with LLR and a hybrid 2D/3D unrolled baseline. Whole-brain reconstructions complete in under 15-seconds, with up to $times$111 speedup for 2-minute acquisitions. Notably, $T_1$ maps with our method from 30-second scans achieve accuracy on par with or exceeding LLR reconstructions from 2-minute scans. Overall, the framework improves both accuracy and speed in large-scale 3D MRF reconstruction, enabling efficient and reliable accelerated quantitative imaging.

磁共振指纹(MRF)可以实现快速定量成像,但重建高分辨率3D数据仍然需要计算。非笛卡尔重建需要重复的非均匀fft,通常使用的局部低秩(LLR)先验增加了计算开销,并且在高加速度下变得不足。学习3D先验可以解决这些限制,但由于内存和运行时需求,大规模训练它们是具有挑战性的。我们提出了spr - ig,这是一个全3D深度展开子空间重建框架,它将高效的数据一致性与渐进式训练策略相结合。数据一致性利用隐式GROG,它将非笛卡尔数据网格化到具有隐式学习内核的笛卡尔网格上,以最少的工件实现基于fft的更新。训练分三个阶段进行:(1)使用广泛的数据增强预训练去噪器,(2)贪婪的逐迭代展开训练,以及(3)使用梯度检查点进行最终微调。总之,这些阶段使大规模3D展开学习在合理的计算预算内可行。在具有回顾性欠采样的大型体内数据集上,与LLR和混合2D/3D展开基线相比,spr - ig在1毫米各向同性分辨率下提高了子空间系数图的质量和定量精度。全脑重建在15秒内完成,对于2分钟的采集,加速高达111美元。值得注意的是,使用我们的方法进行30秒扫描的$T_1$映射,其精度相当于或超过2分钟扫描的LLR重建。总体而言,该框架提高了大规模3D磁共振成像重建的准确性和速度,实现了高效可靠的加速定量成像。
{"title":"Fully 3D Unrolled Magnetic Resonance Fingerprinting Reconstruction via Staged Pretraining and Implicit Gridding.","authors":"Yonatan Urman, Mark Nishimura, Daniel Abraham, Xiaozhi Cao, Kawin Setsompop","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Magnetic Resonance Fingerprinting (MRF) enables fast quantitative imaging, yet reconstructing high-resolution 3D data remains computationally demanding. Non-Cartesian reconstructions require repeated non-uniform FFTs, and the commonly used Locally Low Rank (LLR) prior adds computational overhead and becomes insufficient at high accelerations. Learned 3D priors could address these limitations, but training them at scale is challenging due to memory and runtime demands. We propose SPUR-iG, a fully 3D deep unrolled subspace reconstruction framework that integrates efficient data consistency with a progressive training strategy. Data consistency leverages implicit GROG, which grids non-Cartesian data onto a Cartesian grid with an implicitly learned kernel, enabling FFT-based updates with minimal artifacts. Training proceeds in three stages: (1) pretraining a denoiser with extensive data augmentation, (2) greedy per-iteration unrolled training, and (3) final fine-tuning with gradient checkpointing. Together, these stages make large-scale 3D unrolled learning feasible within a reasonable compute budget. On a large in vivo dataset with retrospective undersampling, SPUR-iG improves subspace coefficient maps quality and quantitative accuracy at 1-mm isotropic resolution compared with LLR and a hybrid 2D/3D unrolled baseline. Whole-brain reconstructions complete in under 15-seconds, with up to $times$111 speedup for 2-minute acquisitions. Notably, $T_1$ maps with our method from 30-second scans achieve accuracy on par with or exceeding LLR reconstructions from 2-minute scans. Overall, the framework improves both accuracy and speed in large-scale 3D MRF reconstruction, enabling efficient and reliable accelerated quantitative imaging.</p>","PeriodicalId":93888,"journal":{"name":"ArXiv","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2026-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12869386/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146127571","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Biomechanically Informed Image Registration for Patient-Specific Aortic Valve Strain Analysis. 生物力学信息图像配准用于患者特异性主动脉瓣应变分析。
Pub Date : 2026-02-12
Mohsen Nakhaei, Alison M Pouch, Silvani Amin, Matthew Daemer, Christian Herz, Natalie Yushkevich, Lourdes Al Ghofaily, Nimesh Desai, Joseph Bavaria, Matthew A Jolley, Wensi Wu

Purpose: Aortic valve (AV) biomechanics play a critical role in maintaining normal cardiac function. Pathological variations, particularly in bicuspid aortic valves, alter leaflet loading, increase strain, and accelerate disease progression. Accurate patient-specific characterization of valve geometry and deformation is therefore essential for predicting disease progression and guiding durable repair. However, existing imaging and computational methods often fail to capture rapid valve motion and complex patient-specific features, limiting precise biomechanical assessment.

Methods: To address these limitations, we developed an image registration framework coupled with the finite element method (FEM) to improve AV tracking and biomechanical evaluation. Patient-specific valve geometries derived from 4D echocardiography and CT were used to simulate AV closure and generate intermediate deformation states. These FEM-generated states facilitated leaflet tracking, while image registration corrected misalignment between simulations and imaging data.

Results: In 20 patients, FEM-augmented registration improved tracking accuracy by 40% compared with direct registration. This improvement enabled more reliable strain estimation by measuring leaflet deformation directly from imaging and reducing uncertainties associated with boundary conditions and material assumptions. Using the improved tracking results, areal, Green-Lagrange, and deviatoric strains were quantified in adult trileaflet and bicuspid valves, as well as pediatric patients, revealing distinct deformation patterns across valve groups. Convergence in mean deviatoric strain between adult trileaflet and pediatric valves suggests volumetric deformation underlies age- and size-related differences in AV mechanics.

Conclusion: Overall, this FEM-augmented registration framework enhances geometric tracking and biomechanical evaluation accuracy, providing clinically relevant insights into patient-specific AV deformation to support individualized medical and intervention planning.

主动脉瓣生物力学在维持正常心功能中起着至关重要的作用。病理变异,特别是在二尖瓣主动脉瓣(bav),改变小叶负荷,增加应变,并加速疾病进展。准确的、患者特异性的瓣膜几何形状和变形特征对于预测疾病进展和指导持久修复至关重要。目前的成像和计算方法往往不能捕捉快速的瓣膜运动和复杂的患者特异性特征。为了解决这些挑战,我们将图像配准与有限元法(FEM)相结合,以增强AV跟踪和生物力学评估。通过4D经食管超声心动图(TEE)和CT的患者特异性瓣膜几何形状在FEM中模拟房室关闭并产生中间变形状态。fem生成的状态促进了传单的跟踪,而配准算法则纠正了模拟与图像之间的不匹配。在20名患者中,与直接登记相比,fem增强登记的准确性提高了40% (TEE为33%,CT为46%)。这种改进可以直接从成像中获得更可靠的应变估计,并减少边界条件和材料假设的不确定性。在成人三叶/二尖叶和儿童患者中定量测定Areal和Green-Lagrange菌株以及有效菌株。成年三叶瓣膜表现为均匀变形,bav表现为不对称应变,儿童瓣膜具有低平均面应变和高变异性。成人和儿童三叶瓣膜的平均有效应变趋同表明体积变形驱动年龄和尺寸相关的差异。fem增强的注册框架增强了几何跟踪,并为患者特异性房室变形提供了临床相关的见解,支持个性化干预计划。
{"title":"Biomechanically Informed Image Registration for Patient-Specific Aortic Valve Strain Analysis.","authors":"Mohsen Nakhaei, Alison M Pouch, Silvani Amin, Matthew Daemer, Christian Herz, Natalie Yushkevich, Lourdes Al Ghofaily, Nimesh Desai, Joseph Bavaria, Matthew A Jolley, Wensi Wu","doi":"","DOIUrl":"","url":null,"abstract":"<p><strong>Purpose: </strong>Aortic valve (AV) biomechanics play a critical role in maintaining normal cardiac function. Pathological variations, particularly in bicuspid aortic valves, alter leaflet loading, increase strain, and accelerate disease progression. Accurate patient-specific characterization of valve geometry and deformation is therefore essential for predicting disease progression and guiding durable repair. However, existing imaging and computational methods often fail to capture rapid valve motion and complex patient-specific features, limiting precise biomechanical assessment.</p><p><strong>Methods: </strong>To address these limitations, we developed an image registration framework coupled with the finite element method (FEM) to improve AV tracking and biomechanical evaluation. Patient-specific valve geometries derived from 4D echocardiography and CT were used to simulate AV closure and generate intermediate deformation states. These FEM-generated states facilitated leaflet tracking, while image registration corrected misalignment between simulations and imaging data.</p><p><strong>Results: </strong>In 20 patients, FEM-augmented registration improved tracking accuracy by 40% compared with direct registration. This improvement enabled more reliable strain estimation by measuring leaflet deformation directly from imaging and reducing uncertainties associated with boundary conditions and material assumptions. Using the improved tracking results, areal, Green-Lagrange, and deviatoric strains were quantified in adult trileaflet and bicuspid valves, as well as pediatric patients, revealing distinct deformation patterns across valve groups. Convergence in mean deviatoric strain between adult trileaflet and pediatric valves suggests volumetric deformation underlies age- and size-related differences in AV mechanics.</p><p><strong>Conclusion: </strong>Overall, this FEM-augmented registration framework enhances geometric tracking and biomechanical evaluation accuracy, providing clinically relevant insights into patient-specific AV deformation to support individualized medical and intervention planning.</p>","PeriodicalId":93888,"journal":{"name":"ArXiv","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2026-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12803325/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145992117","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
MOTGNN: Interpretable Graph Neural Networks for Multi-Omics Disease Classification. 多组疾病分类的可解释图神经网络。
Pub Date : 2026-02-11
Tiantian Yang, Zhiqian Chen

Integrating multi-omics data, such as DNA methylation, mRNA expression, and microRNA (miRNA) expression, offers a comprehensive view of the biological mechanisms underlying disease. However, the high dimensionality of multi-omics data, the heterogeneity across modalities, and the lack of reliable biological interaction networks make meaningful integration challenging. In addition, many existing models rely on handcrafted similarity graphs, are vulnerable to class imbalance, and often lack built-in interpretability, limiting their usefulness in biomedical applications. We propose Multi-Omics integration with Tree-generated Graph Neural Network (MOTGNN), a novel and interpretable framework for binary disease classification. MOTGNN employs eXtreme Gradient Boosting (XGBoost) for omics-specific supervised graph construction, followed by modality-specific Graph Neural Networks (GNNs) for hierarchical representation learning, and a deep feedforward network for cross-omics integration. Across three real-world disease datasets, MOTGNN outperforms state-of-the-art baselines by 5-10% in accuracy, ROC-AUC, and F1-score, and remains robust to severe class imbalance. The model maintains computational efficiency through the use of sparse graphs and provides built-in interpretability, revealing both top-ranked biomarkers and the relative contributions of each omics modality. These results highlight the potential of MOTGNN to improve both predictive accuracy and interpretability in multi-omics disease modeling.

整合多组学数据,如DNA甲基化、mRNA表达和microRNA (miRNA)表达,可以全面了解疾病的生物学机制。然而,多组学数据的高维性、跨模式的异质性以及缺乏可靠的生物相互作用网络使得有意义的集成具有挑战性。此外,许多现有的模型依赖于手工制作的相似图,容易受到类不平衡的影响,并且往往缺乏内置的可解释性,限制了它们在生物医学应用中的用处。我们提出多组学集成与树生成图神经网络(MOTGNN),一个新的和可解释的框架二元疾病分类。MOTGNN采用极端梯度增强(XGBoost)进行组学特定的监督图构建,然后使用特定模态的图神经网络(gnn)进行分层表示学习,并使用深度前馈网络进行跨组学集成。在三个真实世界疾病数据集中,MOTGNN在准确率、ROC-AUC和f1评分方面优于最先进的基线5-10%,并且对严重的类别不平衡保持稳健。该模型通过使用稀疏图来保持计算效率,并提供内置的可解释性,揭示了排名最高的生物标志物和每种组学模式的相对贡献。这些结果突出了MOTGNN在多组学疾病建模中提高预测准确性和可解释性的潜力。
{"title":"MOTGNN: Interpretable Graph Neural Networks for Multi-Omics Disease Classification.","authors":"Tiantian Yang, Zhiqian Chen","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Integrating multi-omics data, such as DNA methylation, mRNA expression, and microRNA (miRNA) expression, offers a comprehensive view of the biological mechanisms underlying disease. However, the high dimensionality of multi-omics data, the heterogeneity across modalities, and the lack of reliable biological interaction networks make meaningful integration challenging. In addition, many existing models rely on handcrafted similarity graphs, are vulnerable to class imbalance, and often lack built-in interpretability, limiting their usefulness in biomedical applications. We propose Multi-Omics integration with Tree-generated Graph Neural Network (<b>MOTGNN</b>), a novel and interpretable framework for binary disease classification. MOTGNN employs eXtreme Gradient Boosting (XGBoost) for omics-specific supervised graph construction, followed by modality-specific Graph Neural Networks (GNNs) for hierarchical representation learning, and a deep feedforward network for cross-omics integration. Across three real-world disease datasets, MOTGNN outperforms state-of-the-art baselines by 5-10% in accuracy, ROC-AUC, and F1-score, and remains robust to severe class imbalance. The model maintains computational efficiency through the use of sparse graphs and provides built-in interpretability, revealing both top-ranked biomarkers and the relative contributions of each omics modality. These results highlight the potential of MOTGNN to improve both predictive accuracy and interpretability in multi-omics disease modeling.</p>","PeriodicalId":93888,"journal":{"name":"ArXiv","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2026-02-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12919230/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147273357","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Universal Approximation Theorems for Dynamical Systems with Infinite-Time Horizon Guarantees. 具有无限时间范围保证的动力系统的普遍逼近定理。
Pub Date : 2026-02-11
Ábel Ságodi, Il Memming Park

Universal approximation theorems establish the expressive capacity of neural network architectures. For dynamical systems, existing results are limited to finite time horizons or systems with a globally stable equilibrium, leaving multistability and limit cycles unaddressed. We prove that Neural ODEs achieve ε - δ closeness-trajectories within error ε except for initial conditions of measure < δ -over the infinite time horizon [ 0 , ) for three target classes: (1) Morse-Smale systems (a structurally stable class) with hyperbolic fixed points, (2) Morse-Smale systems with hyperbolic limit cycles via exact period matching, and (3) systems with normally hyperbolic continuous attractors via discretization. We further establish a temporal generalization bound: ε - δ closeness implies L p error ε p + δ D p for all t 0 , bridging topological guarantees to training metrics. These results provide the first universal approximation framework for multistable infinite-horizon dynamics.

通用逼近定理建立了神经网络结构的表达能力。对于动力系统,现有的结果仅限于有限的时间范围或具有全局稳定平衡的系统,留下了未解决的多稳定性和极限环。我们证明了神经ode在emph{无限}时间范围$[0,infty)$上对三个目标类实现$varepsilon$ - $δ$接近度——误差范围内的轨迹$varepsilon$(除了测量$< δ$的初始条件):(1)具有双曲不动点的morse - small系统(结构稳定的一类),(2)具有精确周期匹配的双曲极限环的morse - small系统,以及(3)具有通常双曲连续吸引子的离散化系统。我们进一步建立了一个时间泛化界:$varepsilon$ - $δ$接近性意味着所有$t geq 0$的$L^p$误差$leq varepsilon^p + δcdot D^p$,桥接拓扑保证到训练指标。这些结果提供了多稳定无限视界动力学的第一个通用近似框架。
{"title":"Universal Approximation Theorems for Dynamical Systems with Infinite-Time Horizon Guarantees.","authors":"Ábel Ságodi, Il Memming Park","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Universal approximation theorems establish the expressive capacity of neural network architectures. For dynamical systems, existing results are limited to finite time horizons or systems with a globally stable equilibrium, leaving multistability and limit cycles unaddressed. We prove that Neural ODEs achieve <math><mi>ε</mi> <mo>-</mo> <mi>δ</mi></math> closeness-trajectories within error <math><mi>ε</mi></math> except for initial conditions of measure <math><mo><</mo> <mi>δ</mi></math> -over the <i>infinite</i> time horizon <math><mo>[</mo> <mn>0</mn> <mo>,</mo> <mi>∞</mi> <mo>)</mo></math> for three target classes: (1) Morse-Smale systems (a structurally stable class) with hyperbolic fixed points, (2) Morse-Smale systems with hyperbolic limit cycles via exact period matching, and (3) systems with normally hyperbolic continuous attractors via discretization. We further establish a temporal generalization bound: <math><mi>ε</mi> <mo>-</mo> <mi>δ</mi></math> closeness implies <math> <msup><mrow><mi>L</mi></mrow> <mrow><mi>p</mi></mrow> </msup> </math> error <math><mo>≤</mo> <msup><mrow><mi>ε</mi></mrow> <mrow><mi>p</mi></mrow> </msup> <mo>+</mo> <mi>δ</mi> <mo>⋅</mo> <msup><mrow><mi>D</mi></mrow> <mrow><mi>p</mi></mrow> </msup> </math> for all <math><mi>t</mi> <mo>≥</mo> <mn>0</mn></math> , bridging topological guarantees to training metrics. These results provide the first universal approximation framework for multistable infinite-horizon dynamics.</p>","PeriodicalId":93888,"journal":{"name":"ArXiv","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2026-02-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12919221/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147273430","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
TF-DWGNet: A Directed Weighted Graph Neural Network with Tensor Fusion for Multi-Omics Cancer Subtype Classification. TF-DWGNet:基于张量融合的有向加权图神经网络用于多组癌亚型分类。
Pub Date : 2026-02-11
Tiantian Yang, Zhiqian Chen

Integration and analysis of multi-omics data provide valuable insights for improving cancer subtype classification. However, such data are inherently heterogeneous, high-dimensional, and exhibit complex intra- and inter-modality dependencies. Graph neural networks (GNNs) offer a principled framework for modeling these structures, but existing approaches often rely on prior knowledge or predefined similarity networks that produce undirected or unweighted graphs and fail to capture task-specific directionality and interaction strength. Interpretability at both the modality and feature levels also remains limited. To address these challenges, we propose TF-DWGNet, a novel Graph Neural Network framework that combines tree-based Directed Weighted graph construction with Tensor Fusion for multiclass cancer subtype classification. TF-DWGNet introduces two key innovations: (i) a supervised tree-based strategy that constructs directed, weighted graphs tailored to each omics modality, and (ii) a tensor fusion mechanism that captures unimodal, bimodal, and trimodal interactions using low-rank decomposition for computational efficiency. Experiments on three real-world cancer datasets demonstrate that TF-DWGNet consistently outperforms state-of-the-art baselines across multiple metrics and statistical tests. In addition, the model provides biologically meaningful insights through modality-level contribution scores and ranked feature importance. These results highlight that TF-DWGNet is an effective and interpretable solution for multi-omics integration in cancer research.

多组学数据的整合和分析为改进癌症亚型分类提供了有价值的见解。然而,这些数据本质上是异构的、高维的,并且表现出复杂的模态内部和模态之间的依赖关系。图神经网络(gnn)为这些结构的建模提供了一个有原则的框架,但现有的方法通常依赖于先验知识或预定义的相似性网络,这些相似性网络产生无向或无加权的图,并且无法捕获特定于任务的方向性和交互强度。在模态和特征层面上的可解释性仍然有限。为了解决这些挑战,我们提出了TF-DWGNet,这是一种新的图神经网络框架,将基于树的有向加权图构建与张量融合相结合,用于多类别癌症亚型分类。TF-DWGNet引入了两个关键创新:(i)基于监督树的策略,构建针对每个组学模态的有向加权图;(ii)张量融合机制,使用低秩分解捕获单峰、双峰和三峰相互作用,以提高计算效率。在三个真实世界癌症数据集上的实验表明,TF-DWGNet在多个指标和统计测试中始终优于最先进的基线。此外,该模型通过模式级贡献分数和特征重要性排名提供了生物学上有意义的见解。这些结果表明TF-DWGNet是癌症研究中多组学整合的有效且可解释的解决方案。
{"title":"TF-DWGNet: A Directed Weighted Graph Neural Network with Tensor Fusion for Multi-Omics Cancer Subtype Classification.","authors":"Tiantian Yang, Zhiqian Chen","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Integration and analysis of multi-omics data provide valuable insights for improving cancer subtype classification. However, such data are inherently heterogeneous, high-dimensional, and exhibit complex intra- and inter-modality dependencies. Graph neural networks (GNNs) offer a principled framework for modeling these structures, but existing approaches often rely on prior knowledge or predefined similarity networks that produce undirected or unweighted graphs and fail to capture task-specific directionality and interaction strength. Interpretability at both the modality and feature levels also remains limited. To address these challenges, we propose <b>TF-DWGNet</b>, a novel <b>G</b>raph Neural <b>N</b>etwork framework that combines tree-based <b>D</b>irected <b>W</b>eighted graph construction with <b>T</b>ensor <b>F</b>usion for multiclass cancer subtype classification. TF-DWGNet introduces two key innovations: (i) a supervised tree-based strategy that constructs directed, weighted graphs tailored to each omics modality, and (ii) a tensor fusion mechanism that captures unimodal, bimodal, and trimodal interactions using low-rank decomposition for computational efficiency. Experiments on three real-world cancer datasets demonstrate that TF-DWGNet consistently outperforms state-of-the-art baselines across multiple metrics and statistical tests. In addition, the model provides biologically meaningful insights through modality-level contribution scores and ranked feature importance. These results highlight that TF-DWGNet is an effective and interpretable solution for multi-omics integration in cancer research.</p>","PeriodicalId":93888,"journal":{"name":"ArXiv","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2026-02-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12919224/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147273353","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
ClusterChirp: A GPU-accelerated Web Server for Natural Language-Guided Interactive Visualization and Analysis of Large Omics Data. ClusterChirp:一个gpu加速的Web服务器,用于自然语言引导的交互式可视化和大型组学数据分析。
Pub Date : 2026-02-09
Osho Rawal, Rex Lu, Edgar Gonzalez-Kozlova, Sacha Gnjatic, Zeynep H Gümüş

Tabular datasets are commonly visualized as heatmaps, where data values are represented as color intensities in a matrix to reveal patterns and correlations. However, modern omics technologies increasingly generate matrices so large that existing visual exploration tools require downsampling or filtering, risking loss of biologically important patterns. Additional barriers arise from tools that require command-line expertise, or fragmented workflows for downstream biological interpretation. We present ClusterChirp, a web-based platform for real-time, interactive exploration of large-scale data matrices enabled by GPU-accelerated rendering and parallelized hierarchical clustering using multiple CPU cores. Built on deck.gl and multi-threaded clustering algorithms, ClusterChirp supports on-the-fly clustering, multi-metric sorting, feature search, and adjustable visualization parameters for interactive explorations. Uniquely, a natural language interface powered by a Large Language Model helps users perform complex operations and build reproducible workflows from conversational commands. Furthermore, users can select clusters to explore interactive within-cluster correlation networks in 2D or 3D, or perform functional enrichment through biological knowledge bases. Developed with iterative user feedback and adhering to FAIR4S principles, ClusterChirp empowers researchers to extract insights from high-dimensional omics data with unprecedented ease and speed. This website is freely available at clusterchirp.mssm.edu, with no login required.

表格数据集通常可视化为热图,其中数据值表示为矩阵中的颜色强度,以显示模式和相关性。然而,现代组学技术越来越多地生成如此大的矩阵,现有的视觉探索工具需要降低采样或过滤,冒着失去重要生物学模式的风险。额外的障碍来自需要命令行专业知识的工具,或者下游生物解释的碎片化工作流程。我们介绍了ClusterChirp,这是一个基于web的平台,通过gpu加速渲染和使用多个CPU内核的并行分层聚类来实现大规模数据矩阵的实时交互式探索。建在甲板上。gl和多线程聚类算法,ClusterChirp支持动态聚类、多度量排序、特征搜索和可调的可视化参数,用于交互式探索。独特的是,由大型语言模型提供支持的自然语言界面可以帮助用户执行复杂的操作,并从会话命令构建可重复的工作流。此外,用户可以选择集群来探索2D或3D的交互集群内关联网络,或通过生物知识库进行功能丰富。根据迭代用户反馈和坚持FAIR4S原则开发,ClusterChirp使研究人员能够以前所未有的轻松和速度从高维组学数据中提取见解。这个网站可以在clusterchirp.mss.edu免费获得,不需要登录。
{"title":"ClusterChirp: A GPU-accelerated Web Server for Natural Language-Guided Interactive Visualization and Analysis of Large Omics Data.","authors":"Osho Rawal, Rex Lu, Edgar Gonzalez-Kozlova, Sacha Gnjatic, Zeynep H Gümüş","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Tabular datasets are commonly visualized as heatmaps, where data values are represented as color intensities in a matrix to reveal patterns and correlations. However, modern omics technologies increasingly generate matrices so large that existing visual exploration tools require downsampling or filtering, risking loss of biologically important patterns. Additional barriers arise from tools that require command-line expertise, or fragmented workflows for downstream biological interpretation. We present ClusterChirp, a web-based platform for real-time, interactive exploration of large-scale data matrices enabled by GPU-accelerated rendering and parallelized hierarchical clustering using multiple CPU cores. Built on deck.gl and multi-threaded clustering algorithms, ClusterChirp supports on-the-fly clustering, multi-metric sorting, feature search, and adjustable visualization parameters for interactive explorations. Uniquely, a natural language interface powered by a Large Language Model helps users perform complex operations and build reproducible workflows from conversational commands. Furthermore, users can select clusters to explore interactive within-cluster correlation networks in 2D or 3D, or perform functional enrichment through biological knowledge bases. Developed with iterative user feedback and adhering to FAIR4S principles, ClusterChirp empowers researchers to extract insights from high-dimensional omics data with unprecedented ease and speed. This website is freely available at clusterchirp.mssm.edu, with no login required.</p>","PeriodicalId":93888,"journal":{"name":"ArXiv","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2026-02-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12919222/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147273267","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Cell strain-stiffening drives cell breakout from embedded spheroids. 细胞株硬化驱动细胞从嵌入的球体突围。
Pub Date : 2026-02-09
Shabeeb Ameen, Kyungeun Kim, Ligesh Theeyancheri, Minh Thanh, Mingming Wu, Alison E Patteson, J M Schwarz, Tao Zhang

Understanding how cells escape from embedded spheroids requires a mechanical framework linking stress generation within cells, across cells, and between cells and the surrounding extracellular matrix (ECM). We develop such a framework by coupling a 3D vertex model of a spheroid to a fibrous ECM network and deriving a 3D Cauchy stress tensor for deformable polyhedral cells, enabling direct cell-level stress quantification in three dimensions. We analyze maximum shear stress in solid-like and fluid-like spheroids: solid-like spheroids exhibit broader stress distributions and radial stress gradients, while fluid-like spheroids show lower stresses with weak spatial organization. Cell shape anisotropy is not generically aligned with principal stress directions, indicating that morphology alone is an unreliable proxy for mechanical state. We further demonstrate strain stiffening at the single-cell level, where elongation produces nonlinear increases in maximum shear stress, allowing boundary cells in otherwise low-stress, fluid-like spheroids to transiently generate forces sufficient to remodel the matrix. To connect strain-induced stress amplification to invasion modes, we introduce an extended 3D vertex model with explicit, tunable cell-cell adhesion springs. In this minimal mechanical framework, single-cell breakout results from strain stiffening combined with reduced adhesion, whereas multi-cell streaming additionally requires anisotropic adhesion strengthened along the elongation axis and weakened orthogonally. Together, these results identify distinct mechanical pathways coupling cell strain, stress amplification, and adhesion organization to spheroid invasion.

了解细胞如何从嵌入的球体中逃逸,需要一个连接细胞内、细胞间、细胞与周围细胞外基质(ECM)之间应力产生的机械框架。我们通过将球体的三维顶点模型耦合到纤维ECM网络,并推导出可变形多面体细胞的三维柯西应力张量,开发了这样一个框架,从而可以在三维上直接进行细胞水平的应力量化。我们分析了类固体和类流体椭球体的最大剪应力:类固体椭球体具有更宽的应力分布和径向应力梯度,而类流体椭球体具有较低的应力和较弱的空间组织。细胞形状的各向异性通常与主应力方向不一致,表明单独的形态学是机械状态的不可靠代理。我们进一步证明了单细胞水平上的应变硬化,其中伸长产生最大剪切应力的非线性增加,允许低应力的边界细胞,类流体球体瞬间产生足够的力来重塑基质。为了将应变引起的应力放大与入侵模式联系起来,我们引入了一个扩展的三维顶点模型,该模型具有明确的、可调的细胞-细胞粘附弹簧。在这个最小的机械框架中,单细胞断裂是由应变硬化和粘附减少引起的,而多细胞流动还需要各向异性粘附,沿延伸轴增强,正交减弱。总之,这些结果确定了不同的机械途径耦合细胞株,应力放大和粘附组织到球体入侵。
{"title":"Cell strain-stiffening drives cell breakout from embedded spheroids.","authors":"Shabeeb Ameen, Kyungeun Kim, Ligesh Theeyancheri, Minh Thanh, Mingming Wu, Alison E Patteson, J M Schwarz, Tao Zhang","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Understanding how cells escape from embedded spheroids requires a mechanical framework linking stress generation within cells, across cells, and between cells and the surrounding extracellular matrix (ECM). We develop such a framework by coupling a 3D vertex model of a spheroid to a fibrous ECM network and deriving a 3D Cauchy stress tensor for deformable polyhedral cells, enabling direct cell-level stress quantification in three dimensions. We analyze maximum shear stress in solid-like and fluid-like spheroids: solid-like spheroids exhibit broader stress distributions and radial stress gradients, while fluid-like spheroids show lower stresses with weak spatial organization. Cell shape anisotropy is not generically aligned with principal stress directions, indicating that morphology alone is an unreliable proxy for mechanical state. We further demonstrate strain stiffening at the single-cell level, where elongation produces nonlinear increases in maximum shear stress, allowing boundary cells in otherwise low-stress, fluid-like spheroids to transiently generate forces sufficient to remodel the matrix. To connect strain-induced stress amplification to invasion modes, we introduce an extended 3D vertex model with explicit, tunable cell-cell adhesion springs. In this minimal mechanical framework, single-cell breakout results from strain stiffening combined with reduced adhesion, whereas multi-cell streaming additionally requires anisotropic adhesion strengthened along the elongation axis and weakened orthogonally. Together, these results identify distinct mechanical pathways coupling cell strain, stress amplification, and adhesion organization to spheroid invasion.</p>","PeriodicalId":93888,"journal":{"name":"ArXiv","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2026-02-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12919209/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147273350","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Maximum-Likelihood--Based Position Decoding of Laser Processed Converging Pixel CsI: Tl Detectors for High-Resolution SPECT. 基于最大似然的高分辨率SPECT激光处理收敛像素CsI: Tl探测器的位置解码。
Pub Date : 2026-02-09
Xi Zhang, Arkadiusz Sitek, Lisa Blackberg, Matthew Kupinski, Lars Furenlid, Hamid Sabet

This study demonstrates the feasibility of a novel fabrication technique for high spatial resolution CsI: Tl scintillation detectors tailored for single photon emission computed tomography (SPECT) systems. Building upon our previously developed laser induced optical barrier (LIOB) method, which achieved high spatial resolution, excellent sensitivity, and 100% fabrication yield in CsI: Tl detectors, we extend this approach to a converging-pixel architecture. A CsI: Tl crystal array with converging pixels was designed and fabricated, featuring entrance-face pixels of 1.6x1.6 mm2 and photodetector side pixels of 2x2 mm2. To localize gamma-ray interactions, both the center of gravity (CoG) algorithm and a maximum-likelihood (ML) based decoding method were implemented. A custom built four axis motion platform was developed to deliver a finely collimated pencil beam at precisely controlled positions and angles across the array, enabling generation of a comprehensive dataset for prior knowledge and validation. The results demonstrate an energy resolution of 11.79+/-0.53% (collimated experiment) and a position localization accuracy of 1.00+/-0.42 mm (nearest neighbor interpolation), confirming that the proposed converging-pixel architecture, combined with statistical decoding algorithms, provides a promising path toward the development of high-performance SPECT detectors.

本研究证明了为单光子发射计算机断层扫描(SPECT)系统量身定制的高空间分辨率CsI: Tl闪烁探测器的新制造技术的可行性。基于我们之前开发的激光诱导光屏障(LIOB)方法,该方法在CsI: Tl探测器中实现了高空间分辨率,出色的灵敏度和100%的制造成良率,我们将这种方法扩展到收敛像素架构。设计并制作了具有会聚像元的CsI: Tl晶体阵列,其入口面像元为1.6x1.6 mm2,光电探测器侧像元为2x2 mm2。为了定位伽玛射线相互作用,实现了重心算法和基于最大似然的解码方法。开发了一个定制的四轴运动平台,以精确控制的位置和角度在阵列上提供精细准直的铅笔光束,从而生成一个全面的数据集,用于先验知识和验证。结果表明,能量分辨率为11.79+/-0.53%(准直实验),位置定位精度为1.00+/-0.42 mm(最近邻插值),证实了所提出的收敛像素架构与统计解码算法相结合,为高性能SPECT检测器的发展提供了一条有前途的道路。
{"title":"Maximum-Likelihood--Based Position Decoding of Laser Processed Converging Pixel CsI: Tl Detectors for High-Resolution SPECT.","authors":"Xi Zhang, Arkadiusz Sitek, Lisa Blackberg, Matthew Kupinski, Lars Furenlid, Hamid Sabet","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>This study demonstrates the feasibility of a novel fabrication technique for high spatial resolution CsI: Tl scintillation detectors tailored for single photon emission computed tomography (SPECT) systems. Building upon our previously developed laser induced optical barrier (LIOB) method, which achieved high spatial resolution, excellent sensitivity, and 100% fabrication yield in CsI: Tl detectors, we extend this approach to a converging-pixel architecture. A CsI: Tl crystal array with converging pixels was designed and fabricated, featuring entrance-face pixels of 1.6x1.6 mm2 and photodetector side pixels of 2x2 mm2. To localize gamma-ray interactions, both the center of gravity (CoG) algorithm and a maximum-likelihood (ML) based decoding method were implemented. A custom built four axis motion platform was developed to deliver a finely collimated pencil beam at precisely controlled positions and angles across the array, enabling generation of a comprehensive dataset for prior knowledge and validation. The results demonstrate an energy resolution of 11.79+/-0.53% (collimated experiment) and a position localization accuracy of 1.00+/-0.42 mm (nearest neighbor interpolation), confirming that the proposed converging-pixel architecture, combined with statistical decoding algorithms, provides a promising path toward the development of high-performance SPECT detectors.</p>","PeriodicalId":93888,"journal":{"name":"ArXiv","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2026-02-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12919228/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147273359","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A tissue-informed deep learning-based method for positron range correction in preclinical 68Ga PET imaging. 基于组织信息的深度学习方法用于临床前68Ga PET成像正电子距离校正。
Pub Date : 2026-02-06
Nerea Encina-Baranda, Robert J Paneque-Yunta, Javier Lopez-Rodriguez, Edwin C Pratt, Trong Nghia Nguyen, Jan Grimm, Alejandro Lopez-Montes, Joaquin L Herraiz

Positron range (PR) limits spatial resolution and quantitative accuracy in PET imaging, particularly for high-energy positron-emitting radionuclides like 68Ga. We propose a deep learning method using 3D residual encoder-decoder convolutional neural networks (3D RED-CNNs), incorporating tissue-dependent anatomical information through a u-map-dependent loss function. Models were trained with realistic simulations and, using initial PET and CT data, generated positron range corrected images. We validated the models in simulations and real acquisitions. Three 3D RED-CNN architectures, Single-channel, Two-channel, and DualEncoder, were trained on simulated PET datasets and evaluated on synthetic and real PET acquisitions from 68Ga-FH and 68Ga-PSMA-617 mouse studies. Performance was compared to a standard Richardson-Lucy-based positron range correction (RL-PRC) method using metrics such as mean absolute error (MAE), structural similarity index (SSIM), contrast recovery (CR), and contrast-to-noise ratio (CNR). CNN-based methods achieved up to 19 percent SSIM improvement and 13 percent MAE reduction compared to RL-PRC. The Two-Channel model achieved the highest CR and CNR, recovering lung activity with 97 percent agreement to ground truth versus 77 percent for RL-PRC. Noise levels remained stable for CNN models (approximately 5.9 percent), while RL-PRC increased noise by 5.8 percent. In preclinical acquisitions, the Two-Channel model achieved the highest CNR across tissues while maintaining the lowest noise level (9.6 percent). Although no ground truth was available for real data, tumor delineation and spillover artifacts improved with the Two-Channel model. These findings highlight the potential of CNN-based PRC to enhance quantitative PET imaging, particularly for 68Ga. Future work will improve model generalization through domain adaptation and hybrid training strategies.

正电子距离(PR)限制了PET成像的空间分辨率和定量精度,特别是对于像68Ga这样的高能正电子发射放射性核素。我们提出了一种使用3D残差编码器-解码器卷积神经网络(3D red - cnn)的深度学习方法,通过u-map依赖的损失函数结合组织相关的解剖信息。模型经过真实模拟训练,并使用初始PET和CT数据生成正电子距离校正图像。我们在模拟和实际采集中验证了这些模型。我们在模拟PET数据集上训练了三种3D RED-CNN架构(单通道、双通道和双通道),并对68Ga-FH和68Ga-PSMA-617小鼠的合成和真实PET采集结果进行了评估。使用平均绝对误差(MAE)、结构相似指数(SSIM)、对比度恢复(CR)和对比噪声比(CNR)等指标,将性能与标准richardson - lucon -based正电子范围校正(RL-PRC)方法进行比较。与RL-PRC相比,基于cnn的方法实现了高达19%的SSIM改进和13%的MAE降低。双通道模型获得了最高的CR和CNR,恢复肺活动的一致性为97%,而RL-PRC为77%。CNN模型的噪声水平保持稳定(约5.9%),而RL-PRC模型的噪声水平增加了5.8%。在临床前采集中,双通道模型在保持最低噪声水平(9.6%)的同时,实现了跨组织的最高CNR。尽管没有真实数据的基础真相,但双通道模型改善了肿瘤描绘和溢出伪影。这些发现强调了基于cnn的PRC增强定量PET成像的潜力,特别是对68Ga。未来的工作将通过领域自适应和混合训练策略来改进模型泛化。
{"title":"A tissue-informed deep learning-based method for positron range correction in preclinical 68Ga PET imaging.","authors":"Nerea Encina-Baranda, Robert J Paneque-Yunta, Javier Lopez-Rodriguez, Edwin C Pratt, Trong Nghia Nguyen, Jan Grimm, Alejandro Lopez-Montes, Joaquin L Herraiz","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Positron range (PR) limits spatial resolution and quantitative accuracy in PET imaging, particularly for high-energy positron-emitting radionuclides like 68Ga. We propose a deep learning method using 3D residual encoder-decoder convolutional neural networks (3D RED-CNNs), incorporating tissue-dependent anatomical information through a u-map-dependent loss function. Models were trained with realistic simulations and, using initial PET and CT data, generated positron range corrected images. We validated the models in simulations and real acquisitions. Three 3D RED-CNN architectures, Single-channel, Two-channel, and DualEncoder, were trained on simulated PET datasets and evaluated on synthetic and real PET acquisitions from 68Ga-FH and 68Ga-PSMA-617 mouse studies. Performance was compared to a standard Richardson-Lucy-based positron range correction (RL-PRC) method using metrics such as mean absolute error (MAE), structural similarity index (SSIM), contrast recovery (CR), and contrast-to-noise ratio (CNR). CNN-based methods achieved up to 19 percent SSIM improvement and 13 percent MAE reduction compared to RL-PRC. The Two-Channel model achieved the highest CR and CNR, recovering lung activity with 97 percent agreement to ground truth versus 77 percent for RL-PRC. Noise levels remained stable for CNN models (approximately 5.9 percent), while RL-PRC increased noise by 5.8 percent. In preclinical acquisitions, the Two-Channel model achieved the highest CNR across tissues while maintaining the lowest noise level (9.6 percent). Although no ground truth was available for real data, tumor delineation and spillover artifacts improved with the Two-Channel model. These findings highlight the potential of CNN-based PRC to enhance quantitative PET imaging, particularly for 68Ga. Future work will improve model generalization through domain adaptation and hybrid training strategies.</p>","PeriodicalId":93888,"journal":{"name":"ArXiv","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2026-02-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12889857/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146168343","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Transformer brain encoders explain human high-level visual responses. 变压器大脑编码器解释了人类高级视觉反应。
Pub Date : 2026-02-05
Hossein Adeli, Sun Minni, Nikolaus Kriegeskorte

A major goal of neuroscience is to understand brain computations during visual processing in naturalistic settings. A dominant approach is to use image-computable deep neural networks trained with different task objectives as a basis for linear encoding models. However, in addition to requiring estimation of a large number of linear encoding parameters, this approach ignores the structure of the feature maps both in the brain and the models. Recently proposed alternatives factor the linear mapping into separate sets of spatial and feature weights, thus finding static receptive fields for units, which is appropriate only for early visual areas. In this work, we employ the attention mechanism used in the transformer architecture to study how retinotopic visual features can be dynamically routed to category-selective areas in high-level visual processing. We show that this computational motif is significantly more powerful than alternative methods in predicting brain activity during natural scene viewing, across different feature basis models and modalities. We also show that this approach is inherently more interpretable as the attention-routing signals for different high-level categorical areas can be easily visualized for any input image. Given its high performance at predicting brain responses to novel images, the model deserves consideration as a candidate mechanistic model of how visual information from retinotopic maps is routed in the human brain based on the relevance of the input content to different category-selective regions. Our code is available at https://github.com/Hosseinadeli/transformer_brain_encoder/.

神经科学的一个主要目标是了解在自然环境下视觉处理过程中的大脑计算。一种主流的方法是使用经过不同任务目标训练的图像可计算深度神经网络作为线性编码模型的基础。然而,除了需要估计大量的线性编码参数外,这种方法还忽略了大脑和模型中特征映射的结构。最近提出的替代方案将线性映射纳入单独的空间和特征权重集,从而为单元找到静态接受域,这只适用于早期视觉区域。在这项工作中,我们采用变压器架构中使用的注意机制来研究视网膜定位视觉特征如何在高级视觉处理中动态路由到类别选择区域。我们发现,在不同的特征基础模型和模式下,这种计算基序在预测自然场景观看期间的大脑活动方面比其他方法要强大得多。我们还表明,这种方法本质上更具可解释性,因为对于任何输入图像,不同高级分类区域的注意路由信号都可以很容易地可视化。鉴于其在预测大脑对新图像的反应方面的高性能,该模型值得考虑作为一个候选的机制模型,如何根据输入内容与不同类别选择区域的相关性,将来自视网膜定位图的视觉信息在人脑中路由。
{"title":"Transformer brain encoders explain human high-level visual responses.","authors":"Hossein Adeli, Sun Minni, Nikolaus Kriegeskorte","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>A major goal of neuroscience is to understand brain computations during visual processing in naturalistic settings. A dominant approach is to use image-computable deep neural networks trained with different task objectives as a basis for linear encoding models. However, in addition to requiring estimation of a large number of linear encoding parameters, this approach ignores the structure of the feature maps both in the brain and the models. Recently proposed alternatives factor the linear mapping into separate sets of spatial and feature weights, thus finding static receptive fields for units, which is appropriate only for early visual areas. In this work, we employ the attention mechanism used in the transformer architecture to study how retinotopic visual features can be dynamically routed to category-selective areas in high-level visual processing. We show that this computational motif is significantly more powerful than alternative methods in predicting brain activity during natural scene viewing, across different feature basis models and modalities. We also show that this approach is inherently more interpretable as the attention-routing signals for different high-level categorical areas can be easily visualized for any input image. Given its high performance at predicting brain responses to novel images, the model deserves consideration as a candidate mechanistic model of how visual information from retinotopic maps is routed in the human brain based on the relevance of the input content to different category-selective regions. Our code is available at https://github.com/Hosseinadeli/transformer_brain_encoder/.</p>","PeriodicalId":93888,"journal":{"name":"ArXiv","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2026-02-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12889847/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146168360","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
ArXiv
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1