Recent studies demonstrate that diffusion models can serve as a strong prior for solving inverse problems. A prominent example is Diffusion Posterior Sampling (DPS), which approximates the posterior distribution of data given the measure using Tweedie's formula. Despite the merits of being versatile in solving various inverse problems without re-training, the performance of DPS is hindered by the fact that this posterior approximation can be inaccurate especially for high noise levels. Therefore, we propose textbf{D}iffusion textbf{P}osterior textbf{MC}MC (textbf{DPMC}), a novel inference algorithm based on Annealed MCMC to solve inverse problems with pretrained diffusion models. We define a series of intermediate distributions inspired by the approximated conditional distributions used by DPS. Through annealed MCMC sampling, we encourage the samples to follow each intermediate distribution more closely before moving to the next distribution at a lower noise level, and therefore reduce the accumulated error along the path. We test our algorithm in various inverse problems, including super resolution, Gaussian deblurring, motion deblurring, inpainting, and phase retrieval. Our algorithm outperforms DPS with less number of evaluations across nearly all tasks, and is competitive among existing approaches.
{"title":"Think Twice Before You Act: Improving Inverse Problem Solving With MCMC","authors":"Yaxuan Zhu, Zehao Dou, Haoxin Zheng, Yasi Zhang, Ying Nian Wu, Ruiqi Gao","doi":"arxiv-2409.08551","DOIUrl":"https://doi.org/arxiv-2409.08551","url":null,"abstract":"Recent studies demonstrate that diffusion models can serve as a strong prior\u0000for solving inverse problems. A prominent example is Diffusion Posterior\u0000Sampling (DPS), which approximates the posterior distribution of data given the\u0000measure using Tweedie's formula. Despite the merits of being versatile in\u0000solving various inverse problems without re-training, the performance of DPS is\u0000hindered by the fact that this posterior approximation can be inaccurate\u0000especially for high noise levels. Therefore, we propose textbf{D}iffusion\u0000textbf{P}osterior textbf{MC}MC (textbf{DPMC}), a novel inference algorithm\u0000based on Annealed MCMC to solve inverse problems with pretrained diffusion\u0000models. We define a series of intermediate distributions inspired by the\u0000approximated conditional distributions used by DPS. Through annealed MCMC\u0000sampling, we encourage the samples to follow each intermediate distribution\u0000more closely before moving to the next distribution at a lower noise level, and\u0000therefore reduce the accumulated error along the path. We test our algorithm in\u0000various inverse problems, including super resolution, Gaussian deblurring,\u0000motion deblurring, inpainting, and phase retrieval. Our algorithm outperforms\u0000DPS with less number of evaluations across nearly all tasks, and is competitive\u0000among existing approaches.","PeriodicalId":501340,"journal":{"name":"arXiv - STAT - Machine Learning","volume":"14 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142261755","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evidential deep learning (EDL) has shown remarkable success in uncertainty estimation. However, there is still room for improvement, particularly in out-of-distribution (OOD) detection and classification tasks. The limited OOD detection performance of EDL arises from its inability to reflect the distance between the testing example and training data when quantifying uncertainty, while its limited classification performance stems from its parameterization of the concentration parameters. To address these limitations, we propose a novel method called Density Aware Evidential Deep Learning (DAEDL). DAEDL integrates the feature space density of the testing example with the output of EDL during the prediction stage, while using a novel parameterization that resolves the issues in the conventional parameterization. We prove that DAEDL enjoys a number of favorable theoretical properties. DAEDL demonstrates state-of-the-art performance across diverse downstream tasks related to uncertainty estimation and classification
{"title":"Uncertainty Estimation by Density Aware Evidential Deep Learning","authors":"Taeseong Yoon, Heeyoung Kim","doi":"arxiv-2409.08754","DOIUrl":"https://doi.org/arxiv-2409.08754","url":null,"abstract":"Evidential deep learning (EDL) has shown remarkable success in uncertainty\u0000estimation. However, there is still room for improvement, particularly in\u0000out-of-distribution (OOD) detection and classification tasks. The limited OOD\u0000detection performance of EDL arises from its inability to reflect the distance\u0000between the testing example and training data when quantifying uncertainty,\u0000while its limited classification performance stems from its parameterization of\u0000the concentration parameters. To address these limitations, we propose a novel\u0000method called Density Aware Evidential Deep Learning (DAEDL). DAEDL integrates\u0000the feature space density of the testing example with the output of EDL during\u0000the prediction stage, while using a novel parameterization that resolves the\u0000issues in the conventional parameterization. We prove that DAEDL enjoys a\u0000number of favorable theoretical properties. DAEDL demonstrates state-of-the-art\u0000performance across diverse downstream tasks related to uncertainty estimation\u0000and classification","PeriodicalId":501340,"journal":{"name":"arXiv - STAT - Machine Learning","volume":"30 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142261811","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Robert Sisneros, Tushar M. Athawale, David Pugmire, Kenneth Moreland
We present a simple comparative framework for testing and developing uncertainty modeling in uncertain marching cubes implementations. The selection of a model to represent the probability distribution of uncertain values directly influences the memory use, run time, and accuracy of an uncertainty visualization algorithm. We use an entropy calculation directly on ensemble data to establish an expected result and then compare the entropy from various probability models, including uniform, Gaussian, histogram, and quantile models. Our results verify that models matching the distribution of the ensemble indeed match the entropy. We further show that fewer bins in nonparametric histogram models are more effective whereas large numbers of bins in quantile models approach data accuracy.
{"title":"An Entropy-Based Test and Development Framework for Uncertainty Modeling in Level-Set Visualizations","authors":"Robert Sisneros, Tushar M. Athawale, David Pugmire, Kenneth Moreland","doi":"arxiv-2409.08445","DOIUrl":"https://doi.org/arxiv-2409.08445","url":null,"abstract":"We present a simple comparative framework for testing and developing\u0000uncertainty modeling in uncertain marching cubes implementations. The selection\u0000of a model to represent the probability distribution of uncertain values\u0000directly influences the memory use, run time, and accuracy of an uncertainty\u0000visualization algorithm. We use an entropy calculation directly on ensemble\u0000data to establish an expected result and then compare the entropy from various\u0000probability models, including uniform, Gaussian, histogram, and quantile\u0000models. Our results verify that models matching the distribution of the\u0000ensemble indeed match the entropy. We further show that fewer bins in\u0000nonparametric histogram models are more effective whereas large numbers of bins\u0000in quantile models approach data accuracy.","PeriodicalId":501340,"journal":{"name":"arXiv - STAT - Machine Learning","volume":"89 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142261816","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Federico Maria Quetti, Silvia Figini, Elena ballante
The paper presents a novel approach for unsupervised techniques in the field of clustering. A new method is proposed to enhance existing literature models using the proper Bayesian bootstrap to improve results in terms of robustness and interpretability. Our approach is organized in two steps: k-means clustering is used for prior elicitation, then proper Bayesian bootstrap is applied as resampling method in an ensemble clustering approach. Results are analyzed introducing measures of uncertainty based on Shannon entropy. The proposal provides clear indication on the optimal number of clusters, as well as a better representation of the clustered data. Empirical results are provided on simulated data showing the methodological and empirical advances obtained.
{"title":"A Bayesian Approach to Clustering via the Proper Bayesian Bootstrap: the Bayesian Bagged Clustering (BBC) algorithm","authors":"Federico Maria Quetti, Silvia Figini, Elena ballante","doi":"arxiv-2409.08954","DOIUrl":"https://doi.org/arxiv-2409.08954","url":null,"abstract":"The paper presents a novel approach for unsupervised techniques in the field\u0000of clustering. A new method is proposed to enhance existing literature models\u0000using the proper Bayesian bootstrap to improve results in terms of robustness\u0000and interpretability. Our approach is organized in two steps: k-means\u0000clustering is used for prior elicitation, then proper Bayesian bootstrap is\u0000applied as resampling method in an ensemble clustering approach. Results are\u0000analyzed introducing measures of uncertainty based on Shannon entropy. The\u0000proposal provides clear indication on the optimal number of clusters, as well\u0000as a better representation of the clustered data. Empirical results are\u0000provided on simulated data showing the methodological and empirical advances\u0000obtained.","PeriodicalId":501340,"journal":{"name":"arXiv - STAT - Machine Learning","volume":"75 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142261754","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Xiaojing Du, Feiyu Yang, Wentao Gao, Xiongren Chen
As network data applications continue to expand, causal inference within networks has garnered increasing attention. However, hidden confounders complicate the estimation of causal effects. Most methods rely on the strong ignorability assumption, which presumes the absence of hidden confounders-an assumption that is both difficult to validate and often unrealistic in practice. To address this issue, we propose CgNN, a novel approach that leverages network structure as instrumental variables (IVs), combined with graph neural networks (GNNs) and attention mechanisms, to mitigate hidden confounder bias and improve causal effect estimation. By utilizing network structure as IVs, we reduce confounder bias while preserving the correlation with treatment. Our integration of attention mechanisms enhances robustness and improves the identification of important nodes. Validated on two real-world datasets, our results demonstrate that CgNN effectively mitigates hidden confounder bias and offers a robust GNN-driven IV framework for causal inference in complex network data.
随着网络数据应用的不断扩大，网络内的因果推断越来越受到关注。然而，隐藏混杂因素使因果效应的估计变得复杂。大多数方法都依赖于强可识别性假设，该假设假定不存在隐藏混杂因素--这种假设既难以验证，在实践中也往往不现实。为了解决这个问题，我们提出了 CgNN，这是一种将网络结构作为工具变量（IVs）的新方法，结合图神经网络（GNNs）和注意力机制，以减轻隐藏混杂因素偏差并改进因果效应估计。通过利用网络结构作为 IVs，我们减少了混杂因素偏差，同时保留了与治疗的相关性。我们对注意力机制的整合增强了稳健性，并改善了重要节点的识别。通过对两个真实世界数据集的验证，我们的结果表明，CgNN 有效地减轻了隐藏的混杂因素偏差，为复杂网络数据的因果推断提供了一个稳健的 GNN 驱动 IV 框架。
{"title":"Causal GNNs: A GNN-Driven Instrumental Variable Approach for Causal Inference in Networks","authors":"Xiaojing Du, Feiyu Yang, Wentao Gao, Xiongren Chen","doi":"arxiv-2409.08544","DOIUrl":"https://doi.org/arxiv-2409.08544","url":null,"abstract":"As network data applications continue to expand, causal inference within\u0000networks has garnered increasing attention. However, hidden confounders\u0000complicate the estimation of causal effects. Most methods rely on the strong\u0000ignorability assumption, which presumes the absence of hidden confounders-an\u0000assumption that is both difficult to validate and often unrealistic in\u0000practice. To address this issue, we propose CgNN, a novel approach that\u0000leverages network structure as instrumental variables (IVs), combined with\u0000graph neural networks (GNNs) and attention mechanisms, to mitigate hidden\u0000confounder bias and improve causal effect estimation. By utilizing network\u0000structure as IVs, we reduce confounder bias while preserving the correlation\u0000with treatment. Our integration of attention mechanisms enhances robustness and\u0000improves the identification of important nodes. Validated on two real-world\u0000datasets, our results demonstrate that CgNN effectively mitigates hidden\u0000confounder bias and offers a robust GNN-driven IV framework for causal\u0000inference in complex network data.","PeriodicalId":501340,"journal":{"name":"arXiv - STAT - Machine Learning","volume":"47 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142261812","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hang Li, Wei Jin, Geri Skenderi, Harry Shomer, Wenzhuo Tang, Wenqi Fan, Jiliang Tang
Denoising Diffusion Probabilistic Models (DDPMs) represent a contemporary class of generative models with exceptional qualities in both synthesis and maximizing the data likelihood. These models work by traversing a forward Markov Chain where data is perturbed, followed by a reverse process where a neural network learns to undo the perturbations and recover the original data. There have been increasing efforts exploring the applications of DDPMs in the graph domain. However, most of them have focused on the generative perspective. In this paper, we aim to build a novel generative model for link prediction. In particular, we treat link prediction between a pair of nodes as a conditional likelihood estimation of its enclosing sub-graph. With a dedicated design to decompose the likelihood estimation process via the Bayesian formula, we are able to separate the estimation of sub-graph structure and its node features. Such designs allow our model to simultaneously enjoy the advantages of inductive learning and the strong generalization capability. Remarkably, comprehensive experiments across various datasets validate that our proposed method presents numerous advantages: (1) transferability across datasets without retraining, (2) promising generalization on limited training data, and (3) robustness against graph adversarial attacks.
{"title":"Sub-graph Based Diffusion Model for Link Prediction","authors":"Hang Li, Wei Jin, Geri Skenderi, Harry Shomer, Wenzhuo Tang, Wenqi Fan, Jiliang Tang","doi":"arxiv-2409.08487","DOIUrl":"https://doi.org/arxiv-2409.08487","url":null,"abstract":"Denoising Diffusion Probabilistic Models (DDPMs) represent a contemporary\u0000class of generative models with exceptional qualities in both synthesis and\u0000maximizing the data likelihood. These models work by traversing a forward\u0000Markov Chain where data is perturbed, followed by a reverse process where a\u0000neural network learns to undo the perturbations and recover the original data.\u0000There have been increasing efforts exploring the applications of DDPMs in the\u0000graph domain. However, most of them have focused on the generative perspective.\u0000In this paper, we aim to build a novel generative model for link prediction. In\u0000particular, we treat link prediction between a pair of nodes as a conditional\u0000likelihood estimation of its enclosing sub-graph. With a dedicated design to\u0000decompose the likelihood estimation process via the Bayesian formula, we are\u0000able to separate the estimation of sub-graph structure and its node features.\u0000Such designs allow our model to simultaneously enjoy the advantages of\u0000inductive learning and the strong generalization capability. Remarkably,\u0000comprehensive experiments across various datasets validate that our proposed\u0000method presents numerous advantages: (1) transferability across datasets\u0000without retraining, (2) promising generalization on limited training data, and\u0000(3) robustness against graph adversarial attacks.","PeriodicalId":501340,"journal":{"name":"arXiv - STAT - Machine Learning","volume":"1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142261817","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Asaf CasselSchool of Computer Science, Tel Aviv University, Orin LevySchool of Computer Science, Tel Aviv University, Yishay MansourSchool of Computer Science, Tel Aviv UniversityGoogle Research, Tel Aviv
Efficiently trading off exploration and exploitation is one of the key challenges in online Reinforcement Learning (RL). Most works achieve this by carefully estimating the model uncertainty and following the so-called optimistic model. Inspired by practical ensemble methods, in this work we propose a simple and novel batch ensemble scheme that provably achieves near-optimal regret for stochastic Multi-Armed Bandits (MAB). Crucially, our algorithm has just a single parameter, namely the number of batches, and its value does not depend on distributional properties such as the scale and variance of the losses. We complement our theoretical results by demonstrating the effectiveness of our algorithm on synthetic benchmarks.
{"title":"Batch Ensemble for Variance Dependent Regret in Stochastic Bandits","authors":"Asaf CasselSchool of Computer Science, Tel Aviv University, Orin LevySchool of Computer Science, Tel Aviv University, Yishay MansourSchool of Computer Science, Tel Aviv UniversityGoogle Research, Tel Aviv","doi":"arxiv-2409.08570","DOIUrl":"https://doi.org/arxiv-2409.08570","url":null,"abstract":"Efficiently trading off exploration and exploitation is one of the key\u0000challenges in online Reinforcement Learning (RL). Most works achieve this by\u0000carefully estimating the model uncertainty and following the so-called\u0000optimistic model. Inspired by practical ensemble methods, in this work we\u0000propose a simple and novel batch ensemble scheme that provably achieves\u0000near-optimal regret for stochastic Multi-Armed Bandits (MAB). Crucially, our\u0000algorithm has just a single parameter, namely the number of batches, and its\u0000value does not depend on distributional properties such as the scale and\u0000variance of the losses. We complement our theoretical results by demonstrating\u0000the effectiveness of our algorithm on synthetic benchmarks.","PeriodicalId":501340,"journal":{"name":"arXiv - STAT - Machine Learning","volume":"177 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142261815","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Carles Domingo-Enrich, Michal Drozdzal, Brian Karrer, Ricky T. Q. Chen
Dynamical generative models that produce samples through an iterative process, such as Flow Matching and denoising diffusion models, have seen widespread use, but there has not been many theoretically-sound methods for improving these models with reward fine-tuning. In this work, we cast reward fine-tuning as stochastic optimal control (SOC). Critically, we prove that a very specific memoryless noise schedule must be enforced during fine-tuning, in order to account for the dependency between the noise variable and the generated samples. We also propose a new algorithm named Adjoint Matching which outperforms existing SOC algorithms, by casting SOC problems as a regression problem. We find that our approach significantly improves over existing methods for reward fine-tuning, achieving better consistency, realism, and generalization to unseen human preference reward models, while retaining sample diversity.
{"title":"Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Control","authors":"Carles Domingo-Enrich, Michal Drozdzal, Brian Karrer, Ricky T. Q. Chen","doi":"arxiv-2409.08861","DOIUrl":"https://doi.org/arxiv-2409.08861","url":null,"abstract":"Dynamical generative models that produce samples through an iterative\u0000process, such as Flow Matching and denoising diffusion models, have seen\u0000widespread use, but there has not been many theoretically-sound methods for\u0000improving these models with reward fine-tuning. In this work, we cast reward\u0000fine-tuning as stochastic optimal control (SOC). Critically, we prove that a\u0000very specific memoryless noise schedule must be enforced during fine-tuning, in\u0000order to account for the dependency between the noise variable and the\u0000generated samples. We also propose a new algorithm named Adjoint Matching which\u0000outperforms existing SOC algorithms, by casting SOC problems as a regression\u0000problem. We find that our approach significantly improves over existing methods\u0000for reward fine-tuning, achieving better consistency, realism, and\u0000generalization to unseen human preference reward models, while retaining sample\u0000diversity.","PeriodicalId":501340,"journal":{"name":"arXiv - STAT - Machine Learning","volume":"53 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142261810","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Accurate imputation is essential for the reliability and success of downstream tasks. Recently, diffusion models have attracted great attention in this field. However, these models neglect the latent distribution in a lower-dimensional space derived from the observed data, which limits the generative capacity of the diffusion model. Additionally, dealing with the original missing data without labels becomes particularly problematic. To address these issues, we propose the Latent Space Score-Based Diffusion Model (LSSDM) for probabilistic multivariate time series imputation. Observed values are projected onto low-dimensional latent space and coarse values of the missing data are reconstructed without knowing their ground truth values by this unsupervised learning approach. Finally, the reconstructed values are fed into a conditional diffusion model to obtain the precise imputed values of the time series. In this way, LSSDM not only possesses the power to identify the latent distribution but also seamlessly integrates the diffusion model to obtain the high-fidelity imputed values and assess the uncertainty of the dataset. Experimental results demonstrate that LSSDM achieves superior imputation performance while also providing a better explanation and uncertainty analysis of the imputation mechanism. The website of the code is textit{https://github.com/gorgen2020/LSSDM_imputation}.
{"title":"Latent Space Score-based Diffusion Model for Probabilistic Multivariate Time Series Imputation","authors":"Guojun Liang, Najmeh Abiri, Atiye Sadat Hashemi, Jens Lundström, Stefan Byttner, Prayag Tiwari","doi":"arxiv-2409.08917","DOIUrl":"https://doi.org/arxiv-2409.08917","url":null,"abstract":"Accurate imputation is essential for the reliability and success of\u0000downstream tasks. Recently, diffusion models have attracted great attention in\u0000this field. However, these models neglect the latent distribution in a\u0000lower-dimensional space derived from the observed data, which limits the\u0000generative capacity of the diffusion model. Additionally, dealing with the\u0000original missing data without labels becomes particularly problematic. To\u0000address these issues, we propose the Latent Space Score-Based Diffusion Model\u0000(LSSDM) for probabilistic multivariate time series imputation. Observed values\u0000are projected onto low-dimensional latent space and coarse values of the\u0000missing data are reconstructed without knowing their ground truth values by\u0000this unsupervised learning approach. Finally, the reconstructed values are fed\u0000into a conditional diffusion model to obtain the precise imputed values of the\u0000time series. In this way, LSSDM not only possesses the power to identify the\u0000latent distribution but also seamlessly integrates the diffusion model to\u0000obtain the high-fidelity imputed values and assess the uncertainty of the\u0000dataset. Experimental results demonstrate that LSSDM achieves superior\u0000imputation performance while also providing a better explanation and\u0000uncertainty analysis of the imputation mechanism. The website of the code is\u0000textit{https://github.com/gorgen2020/LSSDM_imputation}.","PeriodicalId":501340,"journal":{"name":"arXiv - STAT - Machine Learning","volume":"27 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142261808","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Shivam Pandey, Chirag Modi, Benjamin D. Wandelt, Deaglan J. Bartlett, Adrian E. Bayer, Greg L. Bryan, Matthew Ho, Guilhem Lavaux, T. Lucas Makinen, Francisco Villaescusa-Navarro
To maximize the amount of information extracted from cosmological datasets, simulations that accurately represent these observations are necessary. However, traditional simulations that evolve particles under gravity by estimating particle-particle interactions (N-body simulations) are computationally expensive and prohibitive to scale to the large volumes and resolutions necessary for the upcoming datasets. Moreover, modeling the distribution of galaxies typically involves identifying virialized dark matter halos, which is also a time- and memory-consuming process for large N-body simulations, further exacerbating the computational cost. In this study, we introduce CHARM, a novel method for creating mock halo catalogs by matching the spatial, mass, and velocity statistics of halos directly from the large-scale distribution of the dark matter density field. We develop multi-stage neural spline flow-based networks to learn this mapping at redshift z=0.5 directly with computationally cheaper low-resolution particle mesh simulations instead of relying on the high-resolution N-body simulations. We show that the mock halo catalogs and painted galaxy catalogs have the same statistical properties as obtained from $N$-body simulations in both real space and redshift space. Finally, we use these mock catalogs for cosmological inference using redshift-space galaxy power spectrum, bispectrum, and wavelet-based statistics using simulation-based inference, performing the first inference with accelerated forward model simulations and finding unbiased cosmological constraints with well-calibrated posteriors. The code was developed as part of the Simons Collaboration on Learning the Universe and is publicly available at url{https://github.com/shivampcosmo/CHARM}.
然而，传统的模拟是通过估计粒子与粒子之间的相互作用来实现粒子在引力作用下的演化（N-体模拟），其计算成本非常昂贵，而且无法扩展到即将到来的数据集所需的大体积和高分辨率。此外，星系分布建模通常涉及识别病毒化暗物质光环，这对于大型 N-体模拟也是一个耗时耗内存的过程，进一步加剧了计算成本。在这项研究中，我们引入了 CHARM，这是一种通过直接从暗物质密度场的大尺度分布中匹配光环的空间、质量和速度统计来创建模拟光环目录的新方法。我们开发了基于神经线流的多级网络，在红移 z=0.5 时直接利用计算成本更低的低分辨率粒子网格模拟来学习这种映射，而不是依赖高分辨率的 N-体模拟。最后，我们利用这些模拟星表，使用红移空间星系功率谱、双谱和基于小波的统计量进行了基于模拟的宇宙学推断，首次使用加速前向模型模拟进行了推断，并找到了具有良好校准后验的无偏宇宙学约束。该代码是 "学习宇宙 "西蒙斯合作组织（Simons Collaboration on Learning the Universe）的一部分，可在以下网址公开获取：url{https://github.com/shivampcosmo/CHARM}。
{"title":"CHARM: Creating Halos with Auto-Regressive Multi-stage networks","authors":"Shivam Pandey, Chirag Modi, Benjamin D. Wandelt, Deaglan J. Bartlett, Adrian E. Bayer, Greg L. Bryan, Matthew Ho, Guilhem Lavaux, T. Lucas Makinen, Francisco Villaescusa-Navarro","doi":"arxiv-2409.09124","DOIUrl":"https://doi.org/arxiv-2409.09124","url":null,"abstract":"To maximize the amount of information extracted from cosmological datasets,\u0000simulations that accurately represent these observations are necessary.\u0000However, traditional simulations that evolve particles under gravity by\u0000estimating particle-particle interactions (N-body simulations) are\u0000computationally expensive and prohibitive to scale to the large volumes and\u0000resolutions necessary for the upcoming datasets. Moreover, modeling the\u0000distribution of galaxies typically involves identifying virialized dark matter\u0000halos, which is also a time- and memory-consuming process for large N-body\u0000simulations, further exacerbating the computational cost. In this study, we\u0000introduce CHARM, a novel method for creating mock halo catalogs by matching the\u0000spatial, mass, and velocity statistics of halos directly from the large-scale\u0000distribution of the dark matter density field. We develop multi-stage neural\u0000spline flow-based networks to learn this mapping at redshift z=0.5 directly\u0000with computationally cheaper low-resolution particle mesh simulations instead\u0000of relying on the high-resolution N-body simulations. We show that the mock\u0000halo catalogs and painted galaxy catalogs have the same statistical properties\u0000as obtained from $N$-body simulations in both real space and redshift space.\u0000Finally, we use these mock catalogs for cosmological inference using\u0000redshift-space galaxy power spectrum, bispectrum, and wavelet-based statistics\u0000using simulation-based inference, performing the first inference with\u0000accelerated forward model simulations and finding unbiased cosmological\u0000constraints with well-calibrated posteriors. The code was developed as part of\u0000the Simons Collaboration on Learning the Universe and is publicly available at\u0000url{https://github.com/shivampcosmo/CHARM}.","PeriodicalId":501340,"journal":{"name":"arXiv - STAT - Machine Learning","volume":"16 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142261750","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：