Understanding characteristics of temporal correlations in time series is crucial for developing accurate models in natural and social sciences. The burst-tree decomposition method was recently introduced to reveal higher-order temporal correlations in time series in a form of an event sequence, in particular, the hierarchical structure of bursty trains of events for the entire range of timescales [Jo et al., Sci.~Rep.~textbf{10}, 12202 (2020)]. Such structure has been found to be simply characterized by the burst-merging kernel governing which bursts are merged together as the timescale for detecting bursts increases. In this work, we study the effects of kernels on the higher-order temporal correlations in terms of burst size distributions, memory coefficients for bursts, and the autocorrelation function. We employ several kernels, including the constant, additive, and product kernels as well as those inspired by the empirical results. We find that kernels with preferential mixing lead to the heavy-tailed burst size distributions, while kernels with assortative mixing lead to positive correlations between burst sizes. The decaying exponent of the autocorrelation function depends not only on the kernel but also on the power-law exponent of the interevent time distribution. In addition, thanks to the analogy to the coagulation process, analytical solutions of burst size distributions for some kernels could be obtained.
{"title":"Burst-tree structure and higher-order temporal correlations","authors":"Tibebe Birhanu, Hang-Hyun Jo","doi":"arxiv-2409.01674","DOIUrl":"https://doi.org/arxiv-2409.01674","url":null,"abstract":"Understanding characteristics of temporal correlations in time series is\u0000crucial for developing accurate models in natural and social sciences. The\u0000burst-tree decomposition method was recently introduced to reveal higher-order\u0000temporal correlations in time series in a form of an event sequence, in\u0000particular, the hierarchical structure of bursty trains of events for the\u0000entire range of timescales [Jo et al., Sci.~Rep.~textbf{10}, 12202 (2020)].\u0000Such structure has been found to be simply characterized by the burst-merging\u0000kernel governing which bursts are merged together as the timescale for\u0000detecting bursts increases. In this work, we study the effects of kernels on\u0000the higher-order temporal correlations in terms of burst size distributions,\u0000memory coefficients for bursts, and the autocorrelation function. We employ\u0000several kernels, including the constant, additive, and product kernels as well\u0000as those inspired by the empirical results. We find that kernels with\u0000preferential mixing lead to the heavy-tailed burst size distributions, while\u0000kernels with assortative mixing lead to positive correlations between burst\u0000sizes. The decaying exponent of the autocorrelation function depends not only\u0000on the kernel but also on the power-law exponent of the interevent time\u0000distribution. In addition, thanks to the analogy to the coagulation process,\u0000analytical solutions of burst size distributions for some kernels could be\u0000obtained.","PeriodicalId":501065,"journal":{"name":"arXiv - PHYS - Data Analysis, Statistics and Probability","volume":"6 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142179400","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This thesis investigates the mechanically controlled break junctions, with a particular emphasis on elucidating the behaviour of molecular currents at room temperature. The core of this experimental investigation involves a detailed analysis of conductance, examining how it varies over time and with changes in the gap between electrodes. Additionally, this study thoroughly evaluates transmission properties, coupling effects, and current characteristics. A pivotal aspect of the research was the meticulous current measurement, followed by carefully selecting optimal data sets. This process set the stage for an in-depth analysis of resonant tunnelling phenomena observed through a single channel. Notably, these experiments were conducted under open atmospheric conditions at room temperature. A significant finding from this study is the recognition that our current model requires refinement. This adjustment is necessary to encapsulate a broader spectrum of molecular transport mechanisms more accurately. Furthermore, this work significantly advances our comprehension of quantum effects in single-molecule junctions, particularly concerning similar molecules to Corannulene extending to some organometallics. One of the essential disclosures is the identification of deviations in the transport model, primarily attributable to electron-electron interactions. This insight is crucial as it paves the way for developing a more comprehensive and precise model, enhancing our understanding of molecular-scale electronic transport.
{"title":"Categorising current-voltage curves in single-molecule junctions and their comparison to Single-Level Model","authors":"Giovanna Angelis Schmidt","doi":"arxiv-2409.09051","DOIUrl":"https://doi.org/arxiv-2409.09051","url":null,"abstract":"This thesis investigates the mechanically controlled break junctions, with a\u0000particular emphasis on elucidating the behaviour of molecular currents at room\u0000temperature. The core of this experimental investigation involves a detailed\u0000analysis of conductance, examining how it varies over time and with changes in\u0000the gap between electrodes. Additionally, this study thoroughly evaluates\u0000transmission properties, coupling effects, and current characteristics. A\u0000pivotal aspect of the research was the meticulous current measurement, followed\u0000by carefully selecting optimal data sets. This process set the stage for an\u0000in-depth analysis of resonant tunnelling phenomena observed through a single\u0000channel. Notably, these experiments were conducted under open atmospheric\u0000conditions at room temperature. A significant finding from this study is the\u0000recognition that our current model requires refinement. This adjustment is\u0000necessary to encapsulate a broader spectrum of molecular transport mechanisms\u0000more accurately. Furthermore, this work significantly advances our\u0000comprehension of quantum effects in single-molecule junctions, particularly\u0000concerning similar molecules to Corannulene extending to some organometallics.\u0000One of the essential disclosures is the identification of deviations in the\u0000transport model, primarily attributable to electron-electron interactions. This\u0000insight is crucial as it paves the way for developing a more comprehensive and\u0000precise model, enhancing our understanding of molecular-scale electronic\u0000transport.","PeriodicalId":501065,"journal":{"name":"arXiv - PHYS - Data Analysis, Statistics and Probability","volume":"65 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142253356","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The PHENIX Collaboration has actively pursued a Data and Analysis Preservation program since 2019, the first such dedicated effort at RHIC. A particularly challenging aspect of this endeavor is preservation of complex physics analyses, selected for their scientific importance and the value of the specific techniques developed as a part of the research. For this, we have chosen one of the most impactful PHENIX results, the joint study of direct photons and neutral pions in high-energy d+Au collisions. To ensure reproducibility of this analysis going forward, we partitioned it into self-contained tasks and used a combination of containerization techniques, code management, and robust documentation. We then leveraged REANA (the platform for reproducible analysis developed at CERN) to run the required software. We present our experience based on this example, and outline our future plans for analysis preservation.
{"title":"Preservation of the Direct Photon and Neutral Meson Analysis in the PHENIX Experiment at RHIC","authors":"Gabor David, Maxim Potekhin, Dmitri Smirnov","doi":"arxiv-2408.12072","DOIUrl":"https://doi.org/arxiv-2408.12072","url":null,"abstract":"The PHENIX Collaboration has actively pursued a Data and Analysis\u0000Preservation program since 2019, the first such dedicated effort at RHIC. A\u0000particularly challenging aspect of this endeavor is preservation of complex\u0000physics analyses, selected for their scientific importance and the value of the\u0000specific techniques developed as a part of the research. For this, we have\u0000chosen one of the most impactful PHENIX results, the joint study of direct\u0000photons and neutral pions in high-energy d+Au collisions. To ensure\u0000reproducibility of this analysis going forward, we partitioned it into\u0000self-contained tasks and used a combination of containerization techniques,\u0000code management, and robust documentation. We then leveraged REANA (the\u0000platform for reproducible analysis developed at CERN) to run the required\u0000software. We present our experience based on this example, and outline our\u0000future plans for analysis preservation.","PeriodicalId":501065,"journal":{"name":"arXiv - PHYS - Data Analysis, Statistics and Probability","volume":"82 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142179265","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kunal Ghosh, Milica Todorović, Aki Vehtari, Patrick Rinke
Active learning (AL) has shown promise for being a particularly data-efficient machine learning approach. Yet, its performance depends on the application and it is not clear when AL practitioners can expect computational savings. Here, we carry out a systematic AL performance assessment for three diverse molecular datasets and two common scientific tasks: compiling compact, informative datasets and targeted molecular searches. We implemented AL with Gaussian processes (GP) and used the many-body tensor as molecular representation. For the first task, we tested different data acquisition strategies, batch sizes and GP noise settings. AL was insensitive to the acquisition batch size and we observed the best AL performance for the acquisition strategy that combines uncertainty reduction with clustering to promote diversity. However, for optimal GP noise settings, AL did not outperform randomized selection of data points. Conversely, for targeted searches, AL outperformed random sampling and achieved data savings up to 64%. Our analysis provides insight into this task-specific performance difference in terms of target distributions and data collection strategies. We established that the performance of AL depends on the relative distribution of the target molecules in comparison to the total dataset distribution, with the largest computational savings achieved when their overlap is minimal.
主动学习（AL）有望成为一种数据效率特别高的机器学习方法。然而，主动学习的性能取决于应用，目前还不清楚主动学习实践者何时可以期望节省计算量。在这里，我们针对三种不同的分子数据集和两种常见的科学任务进行了系统的 AL 性能评估：编译紧凑、信息丰富的数据集和有针对性的分子搜索。我们用高斯过程（GP）实现了 AL，并使用多体张量作为分子描述。对于第一个任务，我们测试了不同的数据采集策略、批量大小和 GP 噪声设置。AL对采集批量大小不敏感，我们观察到，将减少不确定性与促进多样性的聚类相结合的采集策略具有最佳的AL性能。然而，对于最佳的 GP 噪声设置，AL 的表现并不优于随机选择数据点。相反，对于有针对性的搜索，AL 的性能优于随机抽样，并节省了高达 64% 的数据。我们的分析深入揭示了目标分布和数据采集策略在特定任务中的性能差异。我们的分析深入揭示了目标分布和数据收集策略方面的这种特定任务性能差异。我们发现，AL 的性能取决于目标分子相对于整个数据集分布的相对分布，当两者的重叠最小时，计算量节省最大。
{"title":"Active Learning of Molecular Data for Task-Specific Objectives","authors":"Kunal Ghosh, Milica Todorović, Aki Vehtari, Patrick Rinke","doi":"arxiv-2408.11191","DOIUrl":"https://doi.org/arxiv-2408.11191","url":null,"abstract":"Active learning (AL) has shown promise for being a particularly\u0000data-efficient machine learning approach. Yet, its performance depends on the\u0000application and it is not clear when AL practitioners can expect computational\u0000savings. Here, we carry out a systematic AL performance assessment for three\u0000diverse molecular datasets and two common scientific tasks: compiling compact,\u0000informative datasets and targeted molecular searches. We implemented AL with\u0000Gaussian processes (GP) and used the many-body tensor as molecular\u0000representation. For the first task, we tested different data acquisition\u0000strategies, batch sizes and GP noise settings. AL was insensitive to the\u0000acquisition batch size and we observed the best AL performance for the\u0000acquisition strategy that combines uncertainty reduction with clustering to\u0000promote diversity. However, for optimal GP noise settings, AL did not\u0000outperform randomized selection of data points. Conversely, for targeted\u0000searches, AL outperformed random sampling and achieved data savings up to 64%.\u0000Our analysis provides insight into this task-specific performance difference in\u0000terms of target distributions and data collection strategies. We established\u0000that the performance of AL depends on the relative distribution of the target\u0000molecules in comparison to the total dataset distribution, with the largest\u0000computational savings achieved when their overlap is minimal.","PeriodicalId":501065,"journal":{"name":"arXiv - PHYS - Data Analysis, Statistics and Probability","volume":"37 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142179256","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Daen Jannis, Wouter Van den Broek, Zezhong Zhang, Sandra Van Aert, Jo Verbeeck
By working out the Bethe sum rule, a boundary condition that takes the form of a linear equality is derived for the fine structure observed in ionization edges present in electron energy-loss spectra. This condition is subsequently used as a constraint in the estimation process of the elemental abundances, demonstrating starkly improved precision and accuracy and reduced sensitivity to the number of model parameters. Furthermore, the fine structure is reliably extracted from the spectra in an automated way, thus providing critical information on the sample's electronic properties that is hard or impossible to obtain otherwise. Since this approach allows dispensing with the need for user-provided input, a potential source of bias is prevented.
{"title":"Improved precision and accuracy of electron energy-loss spectroscopy quantification via fine structure fitting with constrained optimization","authors":"Daen Jannis, Wouter Van den Broek, Zezhong Zhang, Sandra Van Aert, Jo Verbeeck","doi":"arxiv-2408.11870","DOIUrl":"https://doi.org/arxiv-2408.11870","url":null,"abstract":"By working out the Bethe sum rule, a boundary condition that takes the form\u0000of a linear equality is derived for the fine structure observed in ionization\u0000edges present in electron energy-loss spectra. This condition is subsequently\u0000used as a constraint in the estimation process of the elemental abundances,\u0000demonstrating starkly improved precision and accuracy and reduced sensitivity\u0000to the number of model parameters. Furthermore, the fine structure is reliably\u0000extracted from the spectra in an automated way, thus providing critical\u0000information on the sample's electronic properties that is hard or impossible to\u0000obtain otherwise. Since this approach allows dispensing with the need for\u0000user-provided input, a potential source of bias is prevented.","PeriodicalId":501065,"journal":{"name":"arXiv - PHYS - Data Analysis, Statistics and Probability","volume":"15 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142179255","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ziming Liu, Pingchuan Ma, Yixuan Wang, Wojciech Matusik, Max Tegmark
A major challenge of AI + Science lies in their inherent incompatibility: today's AI is primarily based on connectionism, while science depends on symbolism. To bridge the two worlds, we propose a framework to seamlessly synergize Kolmogorov-Arnold Networks (KANs) and science. The framework highlights KANs' usage for three aspects of scientific discovery: identifying relevant features, revealing modular structures, and discovering symbolic formulas. The synergy is bidirectional: science to KAN (incorporating scientific knowledge into KANs), and KAN to science (extracting scientific insights from KANs). We highlight major new functionalities in the pykan package: (1) MultKAN: KANs with multiplication nodes. (2) kanpiler: a KAN compiler that compiles symbolic formulas into KANs. (3) tree converter: convert KANs (or any neural networks) to tree graphs. Based on these tools, we demonstrate KANs' capability to discover various types of physical laws, including conserved quantities, Lagrangians, symmetries, and constitutive laws.
当今的人工智能主要基于连接主义，而科学则依赖于符号主义。为了沟通这两个世界，我们提出了一个将科尔莫哥罗夫-阿诺德网络（KANs）与科学无缝协同的框架。该框架强调了 KAN 在科学发现三个方面的用途：识别相关特征、揭示模块结构和发现符号公式。协同作用是双向的：科学到 KAN（将科学知识纳入 KAN），KAN 到科学（从 KAN 中提取科学见解）。我们重点介绍 pykanpackage 中的主要新功能：(1) MultKAN：带有乘法节点的 KAN。(2) kanpiler：将符号公式编译成 KAN 的 KAN 编译器。(3) 树状图转换器：将 KAN（或任何神经网络）转换为树状图。基于这些工具，我们展示了 KAN 发现各类物理定律的能力，包括守恒量、拉格朗日、对称性和构成定律。
{"title":"KAN 2.0: Kolmogorov-Arnold Networks Meet Science","authors":"Ziming Liu, Pingchuan Ma, Yixuan Wang, Wojciech Matusik, Max Tegmark","doi":"arxiv-2408.10205","DOIUrl":"https://doi.org/arxiv-2408.10205","url":null,"abstract":"A major challenge of AI + Science lies in their inherent incompatibility:\u0000today's AI is primarily based on connectionism, while science depends on\u0000symbolism. To bridge the two worlds, we propose a framework to seamlessly\u0000synergize Kolmogorov-Arnold Networks (KANs) and science. The framework\u0000highlights KANs' usage for three aspects of scientific discovery: identifying\u0000relevant features, revealing modular structures, and discovering symbolic\u0000formulas. The synergy is bidirectional: science to KAN (incorporating\u0000scientific knowledge into KANs), and KAN to science (extracting scientific\u0000insights from KANs). We highlight major new functionalities in the pykan\u0000package: (1) MultKAN: KANs with multiplication nodes. (2) kanpiler: a KAN\u0000compiler that compiles symbolic formulas into KANs. (3) tree converter: convert\u0000KANs (or any neural networks) to tree graphs. Based on these tools, we\u0000demonstrate KANs' capability to discover various types of physical laws,\u0000including conserved quantities, Lagrangians, symmetries, and constitutive laws.","PeriodicalId":501065,"journal":{"name":"arXiv - PHYS - Data Analysis, Statistics and Probability","volume":"153 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142179257","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Prognosis and diagnosis play an important role in accelerating the development of lithium-ion batteries, as well as reliable and long-life operation. In this work, we answer an important question: What is the minimum amount of data required to extract features for accurate battery prognosis and diagnosis? Based on the first principle, we successfully extracted the best two-point feature (BTPF) for accurate battery prognosis and diagnosis using the fewest data points (only two) and the simplest feature selection method (Pearson correlation coefficient). The BTPF extraction method is tested on 820 cells from 6 open-source datasets (covering five different chemistry types, seven manufacturers, and three data types). It achieves comparable accuracy to state-of-the-art features in both prognosis and diagnosis tasks. This work challenges the cognition of existing studies on the difficulty of battery prognosis and diagnosis tasks, subverts the fixed pattern of establishing prognosis and diagnosis methods for complex dynamic systems through deliberate feature engineering, highlights the promise of data-driven methods for field battery prognosis and diagnosis applications, and provides a new benchmark for future studies.
{"title":"Two points are enough","authors":"Hao Liu, Yanbin Zhao, Huarong Zheng, Xiulin Fan, Zhihua Deng, Mengchi Chen, Xingkai Wang, Zhiyang Liu, Jianguo Lu, Jian Chen","doi":"arxiv-2408.11872","DOIUrl":"https://doi.org/arxiv-2408.11872","url":null,"abstract":"Prognosis and diagnosis play an important role in accelerating the\u0000development of lithium-ion batteries, as well as reliable and long-life\u0000operation. In this work, we answer an important question: What is the minimum\u0000amount of data required to extract features for accurate battery prognosis and\u0000diagnosis? Based on the first principle, we successfully extracted the best\u0000two-point feature (BTPF) for accurate battery prognosis and diagnosis using the\u0000fewest data points (only two) and the simplest feature selection method\u0000(Pearson correlation coefficient). The BTPF extraction method is tested on 820\u0000cells from 6 open-source datasets (covering five different chemistry types,\u0000seven manufacturers, and three data types). It achieves comparable accuracy to\u0000state-of-the-art features in both prognosis and diagnosis tasks. This work\u0000challenges the cognition of existing studies on the difficulty of battery\u0000prognosis and diagnosis tasks, subverts the fixed pattern of establishing\u0000prognosis and diagnosis methods for complex dynamic systems through deliberate\u0000feature engineering, highlights the promise of data-driven methods for field\u0000battery prognosis and diagnosis applications, and provides a new benchmark for\u0000future studies.","PeriodicalId":501065,"journal":{"name":"arXiv - PHYS - Data Analysis, Statistics and Probability","volume":"1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142179254","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Zihan Zhao, Farouk Mokhtar, Raghav Kansal, Haoyang Li, Javier Duarte
This study introduces an innovative approach to analyzing unlabeled data in high-energy physics (HEP) through the application of self-supervised learning (SSL). Faced with the increasing computational cost of producing high-quality labeled simulation samples at the CERN LHC, we propose leveraging large volumes of unlabeled data to overcome the limitations of supervised learning methods, which heavily rely on detailed labeled simulations. By pretraining models on these vast, mostly untapped datasets, we aim to learn generic representations that can be finetuned with smaller quantities of labeled data. Our methodology employs contrastive learning with augmentations on jet datasets to teach the model to recognize common representations of jets, addressing the unique challenges of LHC physics. Building on the groundwork laid by previous studies, our work demonstrates the critical ability of SSL to utilize large-scale unlabeled data effectively. We showcase the scalability and effectiveness of our models by gradually increasing the size of the pretraining dataset and assessing the resultant performance enhancements. Our results, obtained from experiments on two datasets -- JetClass, representing unlabeled data, and Top Tagging, serving as labeled simulation data -- show significant improvements in data efficiency, computational efficiency, and overall performance. These findings suggest that SSL can greatly enhance the adaptability of ML models to the HEP domain. This work opens new avenues for the use of unlabeled data in HEP and contributes to a better understanding the potential of SSL for scientific discovery.
本研究介绍了一种通过应用自监督学习（SSL）来分析高能物理（HEP）中未标记数据的创新方法。面对欧洲核子研究中心大型强子对撞机（CERN LHC）制作高质量标签模拟样本的计算成本不断增加的问题，我们建议利用大量未标签数据来克服监督学习方法的局限性，因为监督学习方法严重依赖于详细的标签模拟。通过在这些庞大的、大部分尚未开发的数据集上预训练模型，我们的目标是学习通用表示法，然后再用较小数量的标注数据进行微调。我们的方法利用对比学习和喷流数据集上的增强来教模型识别喷流的常见表示，从而解决大型强子对撞机物理的独特挑战。在以往研究奠定的基础上，我们的工作展示了 SSL 有效利用大规模无标记数据的关键能力。我们通过逐步增加预训练数据集的规模和评估由此带来的性能提升，展示了我们模型的可扩展性和有效性。我们在两个数据集（代表无标签数据的 JetClass 和作为有标签模拟数据的 TopTagging）上的实验结果表明，数据效率、计算效率和整体性能都有显著提高。这些发现表明，SSL 可以大大提高 ML 模型在 HEP 领域的适应性。这项工作为在 HEP 中使用无标记数据开辟了新途径，有助于更好地了解 SSL 在科学发现方面的潜力。
{"title":"Large-Scale Pretraining and Finetuning for Efficient Jet Classification in Particle Physics","authors":"Zihan Zhao, Farouk Mokhtar, Raghav Kansal, Haoyang Li, Javier Duarte","doi":"arxiv-2408.09343","DOIUrl":"https://doi.org/arxiv-2408.09343","url":null,"abstract":"This study introduces an innovative approach to analyzing unlabeled data in\u0000high-energy physics (HEP) through the application of self-supervised learning\u0000(SSL). Faced with the increasing computational cost of producing high-quality\u0000labeled simulation samples at the CERN LHC, we propose leveraging large volumes\u0000of unlabeled data to overcome the limitations of supervised learning methods,\u0000which heavily rely on detailed labeled simulations. By pretraining models on\u0000these vast, mostly untapped datasets, we aim to learn generic representations\u0000that can be finetuned with smaller quantities of labeled data. Our methodology\u0000employs contrastive learning with augmentations on jet datasets to teach the\u0000model to recognize common representations of jets, addressing the unique\u0000challenges of LHC physics. Building on the groundwork laid by previous studies,\u0000our work demonstrates the critical ability of SSL to utilize large-scale\u0000unlabeled data effectively. We showcase the scalability and effectiveness of\u0000our models by gradually increasing the size of the pretraining dataset and\u0000assessing the resultant performance enhancements. Our results, obtained from\u0000experiments on two datasets -- JetClass, representing unlabeled data, and Top\u0000Tagging, serving as labeled simulation data -- show significant improvements in\u0000data efficiency, computational efficiency, and overall performance. These\u0000findings suggest that SSL can greatly enhance the adaptability of ML models to\u0000the HEP domain. This work opens new avenues for the use of unlabeled data in\u0000HEP and contributes to a better understanding the potential of SSL for\u0000scientific discovery.","PeriodicalId":501065,"journal":{"name":"arXiv - PHYS - Data Analysis, Statistics and Probability","volume":"47 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142179259","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We present Euler Characteristic Surfaces as a multiscale spatiotemporal topological summary of time series data encapsulating the topology of the system at different time instants and length scales. Euler Characteristic Surfaces with an appropriate metric is used to quantify stability and locate critical changes in a dynamical system with respect to variations in a parameter, while being substantially computationally cheaper than available alternate methods such as persistent homology. The stability of the construction is demonstrated by a quantitative comparison bound with persistent homology, and a quantitative stability bound under small changes in time is established. The proposed construction is used to analyze two different kinds of simulated disordered flow situations.
{"title":"Euler Characteristic Surfaces: A Stable Multiscale Topological Summary of Time Series Data","authors":"Anamika Roy, Atish J. Mitra, Tapati Dutta","doi":"arxiv-2408.09400","DOIUrl":"https://doi.org/arxiv-2408.09400","url":null,"abstract":"We present Euler Characteristic Surfaces as a multiscale spatiotemporal\u0000topological summary of time series data encapsulating the topology of the\u0000system at different time instants and length scales. Euler Characteristic\u0000Surfaces with an appropriate metric is used to quantify stability and locate\u0000critical changes in a dynamical system with respect to variations in a\u0000parameter, while being substantially computationally cheaper than available\u0000alternate methods such as persistent homology. The stability of the\u0000construction is demonstrated by a quantitative comparison bound with persistent\u0000homology, and a quantitative stability bound under small changes in time is\u0000established. The proposed construction is used to analyze two different kinds\u0000of simulated disordered flow situations.","PeriodicalId":501065,"journal":{"name":"arXiv - PHYS - Data Analysis, Statistics and Probability","volume":"110 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142179258","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Raphael T. Husistein, Markus Reiher, Marco Eckhoff
Artificial neural networks have been shown to be state-of-the-art machine learning models in a wide variety of applications, including natural language processing and image recognition. However, building a performant neural network is a laborious task and requires substantial computing power. Neural Architecture Search (NAS) addresses this issue by an automatic selection of the optimal network from a set of potential candidates. While many NAS methods still require training of (some) neural networks, zero-cost proxies promise to identify the optimal network without training. In this work, we propose the zero-cost proxy Network Expressivity by Activation Rank (NEAR). It is based on the effective rank of the pre- and post-activation matrix, i.e., the values of a neural network layer before and after applying its activation function. We demonstrate the cutting-edge correlation between this network score and the model accuracy on NAS-Bench-101 and NATS-Bench-SSS/TSS. In addition, we present a simple approach to estimate the optimal layer sizes in multi-layer perceptrons. Furthermore, we show that this score can be utilized to select hyperparameters such as the activation function and the neural network weight initialization scheme.
在自然语言处理和图像识别等多种应用中，人工神经网络已被证明是最先进的机器学习模型。然而，构建一个性能良好的神经网络是一项艰巨的任务，需要强大的计算能力。神经架构搜索（NAS）通过从一组潜在候选网络中自动选择最佳网络来解决这一问题。虽然许多 NAS 方法仍需要对（某些）神经网络进行训练，但零成本代理有望在无需训练的情况下识别出最佳网络。在这项工作中，我们提出了零成本代理 "激活等级网络表达性"（NEAR）。它基于激活前和激活后矩阵的有效秩，即神经网络层在应用激活函数前后的值。我们在 NAS-Bench-101 和 NATS-Bench-SSS/TSS 上展示了该网络得分与模型准确性之间的尖端相关性。此外，我们还介绍了一种估算多层感知器最佳层大小的简单方法。此外，我们还展示了可以利用这一分数来选择激活函数和神经网络权重初始化方案等参数。
{"title":"NEAR: A Training-Free Pre-Estimator of Machine Learning Model Performance","authors":"Raphael T. Husistein, Markus Reiher, Marco Eckhoff","doi":"arxiv-2408.08776","DOIUrl":"https://doi.org/arxiv-2408.08776","url":null,"abstract":"Artificial neural networks have been shown to be state-of-the-art machine\u0000learning models in a wide variety of applications, including natural language\u0000processing and image recognition. However, building a performant neural network\u0000is a laborious task and requires substantial computing power. Neural\u0000Architecture Search (NAS) addresses this issue by an automatic selection of the\u0000optimal network from a set of potential candidates. While many NAS methods\u0000still require training of (some) neural networks, zero-cost proxies promise to\u0000identify the optimal network without training. In this work, we propose the\u0000zero-cost proxy Network Expressivity by Activation Rank (NEAR). It is based on\u0000the effective rank of the pre- and post-activation matrix, i.e., the values of\u0000a neural network layer before and after applying its activation function. We\u0000demonstrate the cutting-edge correlation between this network score and the\u0000model accuracy on NAS-Bench-101 and NATS-Bench-SSS/TSS. In addition, we present\u0000a simple approach to estimate the optimal layer sizes in multi-layer\u0000perceptrons. Furthermore, we show that this score can be utilized to select\u0000hyperparameters such as the activation function and the neural network weight\u0000initialization scheme.","PeriodicalId":501065,"journal":{"name":"arXiv - PHYS - Data Analysis, Statistics and Probability","volume":"15 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142179260","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：