首页 > 最新文献

arXiv - CS - Mathematical Software最新文献

英文 中文
On the Average Runtime of an Open Source Binomial Random Variate Generation Algorithm 论开源二项式随机变量生成算法的平均运行时间
Pub Date : 2024-03-16 DOI: arxiv-2403.11018
Vincent A. Cicirello
The BTPE algorithm (Binomial, Triangle, Parallelogram, Exponential) ofKachitvichyanukul and Schmeiser is one of the faster and more widely utilizedalgorithms for generating binomial random variates. Cicirello's open sourceJava library, $rhomu$, includes an implementation of BTPE as well as avariety of other random number related utilities. In this report, I explore theaverage case runtime of the BTPE algorithm when generating random values frombinomial distribution $B(n,p)$. Beginning with Kachitvichyanukul andSchmeiser's formula for the expected number of acceptance-rejection samplingiterations, I analyze the limit behavior as $n$ approaches infinity, and showthat the average runtime of BTPE converges to a constant. I instrument the opensource Java implementation from the $rhomu$ library to experimentallyvalidate the analysis.
卡奇特维奇亚努库尔(Kachitvichyanukul)和施迈泽(Schmeiser)的 BTPE 算法(二项式、三角形、平行四边形、指数)是生成二项式随机变量的更快、更广泛使用的算法之一。Cicirello 的开源 Java 库 $rhomu$ 包括 BTPE 的实现以及其他多种与随机数相关的实用程序。在本报告中,我将探讨 BTPE 算法从二项分布 $B(n,p)$生成随机值时的平均运行时间。从 Kachitvichyanukul 和 Schmeiser 的预期接受-拒绝采样iterations 次数公式开始,我分析了当 $n$ 接近无穷大时的极限行为,并证明 BTPE 的平均运行时间收敛于一个常数。我利用 $rhomu$ 库中的开源 Java 实现对该分析进行了实验验证。
{"title":"On the Average Runtime of an Open Source Binomial Random Variate Generation Algorithm","authors":"Vincent A. Cicirello","doi":"arxiv-2403.11018","DOIUrl":"https://doi.org/arxiv-2403.11018","url":null,"abstract":"The BTPE algorithm (Binomial, Triangle, Parallelogram, Exponential) of\u0000Kachitvichyanukul and Schmeiser is one of the faster and more widely utilized\u0000algorithms for generating binomial random variates. Cicirello's open source\u0000Java library, $rhomu$, includes an implementation of BTPE as well as a\u0000variety of other random number related utilities. In this report, I explore the\u0000average case runtime of the BTPE algorithm when generating random values from\u0000binomial distribution $B(n,p)$. Beginning with Kachitvichyanukul and\u0000Schmeiser's formula for the expected number of acceptance-rejection sampling\u0000iterations, I analyze the limit behavior as $n$ approaches infinity, and show\u0000that the average runtime of BTPE converges to a constant. I instrument the open\u0000source Java implementation from the $rhomu$ library to experimentally\u0000validate the analysis.","PeriodicalId":501256,"journal":{"name":"arXiv - CS - Mathematical Software","volume":"54 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-03-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140172911","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
On a vectorized basic linear algebra package for prototyping codes in MATLAB 关于用于 MATLAB 代码原型开发的矢量化基本线性代数软件包
Pub Date : 2024-03-15 DOI: arxiv-2404.16039
Alexej Moskovka, Talal Rahman, Jan Valdman, Jon Eivind Vatne
When writing high-performance code for numerical computation in a scriptinglanguage like MATLAB, it is crucial to have the operations in a large for-loopvectorized. If not, the code becomes too slow to use, even for a moderatelylarge problem. However, in the process of vectorizing, the code often loses itsoriginal structure and becomes less readable. This is particularly true in thecase of a finite element implementation, even though finite element methods areinherently structured. A basic remedy to this is the separation of thevectorization part from the mathematics part of the code, which is easilyachieved through building the code on top of the basic linear algebrasubprograms that are already vectorized codes, an idea that has been used in aseries of papers over the last fifteen years, developing codes that are fastand still structured and readable. We discuss the vectorized basic linearalgebra package and introduce a formalism using multi-linear algebra to explainand define formally the functions in the package, as well as MATLAB pagetimefunctions. We provide examples from computations of varying complexity,including the computation of normal vectors, volumes, and finite elementmethods. Benchmarking shows that we also get fast computations. Using thelibrary, we can write codes that closely follow our mathematical thinking,making writing, following, reusing, and extending the code easier.
在 MATLAB 等脚本语言中编写高性能数值计算代码时,将大型 for 循环中的操作矢量化至关重要。否则,即使是处理中等规模的问题,代码也会变得太慢而无法使用。然而,在矢量化的过程中,代码往往会失去原有的结构,可读性也会降低。这在有限元实现中尤为明显,尽管有限元方法本身就是结构化的。解决这一问题的基本方法是将代码的矢量化部分与数学部分分离,这可以通过在已经是矢量化代码的基本线性代数子程序之上构建代码来轻松实现,过去 15 年中的一系列论文都采用了这一思路,开发出了既快速又具有结构性和可读性的代码。我们讨论了矢量化基本线性代数软件包,并介绍了一种使用多线性代数的形式主义,以正式解释和定义软件包中的函数以及 MATLAB 分页函数。我们提供了不同复杂度的计算实例,包括法向量、体积和有限元素方法的计算。基准测试表明,我们也能获得快速计算。使用该库,我们可以编写紧跟数学思维的代码,使代码的编写、跟踪、重用和扩展变得更加容易。
{"title":"On a vectorized basic linear algebra package for prototyping codes in MATLAB","authors":"Alexej Moskovka, Talal Rahman, Jan Valdman, Jon Eivind Vatne","doi":"arxiv-2404.16039","DOIUrl":"https://doi.org/arxiv-2404.16039","url":null,"abstract":"When writing high-performance code for numerical computation in a scripting\u0000language like MATLAB, it is crucial to have the operations in a large for-loop\u0000vectorized. If not, the code becomes too slow to use, even for a moderately\u0000large problem. However, in the process of vectorizing, the code often loses its\u0000original structure and becomes less readable. This is particularly true in the\u0000case of a finite element implementation, even though finite element methods are\u0000inherently structured. A basic remedy to this is the separation of the\u0000vectorization part from the mathematics part of the code, which is easily\u0000achieved through building the code on top of the basic linear algebra\u0000subprograms that are already vectorized codes, an idea that has been used in a\u0000series of papers over the last fifteen years, developing codes that are fast\u0000and still structured and readable. We discuss the vectorized basic linear\u0000algebra package and introduce a formalism using multi-linear algebra to explain\u0000and define formally the functions in the package, as well as MATLAB pagetime\u0000functions. We provide examples from computations of varying complexity,\u0000including the computation of normal vectors, volumes, and finite element\u0000methods. Benchmarking shows that we also get fast computations. Using the\u0000library, we can write codes that closely follow our mathematical thinking,\u0000making writing, following, reusing, and extending the code easier.","PeriodicalId":501256,"journal":{"name":"arXiv - CS - Mathematical Software","volume":"105 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-03-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140800882","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Cyclical Log Annealing as a Learning Rate Scheduler 作为学习率调度器的循环对数退火法
Pub Date : 2024-03-13 DOI: arxiv-2403.14685
Philip Naveen
A learning rate scheduler is a predefined set of instructions for varyingsearch stepsizes during model training processes. This paper introduces a newlogarithmic method using harsh restarting of step sizes through stochasticgradient descent. Cyclical log annealing implements the restart pattern moreaggressively to maybe allow the usage of more greedy algorithms on the onlineconvex optimization framework. The algorithm was tested on the CIFAR-10 imagedatasets, and seemed to perform analogously with cosine annealing on largetransformer-enhanced residual neural networks. Future experiments would involvetesting the scheduler in generative adversarial networks and finding the bestparameters for the scheduler with more experiments.
学习率调度器是一套预定义的指令,用于在模型训练过程中改变搜索步长。本文介绍了一种新的对数方法,该方法通过随机梯度下降对步长进行苛刻的重启。循环对数退火法以更激进的方式实现了重启模式,从而可以在在线凸优化框架中使用更贪婪的算法。该算法在 CIFAR-10 图像集上进行了测试,在大型变压器增强残差神经网络上的表现似乎与余弦退火类似。未来的实验将包括在生成对抗网络中测试调度器,并通过更多实验找到调度器的最佳参数。
{"title":"Cyclical Log Annealing as a Learning Rate Scheduler","authors":"Philip Naveen","doi":"arxiv-2403.14685","DOIUrl":"https://doi.org/arxiv-2403.14685","url":null,"abstract":"A learning rate scheduler is a predefined set of instructions for varying\u0000search stepsizes during model training processes. This paper introduces a new\u0000logarithmic method using harsh restarting of step sizes through stochastic\u0000gradient descent. Cyclical log annealing implements the restart pattern more\u0000aggressively to maybe allow the usage of more greedy algorithms on the online\u0000convex optimization framework. The algorithm was tested on the CIFAR-10 image\u0000datasets, and seemed to perform analogously with cosine annealing on large\u0000transformer-enhanced residual neural networks. Future experiments would involve\u0000testing the scheduler in generative adversarial networks and finding the best\u0000parameters for the scheduler with more experiments.","PeriodicalId":501256,"journal":{"name":"arXiv - CS - Mathematical Software","volume":"34 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141550885","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Efficient Calculations for k-diagonal Circulant Matrices and Cyclic Banded Matrices k 对角圆周矩阵和环带矩阵的高效计算
Pub Date : 2024-03-08 DOI: arxiv-2403.05048
Chen Wang, Chao Wang
Calculating the inverse of $k$-diagonal circulant matrices and cyclic bandedmatrices is a more challenging problem than calculating their determinants.Algorithms that directly involve or specify linear or quadratic complexity forthe inverses of these two types of matrices are rare. This paper presents twofast algorithms that can compute the complexity of a $k$-diagonal circulantmatrix within complexity $O(k^3 log n+k^4)+kn$, and for $k$-diagonal cyclicbanded matrices it is $O(k^3 n+k^5)+kn^2$. Since $k$ is generally much smallerthan $n$, the cost of these two algorithms can be approximated as $kn$ and$kn^2$.
计算$k$对角环形矩阵和环形带状矩阵的逆是一个比计算它们的行列式更具挑战性的问题。本文提出了两种快速算法,可以在复杂度为 $O(k^3 log n+k^4)+kn$ 的范围内计算 $k$ 对角环带矩阵的复杂度,而对于 $k$ 对角环带矩阵,计算复杂度为 $O(k^3 n+k^5)+kn^2$ 。由于 $k$ 通常比 $n$ 小得多,这两种算法的成本可以近似为 $kn$ 和 $kn^2$。
{"title":"Efficient Calculations for k-diagonal Circulant Matrices and Cyclic Banded Matrices","authors":"Chen Wang, Chao Wang","doi":"arxiv-2403.05048","DOIUrl":"https://doi.org/arxiv-2403.05048","url":null,"abstract":"Calculating the inverse of $k$-diagonal circulant matrices and cyclic banded\u0000matrices is a more challenging problem than calculating their determinants.\u0000Algorithms that directly involve or specify linear or quadratic complexity for\u0000the inverses of these two types of matrices are rare. This paper presents two\u0000fast algorithms that can compute the complexity of a $k$-diagonal circulant\u0000matrix within complexity $O(k^3 log n+k^4)+kn$, and for $k$-diagonal cyclic\u0000banded matrices it is $O(k^3 n+k^5)+kn^2$. Since $k$ is generally much smaller\u0000than $n$, the cost of these two algorithms can be approximated as $kn$ and\u0000$kn^2$.","PeriodicalId":501256,"journal":{"name":"arXiv - CS - Mathematical Software","volume":"44 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-03-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140097483","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
TopoX: A Suite of Python Packages for Machine Learning on Topological Domains TopoX:拓扑域机器学习 Python 软件包套件
Pub Date : 2024-02-04 DOI: arxiv-2402.02441
Mustafa Hajij, Mathilde Papillon, Florian Frantzen, Jens Agerberg, Ibrahem AlJabea, Ruben Ballester, Claudio Battiloro, Guillermo Bernárdez, Tolga Birdal, Aiden Brent, Peter Chin, Sergio Escalera, Simone Fiorellino, Odin Hoff Gardaa, Gurusankar Gopalakrishnan, Devendra Govil, Josef Hoppe, Maneel Reddy Karri, Jude Khouja, Manuel Lecha, Neal Livesay, Jan Meißner, Soham Mukherjee, Alexander Nikitin, Theodore Papamarkou, Jaro Prílepok, Karthikeyan Natesan Ramamurthy, Paul Rosen, Aldo Guzmán-Sáenz, Alessandro Salatiello, Shreyas N. Samaga, Simone Scardapane, Michael T. Schaub, Luca Scofano, Indro Spinelli, Lev Telyatnikov, Quang Truong, Robin Walters, Maosheng Yang, Olga Zaghen, Ghada Zamzmi, Ali Zia, Nina Miolane
We introduce topox, a Python software suite that provides reliable anduser-friendly building blocks for computing and machine learning on topologicaldomains that extend graphs: hypergraphs, simplicial, cellular, path andcombinatorial complexes. topox consists of three packages: toponetx facilitatesconstructing and computing on these domains, including working with nodes,edges and higher-order cells; topoembedx provides methods to embed topologicaldomains into vector spaces, akin to popular graph-based embedding algorithmssuch as node2vec; topomodelx is built on top of PyTorch and offers acomprehensive toolbox of higher-order message passing functions for neuralnetworks on topological domains. The extensively documented and unit-testedsource code of topox is available under MIT license athttps://github.com/pyt-team.
我们介绍的 topox 是一套 Python 软件,它为拓扑域的计算和机器学习提供了可靠、用户友好的构建模块,拓扑域包括:超图、单曲面、单元、路径和组合复合物。topox 由三个软件包组成:toponetx 方便在这些域上构建和计算,包括处理节点、边和高阶单元;topoembedx 提供将拓扑域嵌入向量空间的方法,类似于流行的基于图的嵌入算法,如 node2vec;topomodelx 基于 PyTorch 构建,为拓扑域上的神经网络提供了高阶消息传递函数的综合工具箱。topox 的源代码经过大量文档和单元测试,可在 MIT 许可下使用:https://github.com/pyt-team。
{"title":"TopoX: A Suite of Python Packages for Machine Learning on Topological Domains","authors":"Mustafa Hajij, Mathilde Papillon, Florian Frantzen, Jens Agerberg, Ibrahem AlJabea, Ruben Ballester, Claudio Battiloro, Guillermo Bernárdez, Tolga Birdal, Aiden Brent, Peter Chin, Sergio Escalera, Simone Fiorellino, Odin Hoff Gardaa, Gurusankar Gopalakrishnan, Devendra Govil, Josef Hoppe, Maneel Reddy Karri, Jude Khouja, Manuel Lecha, Neal Livesay, Jan Meißner, Soham Mukherjee, Alexander Nikitin, Theodore Papamarkou, Jaro Prílepok, Karthikeyan Natesan Ramamurthy, Paul Rosen, Aldo Guzmán-Sáenz, Alessandro Salatiello, Shreyas N. Samaga, Simone Scardapane, Michael T. Schaub, Luca Scofano, Indro Spinelli, Lev Telyatnikov, Quang Truong, Robin Walters, Maosheng Yang, Olga Zaghen, Ghada Zamzmi, Ali Zia, Nina Miolane","doi":"arxiv-2402.02441","DOIUrl":"https://doi.org/arxiv-2402.02441","url":null,"abstract":"We introduce topox, a Python software suite that provides reliable and\u0000user-friendly building blocks for computing and machine learning on topological\u0000domains that extend graphs: hypergraphs, simplicial, cellular, path and\u0000combinatorial complexes. topox consists of three packages: toponetx facilitates\u0000constructing and computing on these domains, including working with nodes,\u0000edges and higher-order cells; topoembedx provides methods to embed topological\u0000domains into vector spaces, akin to popular graph-based embedding algorithms\u0000such as node2vec; topomodelx is built on top of PyTorch and offers a\u0000comprehensive toolbox of higher-order message passing functions for neural\u0000networks on topological domains. The extensively documented and unit-tested\u0000source code of topox is available under MIT license at\u0000https://github.com/pyt-team.","PeriodicalId":501256,"journal":{"name":"arXiv - CS - Mathematical Software","volume":"45 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139752308","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
cmaes : A Simple yet Practical Python Library for CMA-ES cmaes :用于 CMA-ES 的简单实用的 Python 库
Pub Date : 2024-02-02 DOI: arxiv-2402.01373
Masahiro Nomura, Masashi Shibata
The covariance matrix adaptation evolution strategy (CMA-ES) has been highlyeffective in black-box continuous optimization, as demonstrated by its successin both benchmark problems and various real-world applications. To address theneed for an accessible yet potent tool in this domain, we developed cmaes, asimple and practical Python library for CMA-ES. cmaes is characterized by itssimplicity, offering intuitive use and high code readability. This makes itsuitable for quickly using CMA-ES, as well as for educational purposes andseamless integration into other libraries. Despite its simplistic design, cmaesmaintains enhanced functionality. It incorporates recent advancements inCMA-ES, such as learning rate adaptation for challenging scenarios, transferlearning, and mixed-integer optimization capabilities. These advanced featuresare accessible through a user-friendly API, ensuring that cmaes can be easilyadopted in practical applications. We regard cmaes as the first choice for aPython CMA-ES library among practitioners. The software is available under theMIT license at https://github.com/CyberAgentAILab/cmaes.
协方差矩阵适应演化策略(CMA-ES)在黑盒连续优化中非常有效,它在基准问题和各种实际应用中的成功都证明了这一点。为了满足这一领域对易用而强大的工具的需求,我们开发了用于 CMA-ES 的简单实用的 Python 库 cmaes。这使它既适合快速使用 CMA-ES,也适合教育目的和与其他库的无缝集成。尽管设计简单,但 cmaes 仍保持了增强的功能。它集成了 CMA-ES 的最新进展,如针对挑战性场景的学习率适应、迁移学习和混合整数优化功能。这些高级功能可通过用户友好的应用程序接口访问,确保 cmaes 可以轻松应用于实际应用中。我们认为 cmaes 是从业人员首选的 Python CMA-ES 库。该软件采用麻省理工学院(MIT)许可证,网址为 https://github.com/CyberAgentAILab/cmaes。
{"title":"cmaes : A Simple yet Practical Python Library for CMA-ES","authors":"Masahiro Nomura, Masashi Shibata","doi":"arxiv-2402.01373","DOIUrl":"https://doi.org/arxiv-2402.01373","url":null,"abstract":"The covariance matrix adaptation evolution strategy (CMA-ES) has been highly\u0000effective in black-box continuous optimization, as demonstrated by its success\u0000in both benchmark problems and various real-world applications. To address the\u0000need for an accessible yet potent tool in this domain, we developed cmaes, a\u0000simple and practical Python library for CMA-ES. cmaes is characterized by its\u0000simplicity, offering intuitive use and high code readability. This makes it\u0000suitable for quickly using CMA-ES, as well as for educational purposes and\u0000seamless integration into other libraries. Despite its simplistic design, cmaes\u0000maintains enhanced functionality. It incorporates recent advancements in\u0000CMA-ES, such as learning rate adaptation for challenging scenarios, transfer\u0000learning, and mixed-integer optimization capabilities. These advanced features\u0000are accessible through a user-friendly API, ensuring that cmaes can be easily\u0000adopted in practical applications. We regard cmaes as the first choice for a\u0000Python CMA-ES library among practitioners. The software is available under the\u0000MIT license at https://github.com/CyberAgentAILab/cmaes.","PeriodicalId":501256,"journal":{"name":"arXiv - CS - Mathematical Software","volume":"178 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139688521","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Reproducibility, energy efficiency and performance of pseudorandom number generators in machine learning: a comparative study of python, numpy, tensorflow, and pytorch implementations 机器学习中伪随机数生成器的可重复性、能效和性能:python、numpy、tensorflow 和 pytorch 实现的比较研究
Pub Date : 2024-01-30 DOI: arxiv-2401.17345
Benjamin Antunes, David R. C Hill
Pseudo-Random Number Generators (PRNGs) have become ubiquitous in machinelearning technologies because they are interesting for numerous methods. Thefield of machine learning holds the potential for substantial advancementsacross various domains, as exemplified by recent breakthroughs in LargeLanguage Models (LLMs). However, despite the growing interest, persistentconcerns include issues related to reproducibility and energy consumption.Reproducibility is crucial for robust scientific inquiry and explainability,while energy efficiency underscores the imperative to conserve finite globalresources. This study delves into the investigation of whether the leadingPseudo-Random Number Generators (PRNGs) employed in machine learning languages,libraries, and frameworks uphold statistical quality and numericalreproducibility when compared to the original C implementation of therespective PRNG algorithms. Additionally, we aim to evaluate the timeefficiency and energy consumption of various implementations. Our experimentsencompass Python, NumPy, TensorFlow, and PyTorch, utilizing the MersenneTwister, PCG, and Philox algorithms. Remarkably, we verified that the temporalperformance of machine learning technologies closely aligns with that ofC-based implementations, with instances of achieving even superiorperformances. On the other hand, it is noteworthy that ML technologies consumedonly 10% more energy than their C-implementation counterparts. However, whilestatistical quality was found to be comparable, achieving numericalreproducibility across different platforms for identical seeds and algorithmswas not achieved.
伪随机数发生器(PRNG)在机器学习技术中无处不在,因为它们对许多方法都很有趣。机器学习领域有可能在各个领域取得重大进展,最近在大型语言模型(LLMs)方面取得的突破就是例证。可重复性对于科学探索的稳健性和可解释性至关重要,而能效则强调了保护有限的全球资源的必要性。本研究深入探讨了机器学习语言、程序库和框架中使用的主要伪随机数发生器(PRNG)与相关 PRNG 算法的原始 C 语言实现相比,是否能够保证统计质量和数值可重复性。此外,我们还旨在评估各种实现的时间效率和能耗。我们的实验涵盖 Python、NumPy、TensorFlow 和 PyTorch,使用了梅森孪生、PCG 和 Philox 算法。值得注意的是,我们验证了机器学习技术的时间性能与基于C的实现非常接近,甚至有实现更优性能的实例。另一方面,值得注意的是,机器学习技术的能耗仅比基于 C 实现的技术高 10%。不过,虽然统计质量不相上下,但在不同平台上使用相同的种子和算法却无法实现数值上的可重复性。
{"title":"Reproducibility, energy efficiency and performance of pseudorandom number generators in machine learning: a comparative study of python, numpy, tensorflow, and pytorch implementations","authors":"Benjamin Antunes, David R. C Hill","doi":"arxiv-2401.17345","DOIUrl":"https://doi.org/arxiv-2401.17345","url":null,"abstract":"Pseudo-Random Number Generators (PRNGs) have become ubiquitous in machine\u0000learning technologies because they are interesting for numerous methods. The\u0000field of machine learning holds the potential for substantial advancements\u0000across various domains, as exemplified by recent breakthroughs in Large\u0000Language Models (LLMs). However, despite the growing interest, persistent\u0000concerns include issues related to reproducibility and energy consumption.\u0000Reproducibility is crucial for robust scientific inquiry and explainability,\u0000while energy efficiency underscores the imperative to conserve finite global\u0000resources. This study delves into the investigation of whether the leading\u0000Pseudo-Random Number Generators (PRNGs) employed in machine learning languages,\u0000libraries, and frameworks uphold statistical quality and numerical\u0000reproducibility when compared to the original C implementation of the\u0000respective PRNG algorithms. Additionally, we aim to evaluate the time\u0000efficiency and energy consumption of various implementations. Our experiments\u0000encompass Python, NumPy, TensorFlow, and PyTorch, utilizing the Mersenne\u0000Twister, PCG, and Philox algorithms. Remarkably, we verified that the temporal\u0000performance of machine learning technologies closely aligns with that of\u0000C-based implementations, with instances of achieving even superior\u0000performances. On the other hand, it is noteworthy that ML technologies consumed\u0000only 10% more energy than their C-implementation counterparts. However, while\u0000statistical quality was found to be comparable, achieving numerical\u0000reproducibility across different platforms for identical seeds and algorithms\u0000was not achieved.","PeriodicalId":501256,"journal":{"name":"arXiv - CS - Mathematical Software","volume":"12 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-01-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139657139","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Shortcutting Cross-Validation: Efficiently Deriving Column-Wise Centered and Scaled Training Set $mathbf{X}^mathbf{T}mathbf{X}$ and $mathbf{X}^mathbf{T}mathbf{Y}$ Without Full Recomputation of Matrix Products or Statistical Moments 交叉验证捷径:在不重新计算矩阵乘积或统计矩的情况下,高效地得出列居中和缩放的训练集 $mathbf{X}^mathbf{T}mathbf{X}$ 和 $mathbf{X}^mathbf{T}mathbf{Y}$
Pub Date : 2024-01-24 DOI: arxiv-2401.13185
Ole-Christian Galbo Engstrøm
Cross-validation is a widely used technique for assessing the performance ofpredictive models on unseen data. Many predictive models, such as Kernel-BasedPartial Least-Squares (PLS) models, require the computation of$mathbf{X}^{mathbf{T}}mathbf{X}$ and $mathbf{X}^{mathbf{T}}mathbf{Y}$using only training set samples from the input and output matrices,$mathbf{X}$ and $mathbf{Y}$, respectively. In this work, we present threealgorithms that efficiently compute these matrices. The first one allows nocolumn-wise preprocessing. The second one allows column-wise centering aroundthe training set means. The third one allows column-wise centering andcolumn-wise scaling around the training set means and standard deviations.Demonstrating correctness and superior computational complexity, they offersignificant cross-validation speedup compared with straight-forwardcross-validation and previous work on fast cross-validation - all without dataleakage. Their suitability for parallelization is highlighted with anopen-source Python implementation combining our algorithms with Improved KernelPLS.
交叉验证是一种广泛使用的技术,用于评估预测模型在未见数据上的性能。许多预测模型,如基于核的局部最小二乘(PLS)模型,需要分别使用输入矩阵 $mathbf{X}^{mathbf{T}}mathbf{X}$ 和输出矩阵 $mathbf{X}^{mathbf{T}}mathbf{Y}$ 的训练集样本来计算。在这项工作中,我们提出了三种能高效计算这些矩阵的算法。第一种算法允许进行无列预处理。第二种算法允许围绕训练集均值进行列居中。这些算法证明了其正确性和出色的计算复杂性,与直接向前交叉验证和以前的快速交叉验证相比,交叉验证的速度有了显著提高,而且没有数据损失。结合我们的算法和改进型 KernelPLS 的开源 Python 实现突出了它们的并行化适用性。
{"title":"Shortcutting Cross-Validation: Efficiently Deriving Column-Wise Centered and Scaled Training Set $mathbf{X}^mathbf{T}mathbf{X}$ and $mathbf{X}^mathbf{T}mathbf{Y}$ Without Full Recomputation of Matrix Products or Statistical Moments","authors":"Ole-Christian Galbo Engstrøm","doi":"arxiv-2401.13185","DOIUrl":"https://doi.org/arxiv-2401.13185","url":null,"abstract":"Cross-validation is a widely used technique for assessing the performance of\u0000predictive models on unseen data. Many predictive models, such as Kernel-Based\u0000Partial Least-Squares (PLS) models, require the computation of\u0000$mathbf{X}^{mathbf{T}}mathbf{X}$ and $mathbf{X}^{mathbf{T}}mathbf{Y}$\u0000using only training set samples from the input and output matrices,\u0000$mathbf{X}$ and $mathbf{Y}$, respectively. In this work, we present three\u0000algorithms that efficiently compute these matrices. The first one allows no\u0000column-wise preprocessing. The second one allows column-wise centering around\u0000the training set means. The third one allows column-wise centering and\u0000column-wise scaling around the training set means and standard deviations.\u0000Demonstrating correctness and superior computational complexity, they offer\u0000significant cross-validation speedup compared with straight-forward\u0000cross-validation and previous work on fast cross-validation - all without data\u0000leakage. Their suitability for parallelization is highlighted with an\u0000open-source Python implementation combining our algorithms with Improved Kernel\u0000PLS.","PeriodicalId":501256,"journal":{"name":"arXiv - CS - Mathematical Software","volume":"15 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-01-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139559558","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Theorem Discovery Amongst Cyclic Polygons 循环多边形中的定理发现
Pub Date : 2024-01-22 DOI: arxiv-2401.13002
Philip ToddSaltire Software
We examine a class of geometric theorems on cyclic 2n-gons. We prove that ifwe take n disjoint pairs of sides, each pair separated by an even number ofpolygon sides, then there is a linear combination of the angles between thosesides which is constant. We present a formula for the linear combination, whichprovides a theorem statement in terms of those angles. We describe a programwhich uses this result to generate new geometry proof problems and theirsolutions.
我们研究了关于循环 2n 边形的一类几何定理。我们证明,如果取 n 对互不相交的边,每对边之间隔着偶数条多边形边,那么这些边之间的角的线性组合是常数。我们给出了线性组合的公式,并用这些角给出了定理说明。我们描述了一个程序,该程序利用这一结果生成新的几何证明问题及其解答。
{"title":"Theorem Discovery Amongst Cyclic Polygons","authors":"Philip ToddSaltire Software","doi":"arxiv-2401.13002","DOIUrl":"https://doi.org/arxiv-2401.13002","url":null,"abstract":"We examine a class of geometric theorems on cyclic 2n-gons. We prove that if\u0000we take n disjoint pairs of sides, each pair separated by an even number of\u0000polygon sides, then there is a linear combination of the angles between those\u0000sides which is constant. We present a formula for the linear combination, which\u0000provides a theorem statement in terms of those angles. We describe a program\u0000which uses this result to generate new geometry proof problems and their\u0000solutions.","PeriodicalId":501256,"journal":{"name":"arXiv - CS - Mathematical Software","volume":"57 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-01-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139559561","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A Simulation of Optimal Dryness When Moving in the Rain or Snow Using MATLAB 使用 MATLAB 模拟在雨雪中移动时的最佳干燥度
Pub Date : 2024-01-22 DOI: arxiv-2401.12023
Neil Zhao, Emilee Brockner, Asia Winslow, Megan Seraydarian
The classic question of whether one should walk or run in the rain to remainthe least wet has inspired a myriad of solutions ranging from physicallyperforming test runs in raining conditions to mathematically modeling humanmovement through rain. This manuscript approaches the classical problem bysimulating movement through rainfall using MATLAB. Our simulation wasgeneralizable to include snowfall as well. An increase in walking speedresulted in a corresponding decrease in raindrop and snowflake collisions. Whenraindrops or snowflakes were given a horizontal movement vector due to wind, alocal minimum in collisions was achieved when moving in parallel with the samehorizontal speed as the raindrop; no local minimum was detected withantiparallel movement. In general, our simulation revealed that the faster onemoves, the drier one remains.
在雨中行走还是奔跑才能最不被淋湿,这个经典问题激发了无数的解决方案,从在雨中进行物理试运行,到在雨中对人类运动进行数学建模,不一而足。本手稿通过使用 MATLAB 模拟雨中运动来解决这一经典问题。我们的模拟还可以推广到降雪中。步行速度的增加导致雨滴和雪花碰撞的相应减少。当雨滴或雪花因风而具有水平运动矢量时,当以与雨滴相同的水平速度平行运动时,碰撞达到局部最小值;而反平行运动时,则检测不到局部最小值。总的来说,我们的模拟结果表明,移动速度越快的雪花越干燥。
{"title":"A Simulation of Optimal Dryness When Moving in the Rain or Snow Using MATLAB","authors":"Neil Zhao, Emilee Brockner, Asia Winslow, Megan Seraydarian","doi":"arxiv-2401.12023","DOIUrl":"https://doi.org/arxiv-2401.12023","url":null,"abstract":"The classic question of whether one should walk or run in the rain to remain\u0000the least wet has inspired a myriad of solutions ranging from physically\u0000performing test runs in raining conditions to mathematically modeling human\u0000movement through rain. This manuscript approaches the classical problem by\u0000simulating movement through rainfall using MATLAB. Our simulation was\u0000generalizable to include snowfall as well. An increase in walking speed\u0000resulted in a corresponding decrease in raindrop and snowflake collisions. When\u0000raindrops or snowflakes were given a horizontal movement vector due to wind, a\u0000local minimum in collisions was achieved when moving in parallel with the same\u0000horizontal speed as the raindrop; no local minimum was detected with\u0000antiparallel movement. In general, our simulation revealed that the faster one\u0000moves, the drier one remains.","PeriodicalId":501256,"journal":{"name":"arXiv - CS - Mathematical Software","volume":"5 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-01-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139559994","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
arXiv - CS - Mathematical Software
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1