LLVM code optimisation for automatic differentiation: when forward and reverse mode lead in the same direction

Proceedings of the Sixth Workshop on Data Management for End-To-End Machine Learning Pub Date : 2022-06-12 DOI:10.1145/3533028.3533302

Maximilian E. Schüle, M. Springer, A. Kemper

引用次数: 1

Abstract

Both forward and reverse mode automatic differentiation derive a model function as used for gradient descent automatically. Reverse mode calculates all derivatives in one run, whereas forward mode requires rerunning the algorithm with respect to every variable for which the derivative is needed. To allow for in-database machine learning, we have integrated automatic differentiation as an SQL operator inside the Umbra database system. To benchmark code-generation to GPU, we implement forward as well as reverse mode automatic differentiation. The inspection of the optimised LLVM code shows that nearly the same machine code is executed after the generated LLVM code has been optimised. Thus, both modes yield similar runtimes but different compilation times.

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

LLVM代码优化自动区分:当向前和反向模式导致在同一方向

正反两种模式的自动微分都自动导出了用于梯度下降的模型函数。反向模式在一次运行中计算所有导数，而正向模式需要针对需要导数的每个变量重新运行算法。为了允许数据库内机器学习，我们在Umbra数据库系统中集成了自动区分作为SQL操作符。为了基准代码生成到GPU，我们实现了正向和反向模式的自动微分。对优化后的LLVM代码的检查显示，在生成的LLVM代码被优化后，几乎执行了相同的机器码。因此，这两种模式产生相似的运行时，但编译时间不同。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

Proceedings of the Sixth Workshop on Data Management for End-To-End Machine Learning

自引率

0.00%

发文量

期刊最新文献

dcbench GouDa - generation of universal data sets: improving analysis and evaluation of data preparation pipelines How I stopped worrying about training data bugs and started complaining Evaluating model serving strategies over streaming data Accelerating container-based deep learning hyperparameter optimization workloads