Bounds on Nonlinear Errors for Variance Computation with Stochastic Rounding

SIAM Journal on Scientific Computing, Volume 46, Issue 5, Page B579-B599, October 2024.
Abstract. The main objective of this work is to investigate nonlinear errors and pairwise summation using stochastic rounding (SR) in variance computation algorithms. We estimate the forward error of computations under SR through two methods: the first is based on a bound of the variance and the Bienaymé–Chebyshev inequality, while the second is based on martingales and the Azuma–Hoeffding inequality. The study shows that for pairwise summation, using SR results in a probabilistic bound of the forward error proportional to [math] rather than the deterministic bound in [math] when using the default rounding mode. We examine two algorithms that compute the variance, one called “textbook” and the other “two-pass,” which both exhibit nonlinear errors. Using the two methods mentioned above, we show that the forward errors of these algorithms have probabilistic bounds under SR in [math] instead of [math] for the deterministic bounds. We show that this advantage holds using pairwise summation for both textbook and two-pass, with probabilistic bounds of the forward error proportional to [math]. Reproducibility of computational results. This paper has been awarded the “SIAM Reproducibility Badge: Code and data available” as recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow the reader to reproduce the results in this paper are available at https://github.com/verificarlo/sr-non-linear-bounds and in the supplementary material (sr-non-linear-bounds-main.zip [8.62KB]).