Stochastic Rounding Variance and Probabilistic Bounds: A New Approach

DOI10.1137/22M1510819MaRDI QIDQ6050996zbMATH OpenOpenAlexFDO

Authors Devan Sohier, Pablo de Oliveira Castro, Eric Petit

Publication date 12 October 2023

Published in SIAM Journal on Scientific Computing (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2207.10321

zbMATH Keywords

concentration inequalityinner productfloating-point arithmeticpolynomial evaluationHorner algorithmstochastic rounding

Mathematics Subject Classification ID

Direct numerical methods for linear systems and matrix inversion (65F05) Roundoff error (65G50)

Abstract: Stochastic rounding (SR) offers an alternative to the deterministic IEEE-754 floating-point rounding modes. In some applications such as PDEs, ODEs and neural networks, SR empirically improves the numerical behavior and convergence to accurate solutions while no sound theoretical background has been provided. Recent works by Ipsen, Zhou, Higham, and Mary have computed SR probabilistic error bounds for basic linear algebra kernels. For example, the inner product SR probabilistic bound of the forward error is proportional to $sqrt$ nu instead of nu for the default rounding mode. To compute the bounds, these works show that the errors accumulated in computation form a martingale. This paper proposes an alternative framework to characterize SR errors based on the computation of the variance. We pinpoint common error patterns in numerical algorithms and propose a lemma that bounds their variance. For each probability and through Bienaym{'e}-Chebyshev inequality, this bound leads to better probabilistic error bound in several situations. Our method has the advantage of providing a tight probabilistic bound for all algorithms fitting our model. We show how the method can be applied to give SR error bounds for the inner product and Horner polynomial evaluation.

Recommendations

• Stochastic rounding and its probabilistic backward error analysis
• scientific article; zbMATH DE number 4111908
• scientific article; zbMATH DE number 1070527
• Probabilistic Error Analysis for Inner Products
• Rigorous roundoff error analysis of probabilistic floating-point computations

Cites work

• scientific article; zbMATH DE number 862514 (Why is no real title available?)
• A New Approach to Probabilistic Rounding Error Analysis
• Accuracy and Stability of Numerical Algorithms
• CADNA: a library for estimating round-off error propagation
• Concentration inequalities. A nonasymptotic theory of independence
• Confidence Intervals for Stochastic Arithmetic
• Effects of round-to-nearest and stochastic rounding in the numerical solution of the heat equation in low precision
• Numerical inverting of matrices of high order
• Probabilistic Error Analysis for Inner Products
• Probability and Computing
• Stochastic rounding and its probabilistic backward error analysis
• Stochastic rounding and reduced-precision fixed-point arithmetic for solving neural ordinary differential equations

Cited in

(5)

• Rounding probabilities: Maximum probability and minimum complexity multipliers
• Bounds on nonlinear errors for variance computation with stochastic rounding
• Rounding of continuous random variables and oscillatory asymptotics
• Stochastic rounding and its probabilistic backward error analysis
• An optimal (ϵ,δ)‐randomized approximation scheme for the mean of random variables with bounded relative variance

This page was built for publication: Stochastic Rounding Variance and Probabilistic Bounds: A New Approach

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6050996)