Numerical stability of algorithms at extreme scale and low precisions
Summary: The largest dense linear systems that are being solved today are of order \(n=10^7\). Single-precision arithmetic, which has a unit roundoff \(u \approx 10^{-8}\), is widely used in scientific computing, and half-precision arithmetic, with \(u \approx 10^{-4}\), is increasingly being exploited as it becomes more readily available in hardware. Standard rounding error bounds for numerical linear algebra algorithms are proportional to \(p(n)u\), with \(p\) growing at least linearly with \(n\). Therefore we are at the stage where these rounding error bounds are not able to guarantee any accuracy or stability in the computed results for some extreme-scale or low-accuracy computations. We explain how rounding error bounds with much smaller constants can be obtained. Blocked algorithms, which break the data into blocks of size \(b\), lead to a reduction in the error constants by a factor \(b\) or more. Two architectural features also reduce the error constants: extended precision registers and fused multiply-add operations, either at the scalar level or in mixed precision block form. We also discuss a new probabilistic approach to rounding error analysis that provides error constants that are the square roots of those of the worst-case bounds. Combining these different considerations provides new understanding of the numerical stability of extreme scale and low precision computations in numerical linear algebra. For the entire collection see [Zbl 07816361].
- Mixed precision algorithms in numerical linear algebra
- A New Approach to Probabilistic Rounding Error Analysis
- Squeezing a Matrix into Half Precision, with an Application to Solving Linear Systems
- Accelerating the solution of linear systems by iterative refinement in three precisions
- On numerical stability in large scale linear algebraic computations
- A Class of Fast and Accurate Summation Algorithms
- A New Approach to Probabilistic Rounding Error Analysis
- A set of level 3 basic linear algebra subprograms
- Accelerating the solution of linear systems by iterative refinement in three precisions
- Accuracy and Stability of Numerical Algorithms
- Anatomy of high-performance matrix multiplication
- Error Analysis of Direct Methods of Matrix Inversion
- Handbook of floating-point arithmetic
- scientific article; zbMATH DE number 1024452 (Why is no real title available?)
- scientific article; zbMATH DE number 1049347 (Why is no real title available?)
- scientific article; zbMATH DE number 6159604 (Why is no real title available?)
- LAPACK Users' Guide
- Mixed precision block fused multiply-add: error analysis and application to GPU tensor cores
- Mixed-precision iterative refinement using tensor cores on GPUs to accelerate solution of linear systems
- Modern Error Analysis
- Probabilistic Error Analysis for Inner Products
- Random matrices generating large growth in LU factorization with pivoting
- Reducing floating point error in dot product using the superblock family of algorithms
- Sharper probabilistic backward error analysis for basic linear algebra kernels with random data
- Squeezing a Matrix into Half Precision, with an Application to Solving Linear Systems
- Stochastic Perturbation Theory
- Stochastic rounding and its probabilistic backward error analysis
This page was built for publication: Numerical stability of algorithms at extreme scale and low precisions
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6200205)
