A remarkable example of catastrophic cancellation unraveled (Q5943903): Difference between revisions

The authors investigate the Rump's example of computing of following expression \[ y= 333.75b^6+ a^2(11a^2 b^2- b^6- 121b^4- 2)+ 5.5b^8+{a\over 2b}\text{ with }a= 77617\text{ and }b= 33096 \] by different hardware architectures supporting the IEEE 754 or 854 standards for floating-point arithmetics. The cancellation error produced in subtraction of nearby operands leads to different evaluation of \(y\) depending on the hardware platform, even if the warking program uses the apparently same precision for the entries. The so-called arithmetic explorer developed for didactical purpose at the University of Antwerp and available at the authors was used to perform the mathematical and bit level analysis of evaluation of expression under consideration. Some informations are given about the implementation of the multiprecision library with interval, rational endpoints and polynomial arithmetics.