Roundoff error analysis of fast DCT algorithms in fixed point arithmetic (Q2463439)

Processors in digital signal processing often use fixed point arithmetic. In this paper, the author considers the numerical stability of fast algorithms of discrete cosine transforms (DCT) in fixed point arithmetic. The fast DCT algorithms are based on known factorizations of the corresponding cosine matrices into products of sparse, orthogonal matrices of simple structure. Using the fixed point arithmetic model of \textit{J. von Neumann} and \textit{H. H. Goldstine} [Bull. Am. Math. Soc. 53, 1021--1099 (1947; Zbl 0031.31402)], the author presents a worst case roundoff error analysis for fast DCT algorithms. It is shown that under natural conditions, the fast DCT algorithms possess a good numerical behavior in fixed point arithmetic. Numerical tests demonstrate the performance of these results.