scientific article; zbMATH DE number 1756451
Alternating minimization algorithms play a key role in information theory and estimation theory problems. R. E. Blahut described two computational algorithms that are important in information theory. The first algorithm, known as the Blahut-Arimoto algorithm, computes channel capacity and the probability distribution that achieves it. The second algorithm, known as Blahut's rate-distortion algorithm, computes the rate-distortion function and distributions that achieve it. Blahut derived these algorithms as alternating maximization and minimization algorithms. Two other closely related algorithms in estimation theory are the expectation-maximization algorithm and the generalized iterative scaling algorithm. The expectation-maximization algorithm was introduced by \textit{A. Dempster, N. Laird} and \textit{D. Rubin} [J. R. Stat. Soc., Ser. B 39, 1-38 (1997; Zbl 0364.62022)]. The generalized iterative scaling algorithm was introduced by \textit{J. Darroch} and \textit{D. Ratcliff} [Ann. Math. Stat. 43, 1470-1480 (1972; Zbl 0251.62020)] to find the distribution that maximizes entropy subject to a set of linear constraints. In this paper, the author examines these four algorithms in a unified framework and describes them as alternating minimization algorithms. The author also provides historical connections between these four algorithms. For the entire collection see [Zbl 0909.00065].
This page was built for publication:
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4535937)
