A Bit of Information Theory, and the Data Augmentation Algorithm Converges

DOI10.1109/TIT.2008.929918zbMATH Open1319.60154arXiv0811.0174MaRDI QIDQ3604970FDOQ3604970

Authors: Yaming Yu 

Publication date: 24 February 2009

Published in: IEEE Transactions on Information Theory (Search for Journal in Brave)

Abstract: The data augmentation (DA) algorithm is a simple and powerful tool in statistical computing. In this note basic information theory is used to prove a nontrivial convergence theorem for the DA algorithm.

Full work available at URL: https://arxiv.org/abs/0811.0174

Recommendations

• Rates of convergence for data augmentation on finite sample spaces
• scientific article; zbMATH DE number 4032799
• A comparison theorem for data augmentation algorithms with applications
• Improving the convergence properties of the data augmentation algorithm with an application to Bayesian mixture modeling
• The Calculation of Posterior Distributions by Data Augmentation

zbMATH Keywords

total variationMarkov chain Monte CarloKullback-Leibler divergenceGibbs samplingrelative entropyinformation geometryPinsker's inequalityI-projectionreverse I-projection

Mathematics Subject Classification ID

Computational methods in Markov chains (60J22) Monte Carlo methods (65C05) Information theory (general) (94A15)

Cited In (4)

• The data augmentation algorithm: theory and methodology
• Rates of convergence for data augmentation on finite sample spaces
• Spectral analytic comparisons for data augmentation
• Monotonic convergence of a general algorithm for computing optimal designs

This page was built for publication: A Bit of Information Theory, and the Data Augmentation Algorithm Converges

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3604970)