Approximate Message Passing With Consistent Parameter Estimation and Applications to Sparse Learning

DOI10.1109/TIT.2014.2309005MaRDI QIDQ2986410zbMATH OpenOpenAlexFDO

Authors Ulugbek Kamilov, Sundeep Rangan, Alyson K. Fletcher, M. Unser

Publication date 16 May 2017

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1207.3859

Mathematics Subject Classification ID

Asymptotic properties of parametric estimators (62F12) Estimation in multivariate analysis (62H12) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05) Channel models (including quantum) in information and communication theory (94A40)

Abstract: We consider the estimation of an i.i.d. (possibly non-Gaussian) vector $xbfin{R}^n$ from measurements $ybfin{R}^m$ obtained by a general cascade model consisting of a known linear transform followed by a probabilistic componentwise (possibly nonlinear) measurement channel. A novel method, called adaptive generalized approximate message passing (Adaptive GAMP), that enables joint learning of the statistics of the prior and measurement channel along with estimation of the unknown vector $xbf$ is presented. The proposed algorithm is a generalization of a recently-developed EM-GAMP that uses expectation-maximization (EM) iterations where the posteriors in the E-steps are computed via approximate message passing. The methodology can be applied to a large class of learning problems including the learning of sparse priors in compressed sensing or identification of linear-nonlinear cascade models in dynamical systems and neural spiking processes. We prove that for large i.i.d. Gaussian transform matrices the asymptotic componentwise behavior of the adaptive GAMP algorithm is predicted by a simple set of scalar state evolution equations. In addition, we show that when a certain maximum-likelihood estimation can be performed in each step, the adaptive GAMP method can yield asymptotically consistent parameter estimates, which implies that the algorithm achieves a reconstruction quality equivalent to the oracle algorithm that knows the correct parameter values. Remarkably, this result applies to essentially arbitrary parametrizations of the unknown distributions, including ones that are nonlinear and non-Gaussian. The adaptive GAMP methodology thus provides a systematic, general and computationally efficient method applicable to a large range of complex linear-nonlinear models with provable guarantees.

Recommendations

• Consistent parameter estimation for Lasso and approximate message passing
• Consistent parameter estimation for Lasso and approximate message passing
• A Unifying Tutorial on Approximate Message Passing
• Approximate message passing for nonconvex sparse regularization with stability and asymptotic analysis
• Approximate message passing with spectral initialization for generalized linear models*
• On the Convergence of Approximate Message Passing With Arbitrary Matrices
• Approximate Message Passing Algorithm With Universal Denoising and Gaussian Mixture Learning
• Estimation of low-rank matrices via approximate message passing
• Sparse Multinomial Logistic Regression via Approximate Message Passing

Cited in

(6)

• Bilinear Generalized Approximate Message Passing—Part I: Derivation
• Hybrid Approximate Message Passing
• Robustness in sparse high-dimensional linear models: relative efficiency and robust approximate message passing
• Analysis of Approximate Message Passing With Non-Separable Denoisers and Markov Random Field Priors
• Plug in estimation in high dimensional linear inverse problems a rigorous analysis
• A Unifying Tutorial on Approximate Message Passing

This page was built for publication: Approximate Message Passing With Consistent Parameter Estimation and Applications to Sparse Learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2986410)