Inference for Generalized Linear Models via Alternating Directions and Bethe Free Energy Minimization

DOI10.1109/TIT.2016.2619373MaRDI QIDQ2979130zbMATH OpenFDO

Authors Sundeep Rangan, Alyson K. Fletcher, Philip Schniter, Ulugbek Kamilov

Publication date 2 May 2017

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1501.01797

Mathematics Subject Classification ID

Signal theory (characterization, reconstruction, filtering, etc.) (94A12)

Abstract: Generalized Linear Models (GLMs), where a random vector $mathbfx$ is observed through a noisy, possibly nonlinear, function of a linear transform $mathbfz=mathbfAx$ arise in a range of applications in nonlinear filtering and regression. Approximate Message Passing (AMP) methods, based on loopy belief propagation, are a promising class of approaches for approximate inference in these models. AMP methods are computationally simple, general, and admit precise analyses with testable conditions for optimality for large i.i.d. transforms $mathbfA$. However, the algorithms can easily diverge for general $mathbfA$. This paper presents a convergent approach to the generalized AMP (GAMP) algorithm based on direct minimization of a large-system limit approximation of the Bethe Free Energy (LSL-BFE). The proposed method uses a double-loop procedure, where the outer loop successively linearizes the LSL-BFE and the inner loop minimizes the linearized LSL-BFE using the Alternating Direction Method of Multipliers (ADMM). The proposed method, called ADMM-GAMP, is similar in structure to the original GAMP method, but with an additional least-squares minimization. It is shown that for strictly convex, smooth penalties, ADMM-GAMP is guaranteed to converge to a local minima of the LSL-BFE, thus providing a convergent alternative to GAMP that is stable under arbitrary transforms. Simulations are also presented that demonstrate the robustness of the method for non-convex penalties as well.

Cited in

(5)

• A statistical mechanics approach to de-biasing and uncertainty estimation in Lasso for random measurements
• Compressive Computed Tomography Reconstruction through Denoising Approximate Message Passing
• Replica analysis of overfitting in generalized linear regression models
• Free energy of multi-layer generalized linear models
• Local convexity of the TAP free energy and AMP convergence for \(\mathbb{Z}_2\)-synchronization

This page was built for publication: Inference for Generalized Linear Models via Alternating Directions and Bethe Free Energy Minimization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2979130)