Alternating Direction Methods for Latent Variable Gaussian Graphical Model Selection

DOI10.1162/NECO_A_00379zbMATH Open1418.62234arXiv1206.1275OpenAlexW2098589050WikidataQ44914079 ScholiaQ44914079MaRDI QIDQ5378251FDOQ5378251

Authors: Lingzhou Xue, Hui Zou, Shiqian Ma 

Publication date: 12 June 2019

Published in: Neural Computation (Search for Journal in Brave)

Abstract: Chandrasekaran, Parrilo and Willsky (2010) proposed a convex optimization problem to characterize graphical model selection in the presence of unobserved variables. This convex optimization problem aims to estimate an inverse covariance matrix that can be decomposed into a sparse matrix minus a low-rank matrix from sample data. Solving this convex optimization problem is very challenging, especially for large problems. In this paper, we propose two alternating direction methods for solving this problem. The first method is to apply the classical alternating direction method of multipliers to solve the problem as a consensus problem. The second method is a proximal gradient based alternating direction method of multipliers. Our methods exploit and take advantage of the special structure of the problem and thus can solve large problems very efficiently. Global convergence result is established for the proposed methods. Numerical results on both synthetic data and gene expression data show that our methods usually solve problems with one million variables in one to two minutes, and are usually five to thirty five times faster than a state-of-the-art Newton-CG proximal point algorithm.

Full work available at URL: https://arxiv.org/abs/1206.1275

Recommendations

• Latent variable graphical model selection via convex optimization
• Alternating direction method for covariance selection models
• Graphical model selection with latent variables
• Graphical model selection for Gaussian conditional random fields in the presence of latent variables
• Sparse Gaussian graphical model estimation via alternating minimization
• Rejoinder: Latent variable graphical model selection via convex optimization
• Model selection and estimation in the Gaussian graphical model
• Discussion: Latent variable graphical model selection via convex optimization
• Discussion: Latent variable graphical model selection via convex optimization
• Discussion: Latent variable graphical model selection via convex optimization

zbMATH Keywords

convex optimization probleminverse covariance matrixlatent variable Gaussian graphical model selection

Mathematics Subject Classification ID

Asymptotic properties of parametric estimators (62F12) Applications of statistics to biology and medical sciences; meta analysis (62P10) Convex programming (90C25) Applications of graph theory (05C90) Estimation in multivariate analysis (62H12) Protein sequences, DNA sequences (92D20)

Cites Work

• A New Alternating Minimization Algorithm for Total Variation Image Reconstruction
• Introduction to Graphical Modelling
• Title not available (Why is that?)
• Title not available (Why is that?)
• Distributed optimization and statistical learning via the alternating direction method of multipliers
• High-dimensional semiparametric Gaussian copula graphical models
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• High-dimensional graphs and variable selection with the Lasso
• On the Numerical Solution of Heat Conduction Problems in Two and Three Space Variables
• Splitting Algorithms for the Sum of Two Nonlinear Operators
• Regularized rank-based estimation of high-dimensional nonparanormal graphical models
• On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators
• Sparse inverse covariance estimation with the graphical lasso
• Sparse permutation invariant covariance estimation
• On the global and linear convergence of the generalized alternating direction method of multipliers
• Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data
• Model selection and estimation in the Gaussian graphical model
• Fast alternating linearization methods for minimizing the sum of two convex functions
• Title not available (Why is that?)
• Partial correlation estimation by joint sparse regression models
• Regularization methods for semidefinite programming
• Latent variable graphical model selection via convex optimization
• High-dimensional covariance estimation by minimizing \(\ell _{1}\)-penalized log-determinant divergence
• High dimensional inverse covariance matrix estimation via linear programming
• A constrained \(\ell _{1}\) minimization approach to sparse precision matrix estimation
• First-Order Methods for Sparse Covariance Selection
• Hankel matrix rank minimization with applications to system identification and realization
• Solving log-determinant optimization problems by a Newton-CG primal proximal point algorithm
• The Split Bregman Method for L1-Regularized Problems
• Estimating time-varying networks
• Signal Recovery by Proximal Forward-Backward Splitting
• A proximal-based deomposition method for compositions method for convex minimization problems
• Alternating direction method for covariance selection models
• Alternating direction algorithms for \(\ell_1\)-problems in compressive sensing
• A new inexact alternating directions method for monotone variational inequalities
• Alternating proximal gradient method for convex minimization
• Title not available (Why is that?)
• Positive-definite \(\ell_1\)-penalized estimation of large covariance matrices
• Alternating direction augmented Lagrangian methods for semidefinite programming
• An inexact interior point method for \(L_{1}\)-regularized sparse covariance selection
• Recovering Low-Rank and Sparse Components of Matrices from Incomplete and Noisy Observations
• The Numerical Solution of Parabolic and Elliptic Differential Equations
• Smooth Optimization Approach for Sparse Covariance Selection
• Fast multiple-splitting algorithms for convex optimization
• Title not available (Why is that?)
• An alternating direction method for total variation denoising

Cited In (30)

• Linearized symmetric multi-block ADMM with indefinite proximal regularization and optimal proximal parameter
• High-dimensional covariance matrix estimation
• Improved proximal ADMM with partially parallel splitting for multi-block separable convex programming
• Rejoinder: Latent variable graphical model selection via convex optimization
• Learning latent variable Gaussian graphical model for biomolecular network with low sample complexity
• An implementable first-order primal-dual algorithm for structured convex optimization
• Fused Lasso penalized least absolute deviation estimator for high dimensional linear regression
• Detection of block-exchangeable structure in large-scale correlation matrices
• An efficient algorithm for sparse inverse covariance matrix estimation based on dual formulation
• A customized proximal point algorithm for stable principal component pursuit with nonnegative constraint
• Robust Subspace Discovery via Relaxed Rank Minimization
• Alternating direction based method for optimal control problem constrained by Stokes equation
• Sparse estimation of high-dimensional inverse covariance matrices with explicit eigenvalue constraints
• Title not available (Why is that?)
• Estimation of graphical models: an overview of selected topics
• Estimating finite mixtures of ordinal graphical models
• Alternating direction method for covariance selection models
• Characterizing brain connectivity from human electrocorticography recordings with unobserved inputs during epileptic seizures
• Robust Causal Structure Learning with Some Hidden Variables
• Nonparametric Finite Mixture of Gaussian Graphical Models
• Alternating proximal gradient method for convex minimization
• Graphical model selection for Gaussian conditional random fields in the presence of latent variables
• Latent variable graphical model selection via convex optimization
• A distributed Douglas-Rachford splitting method for multi-block convex minimization problems
• Diagonally Dominant Principal Component Analysis
• On the penalized maximum likelihood estimation of high-dimensional approximate factor model
• Estimation of graphical models through structured norm minimization
• Decomposition into low-rank plus additive matrices for background/foreground separation: a review for a comparative evaluation with a large-scale dataset
• Graphical model selection with latent variables
• Sensitivity analysis of the proximal-based parallel decomposition methods

Uses Software

• glasso
• RecPF
• MIM

This page was built for publication: Alternating Direction Methods for Latent Variable Gaussian Graphical Model Selection

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5378251)