Latent variable graphical model selection via convex optimization

DOI10.1214/11-AOS949zbMATH Open1257.62061arXiv1008.1290MaRDI QIDQ132216FDOQ132216

Authors: Venkat Chandrasekaran, Pablo A. Parrilo, Alan S. Willsky, Venkat Chandrasekaran, Pablo A. Parrilo, Alan S. Willsky 

Publication date: 1 August 2012

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: Suppose we observe samples of a subset of a collection of random variables. No additional information is provided about the number of latent variables, nor of the relationship between the latent and observed variables. Is it possible to discover the number of latent components, and to learn a statistical model over the entire collection of variables? We address this question in the setting in which the latent and observed variables are jointly Gaussian, with the conditional statistics of the observed variables conditioned on the latent variables being specified by a graphical model. As a first step we give natural conditions under which such latent-variable Gaussian graphical models are identifiable given marginal statistics of only the observed variables. Essentially these conditions require that the conditional graphical model among the observed variables is sparse, while the effect of the latent variables is "spread out" over most of the observed variables. Next we propose a tractable convex program based on regularized maximum-likelihood for model selection in this latent-variable setting; the regularizer uses both the $el{l}_1$ norm and the nuclear norm. Our modeling framework can be viewed as a combination of dimensionality reduction (to identify latent variables) and graphical modeling (to capture remaining statistical structure not attributable to the latent variables), and it consistently estimates both the number of latent components and the conditional graphical model structure among the observed variables. These results are applicable in the high-dimensional setting in which the number of latent/observed variables grows with the number of samples of the observed variables. The geometric properties of the algebraic varieties of sparse matrices and of low-rank matrices play an important role in our analysis.

Full work available at URL: https://arxiv.org/abs/1008.1290

Recommendations

• Graphical model selection for Gaussian conditional random fields in the presence of latent variables
• Graphical model selection with latent variables
• Learning loopy graphical models with latent variables: efficient methods and guarantees
• Alternating Direction Methods for Latent Variable Gaussian Graphical Model Selection
• Interpreting latent variables in factor models via convex optimization

zbMATH Keywords

regularizationcovariance selectionGaussian graphical modelslatent variablessparsityalgebraic statisticshigh-dimensional asymptoticslow-rank

Mathematics Subject Classification ID

Asymptotic properties of parametric estimators (62F12) Convex programming (90C25) Applications of graph theory (05C90) Estimation in multivariate analysis (62H12)

Cites Work

• SDPT3 — A Matlab software package for semidefinite programming, Version 1.3
• Title not available (Why is that?)
• Title not available (Why is that?)
• High-dimensional graphs and variable selection with the Lasso
• High dimensional covariance matrix estimation using a factor model
• Title not available (Why is that?)
• Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$-Constrained Quadratic Programming (Lasso)
• Covariance regularization by thresholding
• Sparsistency and rates of convergence in large covariance matrix estimation
• A well-conditioned estimator for large-dimensional covariance matrices
• On the distribution of the largest eigenvalue in principal components analysis
• Sparse permutation invariant covariance estimation
• Regularized estimation of large covariance matrices
• Robust principal component analysis?
• Title not available (Why is that?)
• Exact matrix completion via convex optimization
• High-dimensional covariance estimation by minimizing \(\ell _{1}\)-penalized log-determinant divergence
• Local operator theory, random matrices and Banach spaces.
• Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization
• Operator norm consistent estimation of large-dimensional sparse covariance matrices
• Gaussian Markov distributions over finite graphs
• Characterization of the subdifferential of some matrix norms
• Rejoinder: Latent variable graphical model selection via convex optimization
• Nonparametric estimation of large covariance matrices of longitudinal data
• Solving log-determinant optimization problems by a Newton-CG primal proximal point algorithm
• Rank-Sparsity Incoherence for Matrix Decomposition
• ``Ideal parent structure learning for continuous variable Bayesian networks
• Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information
• Equivalent Subgradient Versions of Hamiltonian and Euler–Lagrange Equations in Variational Analysis
• For most large underdetermined systems of linear equations the minimal 𝓁₁‐norm solution is also the sparsest solution
• Title not available (Why is that?)
• Title not available (Why is that?)
• Compressed sensing

Cited In (only showing first 100 items - show all)

• High-dimensional inference: confidence intervals, \(p\)-values and R-software \texttt{hdi}
• Learning a factor model via regularized PCA
• Forward-backward splitting with Bregman distances
• Two-stage convex relaxation approach to low-rank and sparsity regularized least squares loss
• Learning latent tree graphical models
• A multi-stage convex relaxation approach to noisy structured low-rank matrix recovery
• Generalized network psychometrics: combining network and latent variable models
• Learning semidefinite regularizers
• Regularized estimation of high-dimensional factor-augmented vector autoregressive (FAVAR) models
• General parameterized proximal point algorithm with applications in statistical learning
• Adjusted regularization in latent graphical models: application to multiple-neuron spike count data
• Adjusted regularization of cortical covariance
• Improved proximal ADMM with partially parallel splitting for multi-block separable convex programming
• Linearized alternating direction method with parallel splitting and adaptive penalty for separable convex programs in machine learning
• Rejoinder: Latent variable graphical model selection via convex optimization
• Learning latent variable Gaussian graphical model for biomolecular network with low sample complexity
• Adaptive estimation in structured factor models with applications to overlapping clustering
• A parallel splitting ALM-based algorithm for separable convex programming
• Sparse linear identifiable multivariate modeling
• Generalized symmetric ADMM for separable convex optimization
• Detection of block-exchangeable structure in large-scale correlation matrices
• Linear estimating equations for exponential families with application to Gaussian linear concentration models
• Alternating direction based method for optimal control problem constrained by Stokes equation
• Convergence of ADMM for Three-Block Separable Quadratic Programming Problems with Linear Constraints
• On model selection consistency of regularized M-estimators
• A derivative-free \textit{RMIL} conjugate gradient projection method for convex constrained nonlinear monotone equations with applications in compressive sensing
• An algorithm twisted from generalized ADMM for multi-block separable convex minimization models
• On the global and linear convergence of direct extension of ADMM for 3-block separable convex minimization models
• Scalable robust matrix recovery: Frank-Wolfe meets proximal methods
• Toward a multisubject analysis of neural connectivity
• Level-set methods for convex optimization
• Estimating structured high-dimensional covariance and precision matrices: optimal rates and adaptive estimation
• Nonstationary Modeling With Sparsity for Spatial Data via the Basis Graphical Lasso
• Recent developments in high dimensional covariance estimation and its related issues, a review
• A class of ADMM-based algorithms for three-block separable convex programming
• Alternating proximal gradient method for convex minimization
• A partially isochronous splitting algorithm for three-block separable convex minimization problems
• Psychometric network models from time-series and panel data
• Dimension-wise sparse low-rank approximation of a matrix with application to variable selection in high-dimensional integrative analyzes of association
• Large covariance estimation through elliptical factor models
• Bridging convex and nonconvex optimization in robust PCA: noise, outliers and missing data
• Asymptotic normality and optimalities in estimation of large Gaussian graphical models
• Graphical model selection for Gaussian conditional random fields in the presence of latent variables
• Parallel multi-block ADMM with \(o(1/k)\) convergence
• Comment on ``Hypothesis testing by convex optimization
• Learning loopy graphical models with latent variables: efficient methods and guarantees
• Bi-cross-validation for factor analysis
• Proximal Markov chain Monte Carlo algorithms
• The convergence rate of the proximal alternating direction method of multipliers with indefinite proximal regularization
• Adaptive estimation of the copula correlation matrix for semiparametric elliptical copulas
• latentgraph
• Pivotal variable detection of the covariance matrix and its application to high-dimensional factor models
• Alternating Direction Methods for Latent Variable Gaussian Graphical Model Selection
• On the convergence of the direct extension of ADMM for three-block separable convex minimization models with one strongly convex function
• Rejoinder: Invariance, causality and robustness
• Doubly debiased Lasso: high-dimensional inference under hidden confounding
• Estimation of graphical models through structured norm minimization
• Decomposition into low-rank plus additive matrices for background/foreground separation: a review for a comparative evaluation with a large-scale dataset
• The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent
• Interpreting latent variables in factor models via convex optimization
• Graphical model selection with latent variables
• Tightness of the maximum likelihood semidefinite relaxation for angular synchronization
• Joint Gaussian graphical model estimation: a survey
• Structure learning for zero-inflated counts with an application to single-cell RNA sequencing data
• Nonlinear Causal Discovery with Confounders
• An Algebraic Estimator for Large Spectral Density Matrices
• Estimating a covariance matrix for market risk management and the case of credit default swaps
• Bayesian sparse covariance decomposition with a graphical structure
• High-dimensional covariance matrix estimation
• Stable local-smooth principal component pursuit
• Adaptive estimation in multivariate response regression with hidden variables
• Detection of hubs in complex networks by the Laplacian matrix
• Learning linear non-Gaussian causal models in the presence of latent variables
• Incomplete graphical model inference via latent tree aggregation
• A rank-two relaxed parallel splitting version of the augmented Lagrangian method with step size in (0,2) for separable convex programming
• Latent Network Estimation and Variable Selection for Compositional Data Via Variational EM
• Learning a Bayesian network with multiple latent variables for implicit relation representation
• Estimation of graphical models: an overview of selected topics
• Large factor model estimation by nuclear norm plus \(\ell_1\) norm penalization
• Modified proximal symmetric ADMMs for multi-block separable convex optimization with linear constraints
• Characterizing brain connectivity from human electrocorticography recordings with unobserved inputs during epileptic seizures
• Topology identification under spatially correlated noise
• Robust Causal Structure Learning with Some Hidden Variables
• Nonparametric Finite Mixture of Gaussian Graphical Models
• Nonsparse learning with latent variables
• Inference in High-Dimensional Multivariate Response Regression with Hidden Variables
• Title not available (Why is that?)
• Integrative methods for post-selection inference under convex constraints
• Hidden factor estimation in dynamic generalized factor analysis models
• Interpretation of the precision matrix and its application in estimating sparse brain connectivity during sleep spindles from human electrocorticography recordings
• Pairwise sparse + low-rank models for variables of mixed type
• A large covariance matrix estimator under intermediate spikiness regimes
• Scalable interpretable multi-response regression via SEED
• A proximal alternating direction method of multipliers with a substitution procedure
• Structure learning of Bayesian networks with latent variables via sparse and low-rank decomposition
• Title not available (Why is that?)
• Sparse plus low-rank identification for dynamical latent-variable graphical AR models
• A dual-primal balanced augmented Lagrangian method for linearly constrained convex programming
• Latent-variable Gaussian graphical model structure learning by GEMS algorithm
• Learning Gaussian graphical models with latent confounders

Uses Software

• SDPT3
• YALMIP
• PMA

This page was built for publication: Latent variable graphical model selection via convex optimization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q132216)