High-dimensional covariance estimation by minimizing _1-penalized log-determinant divergence

DOI10.1214/11-EJS631MaRDI QIDQ1952214zbMATH OpenOpenAlexFDO

Authors Pradeep Ravikumar, Martin J. Wainwright, Garvesh Raskutti, Bin Yu

Publication date 28 May 2013

Published in Electronic Journal of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0811.3628, https://projecteuclid.org/euclid.ejs/1316092865

zbMATH Keywords

covarianceGaussian graphical modelssparsityconcentrationprecision\(\ell_1\) regularization

Mathematics Subject Classification ID

Asymptotic properties of parametric estimators (62F12) Parametric inference under constraints (62F30)

Abstract: Given i.i.d. observations of a random vector $Xinmathbb{R}^p$, we study the problem of estimating both its covariance matrix $Sigm{a}^{\ast }$, and its inverse covariance or concentration matrix {$Thet{a}^{\ast }=(Sigm{a}^{\ast }{)}^{-1}$.} We estimate $Thet{a}^{\ast }$ by minimizing an $el{l}_1$-penalized log-determinant Bregman divergence; in the multivariate Gaussian case, this approach corresponds to $el{l}_1$-penalized maximum likelihood, and the structure of $Thet{a}^{\ast }$ is specified by the graph of an associated Gaussian Markov random field. We analyze the performance of this estimator under high-dimensional scaling, in which the number of nodes in the graph $p$, the number of edges $s$ and the maximum node degree $d$, are allowed to grow as a function of the sample size $n$. In addition to the parameters $(p,s,d)$, our analysis identifies other key quantities covariance matrix $Sigm{a}^{\ast }$; and (b) the $el{l}_{i}nfty$ operator norm of the sub-matrix $Gamm{a}_{SS}^{\ast }$, where $S$ indexes the graph edges, and $Gamm{a}^{\ast }=(Thet{a}^{\ast }{)}^{-1}otimes(Thet{a}^{\ast }{)}^{-1}$; and (c) a mutual incoherence or irrepresentability measure on the matrix $Gamm{a}^{\ast }$ and (d) the rate of decay $1/f(n,delta)$ on the probabilities $|hat{Sigma}_{ij}^n-Sigm{a}_{ij}^{\ast }|>delta$, where $hat{Sigma}^n$ is the sample covariance based on $n$ samples. Our first result establishes consistency of our estimate $hatTheta$ in the elementwise maximum-norm. This in turn allows us to derive convergence rates in Frobenius and spectral norms, with improvements upon existing results for graphs with maximum node degrees $d=o(sqrts)$. In our second result, we show that with probability converging to one, the estimate $hatTheta$ correctly specifies the zero pattern of the concentration matrix $Thet{a}^{\ast }$.

Recommendations

• High-dimensional covariance estimation based on Gaussian graphical models
• Sparsistency and rates of convergence in large covariance matrix estimation
• Sparse estimation of a covariance matrix
• A constrained \(\ell _{1}\) minimization approach to sparse precision matrix estimation
• High dimensional inverse covariance matrix estimation via linear programming

Cites work

• scientific article; zbMATH DE number 5957408 (Why is no real title available?)
• scientific article; zbMATH DE number 44577 (Why is no real title available?)
• scientific article; zbMATH DE number 741240 (Why is no real title available?)
• scientific article; zbMATH DE number 2107836 (Why is no real title available?)
• scientific article; zbMATH DE number 1433619 (Why is no real title available?)
• scientific article; zbMATH DE number 3296905 (Why is no real title available?)
• scientific article; zbMATH DE number 3381785 (Why is no real title available?)
• A note on the Lasso for Gaussian graphical model selection
• A well-conditioned estimator for large-dimensional covariance matrices
• Covariance matrix selection and estimation via penalised normal likelihood
• Covariance regularization by thresholding
• Estimation of Gaussian graphs by model selection
• Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter vari\-ants
• First-Order Methods for Sparse Covariance Selection
• High-dimensional covariance estimation by minimizing \(\ell _{1}\)-penalized log-determinant divergence
• High-dimensional graphs and variable selection with the Lasso
• Just relax: convex programming methods for identifying sparse signals in noise
• Matrix Analysis
• Model selection and estimation in the Gaussian graphical model
• Nonparametric estimation of large covariance matrices of longitudinal data
• On the distribution of the largest eigenvalue in principal components analysis
• On the subspaces of \(L^p\) \((p > 2)\) spanned by sequences of independent random variables
• Operator norm consistent estimation of large-dimensional sparse covariance matrices
• Optimal rates of convergence for covariance matrix estimation
• Regularized estimation of large covariance matrices
• Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$-Constrained Quadratic Programming (Lasso)
• Sparse inverse covariance estimation with the graphical lasso
• Sparse permutation invariant covariance estimation
• Sparsistency and rates of convergence in large covariance matrix estimation
• The concentration of measure phenomenon

Cited in

(only showing first 100 items - show all)

• De-noising analysis of noisy data under mixed graphical models
• Adjusting for high-dimensional covariates in sparse precision matrix estimation by _1-penalization
• Prediction in abundant high-dimensional linear regression
• Graphical Models and Message-Passing Algorithms: Some Introductory Lectures
• Exact camera location recovery by least unsquared deviations
• High-dimensional graphical inference via partially penalised regression
• Change-point detection in high-dimensional covariance structure
• Confidence intervals for sparse precision matrix estimation via Lasso penalized D-trace loss
• A significance test for covariates in nonparametric regression
• Learning distribution-free anchored linear structural equation models in the presence of measurement error
• Learning a factor model via regularized PCA
• Local Whittle estimation of high-dimensional long-run variance and precision matrices
• Joint Mean and Covariance Estimation with Unreplicated Matrix-Variate Data
• Multivariate sparse Laplacian shrinkage for joint estimation of two graphical structures
• Stochastic approximation: from statistical origin to big-data, multidisciplinary applications
• A self-calibrated direct approach to precision matrix estimation and linear discriminant analysis in high dimensions
• Learning high-dimensional Gaussian linear structural equation models with heterogeneous error variances
• Edge selection for undirected graphs
• High-dimensional consistency in score-based and hybrid structure learning
• Optimal rates of convergence for covariance matrix estimation
• Region selection in Markov random fields: Gaussian case
• Hypothesis Testing of Matrix Graph Model with Application to Brain Connectivity Analysis
• Total positivity in multivariate extremes
• Penalized interaction estimation for ultrahigh dimensional quadratic regression
• Joint semiparametric kernel network regression
• Functional Graphical Models
• Dependence in elliptical partial correlation graphs
• Combinatorial inference for graphical models
• Group symmetry and covariance regularization
• High dimensional sparse covariance estimation via directed acyclic graphs
• Federated sufficient dimension reduction through high-dimensional sparse sliced inverse regression
• Block-diagonal precision matrix regularization for ultra-high dimensional data
• A general family of trimmed estimators for robust high-dimensional data analysis
• A constrained \(\ell _{1}\) minimization approach to sparse precision matrix estimation
• Robust covariance estimation for approximate factor models
• An extreme-value approach for testing the equality of large U-statistic based correlation matrices
• Berry-Esseen bounds for estimating undirected graphs
• Extremal graphical modeling with latent variables via convex optimization
• Sparse directed acyclic graphs incorporating the covariates
• Layer-wise learning strategy for nonparametric tensor product smoothing spline regression and graphical models
• Road traffic estimation and algorithmic routing in a spatially dependent network
• Discussion: Latent variable graphical model selection via convex optimization
• High-dimensional covariance estimation by minimizing \(\ell _{1}\)-penalized log-determinant divergence
• Time-varying minimum variance portfolio
• An Additive Graphical Model for Discrete Data
• Regularity properties for sparse regression
• High-dimensional robust precision matrix estimation: cellwise corruption under \(\epsilon \)-contamination
• Tuning-free heterogeneous inference in massive networks
• Efficient estimation of approximate factor models via penalized maximum likelihood
• The cluster graphical Lasso for improved estimation of Gaussian graphical models
• Adjusted regularization in latent graphical models: application to multiple-neuron spike count data
• Structure learning of exponential family graphical model with false discovery rate control
• Estimating Time-Varying Graphical Models
• The finite sample properties of sparse M-estimators with pseudo-observations
• An exponential inequality for U-statistics under mixing conditions
• Rejoinder: Latent variable graphical model selection via convex optimization
• Learning latent variable Gaussian graphical model for biomolecular network with low sample complexity
• Distributionally robust inverse covariance estimation: the Wasserstein shrinkage estimator
• Time varying undirected graphs
• The masked sample covariance estimator: an analysis using matrix concentration inequalities
• Simultaneous inference for pairwise graphical models with generalized score matching
• High-dimensional covariance matrix estimation
• Robust inference with knockoffs
• Discussion: Latent variable graphical model selection via convex optimization
• An efficient algorithm for sparse inverse covariance matrix estimation based on dual formulation
• Detection of hubs in complex networks by the Laplacian matrix
• Gaussian and robust Kronecker product covariance estimation: existence and uniqueness
• High-dimensional joint estimation of multiple directed Gaussian graphical models
• A two-step method for estimating high-dimensional Gaussian graphical models
• Coordinatewise Gaussianization: Theories and Applications
• Causal Structural Learning via Local Graphs
• An approach to precision matrix estimation based on L₁ norm minimization
• Nonparametric Functional Graphical Modeling Through Functional Additive Regression Operator
• Discussion: Latent variable graphical model selection via convex optimization
• Reproducible learning of Gaussian graphical models via graphical Lasso multiple data splitting
• Gemini: graph estimation with matrix variate normal instances
• Asymptotic theory of _1-regularized PDE identification from a single noisy trajectory
• Discussion: Latent variable graphical model selection via convex optimization
• Learning quadratic variance function (QVF) DAG models via overdispersion scoring (ODS)
• The reduced PC-algorithm: improved causal structure learning in large random networks
• Model selection and estimation in the matrix normal graphical model
• Generalized score matching for non-negative data
• A convex pseudolikelihood framework for high dimensional partial correlation estimation with convergence guarantees
• Bayesian regularization for graphical models with unequal shrinkage
• High-dimensional Poisson structural equation model learning via _1-regularized regression
• Structure estimation for discrete graphical models: generalized covariance matrices and their inverses
• Identifiability of additive noise models using conditional variances
• An efficient ADMM algorithm for high dimensional precision matrix estimation via penalized quadratic loss
• An efficiency upper bound for inverse covariance estimation
• scientific article; zbMATH DE number 7307482 (Why is no real title available?)
• Support recovery of Gaussian graphical model with false discovery rate control
• High-dimensional undirected graphical models for arbitrary mixed data
• Regularized estimation and testing for high-dimensional multi-block vector-autoregressive models
• Optimal estimation and rank detection for sparse spiked covariance matrices
• Asymptotic post-selection inference for regularized graphical models
• Estimation of positive definite M-matrices and structure learning for attractive Gaussian Markov random fields
• Joint estimation of precision matrices in heterogeneous populations
• Covariance and precision matrix estimation for high-dimensional time series
• A partial graphical model with a structural prior on the direct links between predictors and responses
• High-dimensional inference for cluster-based graphical models

Describes a project that uses

Uses Software

• glasso

This page was built for publication: High-dimensional covariance estimation by minimizing \(\ell _{1}\)-penalized log-determinant divergence

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1952214)