A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers
From MaRDI portal
A unified framework for high-dimensional analysis of \(M\)-estimators with decomposable regularizers
Abstract: High-dimensional statistical inference deals with models in which the the number of parameters p is comparable to or larger than the sample size n. Since it is usually impossible to obtain consistent procedures unless , a line of recent work has studied models with various types of low-dimensional structure, including sparse vectors, sparse and structured matrices, low-rank matrices and combinations thereof. In such settings, a general approach to estimation is to solve a regularized optimization problem, which combines a loss function measuring how well the model fits the data with some regularization function that encourages the assumed structure. This paper provides a unified framework for establishing consistency and convergence rates for such regularized M-estimators under high-dimensional scaling. We state one main theorem and show how it can be used to re-derive some existing results, and also to obtain a number of new results on consistency and convergence rates, in both -error and related norms. Our analysis also identifies two key properties of loss and regularization functions, referred to as restricted strong convexity and decomposability, that ensure corresponding regularized M-estimators have fast convergence rates and which are optimal in many well-studied cases.
Recommendations
- On model selection consistency of regularized M-estimators
- Fast global convergence of gradient methods for high-dimensional statistical recovery
- Statistical consistency and asymptotic normality for high-dimensional robust \(M\)-estimators
- The slow, steady ascent of a hot solid sphere in a Newtonian fluid with strongly temperature-dependent viscosity
- The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
Cites work
- A note on the Lasso for Gaussian graphical model selection
- A simpler approach to matrix completion
- A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers
- Aggregation for Gaussian regression
- Algorithms for simultaneous sparse approximation. I: Greedy pursuit
- An overview of recent developments in genomics and associated statistical methods
- Atomic Decomposition by Basis Pursuit
- Blockwise sparse regression
- Characteristic vectors of bordered matrices with infinite dimensions
- Compressed sensing
- Consistency of the group Lasso and multiple kernel learning
- Consistency of trace norm minimization
- Covariance regularization by thresholding
- Decoding by Linear Programming
- Estimation of (near) low-rank matrices with noise and high-dimensional scaling
- Estimation of high-dimensional low-rank matrices
- Exact matrix completion via convex optimization
- Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization
- High-dimensional additive modeling
- High-dimensional covariance estimation by minimizing \(\ell _{1}\)-penalized log-determinant divergence
- High-dimensional generalized linear models and the lasso
- High-dimensional graphs and variable selection with the Lasso
- High-dimensional Ising model selection using \(\ell _{1}\)-regularized logistic regression
- Honest variable selection in linear and logistic regression models via \(\ell _{1}\) and \(\ell _{1}+\ell _{2}\) penalization
- scientific article; zbMATH DE number 5957408 (Why is no real title available?)
- scientific article; zbMATH DE number 51346 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- Information-Theoretic Limits on Sparsity Recovery in the High-Dimensional and Noisy Setting
- Interior-point method for nuclear norm approximation with application to system identification
- Lasso-type recovery of sparse representations for high-dimensional data
- Matrix completion from noisy entries
- Minimax Rates of Estimation for High-Dimensional Linear Regression Over \ell_q-Balls
- Minimax-optimal rates for sparse additive models over kernel classes via convex programming
- Model Selection and Estimation in Regression with Grouped Variables
- Model-Based Compressive Sensing
- Neighborliness of randomly projected simplices in high dimensions
- On the asymptotic properties of the group lasso estimator for linear models
- On the conditions used to prove oracle results for the Lasso
- On the Reconstruction of Block-Sparse Signals With an Optimal Number of Measurements
- Operator norm consistent estimation of large-dimensional sparse covariance matrices
- Optimal rates of convergence for covariance matrix estimation
- Oracle inequalities and optimal inference under group sparsity
- Persistene in high-dimensional linear predictor-selection and the virtue of overparametrization
- Proximal methods for hierarchical sparse coding
- Rank-Sparsity Incoherence for Matrix Decomposition
- Reconstruction From Anisotropic Random Measurements
- Restricted eigenvalue properties for correlated Gaussian designs
- Restricted strong convexity and weighted matrix completion: optimal bounds with noise
- Robust Matrix Decomposition With Sparse Corruptions
- Robust PCA via Outlier Pursuit
- Self-concordant analysis for logistic regression
- Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$-Constrained Quadratic Programming (Lasso)
- Simultaneous analysis of Lasso and Dantzig selector
- Simultaneous Support Recovery in High Dimensions: Benefits and Perils of Block $\ell _{1}/\ell _{\infty} $-Regularization
- Sparse permutation invariant covariance estimation
- Sparsistency and rates of convergence in large covariance matrix estimation
- Sparsity and Smoothness Via the Fused Lasso
- Sparsity in multiple kernel learning
- Sparsity oracle inequalities for the Lasso
- Statistical analysis of observations of increasing dimension. Transl. from the Russian
- Support union recovery in high-dimensional multivariate regression
- The benefit of group sparsity
- The composite absolute penalties family for grouped and hierarchical variable selection
- The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
- The sparsity and bias of the LASSO selection in high-dimensional linear regression
- Two proposals for robust PCA using semidefinite programming
Cited in
(only showing first 100 items - show all)- Transfer Learning under High-dimensional Generalized Linear Models
- Confidence intervals for high-dimensional inverse covariance estimation
- Regularized estimation in sparse high-dimensional time series models
- Model-assisted inference for treatment effects using regularized calibrated estimation with high-dimensional data
- A note on coding and standardization of categorical variables in (sparse) group Lasso regression
- Nonnegative-Lasso and application in index tracking
- A general family of trimmed estimators for robust high-dimensional data analysis
- The slow, steady ascent of a hot solid sphere in a Newtonian fluid with strongly temperature-dependent viscosity
- Distributed testing and estimation under sparse high dimensional models
- Sparse linear models and \(l_1\)-regularized 2SLS with high-dimensional endogenous regressors and instruments
- Scalable methods for Bayesian selective inference
- Trace regression model with simultaneously low rank and row(column) sparse parameter
- Robust shrinkage estimation and selection for functional multiple linear model through LAD loss
- Multivariate factorizable expectile regression with application to fMRI data
- Relaxed sparse eigenvalue conditions for sparse estimation via non-convex regularized regression
- Folded concave penalized sparse linear regression: sparsity, statistical performance, and algorithmic theory for local solutions
- A constrained \(\ell1\) minimization approach for estimating multiple sparse Gaussian or nonparanormal graphical models
- An analysis of the SPARSEVA estimate for the finite sample data case
- High-dimensional grouped folded concave penalized estimation via the LLA algorithm
- Pathwise coordinate optimization for sparse learning: algorithm and theory
- Regularization and the small-ball method. I: Sparse recovery
- I-LAMM for sparse learning: simultaneous control of algorithmic complexity and statistical error
- Semiparametric efficiency bounds for high-dimensional models
- The convex geometry of linear inverse problems
- The Lasso problem and uniqueness
- Restricted strong convexity implies weak submodularity
- The landscape of empirical risk for nonconvex losses
- Consistency bounds and support recovery of d-stationary solutions of sparse sample average approximations
- Asymptotic properties on high-dimensional multivariate regression M-estimation
- Consistent multiple changepoint estimation with fused Gaussian graphical models
- Graphical-model based high dimensional generalized linear models
- Matrix optimization based Euclidean embedding with outliers
- An outer-inner linearization method for non-convex and nondifferentiable composite regularization problems
- The distribution of the Lasso: uniform control over sparse balls and adaptive parameter tuning
- Integrative methods for post-selection inference under convex constraints
- The cost of privacy: optimal rates of convergence for parameter estimation with differential privacy
- Sampling from non-smooth distributions through Langevin diffusion
- Low-rank matrix recovery with composite optimization: good conditioning and rapid convergence
- Inference for high-dimensional varying-coefficient quantile regression
- Sparse regression for extreme values
- The finite sample properties of sparse M-estimators with pseudo-observations
- Asymptotic linear expansion of regularized M-estimators
- Quantile regression feature selection and estimation with grouped variables using Huber approximation
- A Lagrange-Newton algorithm for sparse nonlinear programming
- Nonregular and minimax estimation of individualized thresholds in high dimension with binary responses
- High-performance statistical computing in the computing environments of the 2020s
- On the grouping effect of the \(l_{1-2}\) models
- High dimensional generalized linear models for temporal dependent data
- Fundamental barriers to high-dimensional regression with convex penalties
- Covariate-adjusted inference for differential analysis of high-dimensional networks
- Penalized and constrained LAD estimation in fixed and high dimension
- Testability of high-dimensional linear models with nonsparse structures
- Adaptive log-density estimation
- Doubly robust semiparametric inference using regularized calibrated estimation with high-dimensional data
- Regularized high dimension low tubal-rank tensor regression
- Post-model-selection inference in linear regression models: an integrated review
- Penalized least square in sparse setting with convex penalty and non Gaussian errors
- A data-driven line search rule for support recovery in high-dimensional data analysis
- Gradient projection Newton pursuit for sparsity constrained optimization
- Estimating sparse networks with hubs
- Prediction error after model search
- Robust machine learning by median-of-means: theory and practice
- Lasso guarantees for \(\beta \)-mixing heavy-tailed time series
- A two-step method for estimating high-dimensional Gaussian graphical models
- Statistical analysis of sparse approximate factor models
- Variable selection for sparse logistic regression
- Is distribution-free inference possible for binary regression?
- Model selection for high-dimensional linear regression with dependent observations
- Finite-sample analysis of \(M\)-estimators using self-concordance
- Prediction and estimation consistency of sparse multi-class penalized optimal scoring
- Sorted concave penalized regression
- Recovery of simultaneous low rank and two-way sparse coefficient matrices, a nonconvex approach
- SGL-SVM: a novel method for tumor classification via support vector machine with sparse group lasso
- On rank estimators in increasing dimensions
- Sharp oracle inequalities for low-complexity priors
- Optimality conditions for rank-constrained matrix optimization
- Computational and statistical analyses for robust non-convex sparse regularized regression problem
- Generalized high-dimensional trace regression via nuclear norm regularization
- High-dimensional generalized linear models incorporating graphical structure among predictors
- Sample average approximation with sparsity-inducing penalty for high-dimensional stochastic programming
- Optimality condition and complexity analysis for linearly-constrained optimization without differentiability on the boundary
- Sparsistency and agnostic inference in sparse PCA
- Structured estimation for the nonparametric Cox model
- On model selection consistency of regularized M-estimators
- High dimensional single index models
- A group VISA algorithm for variable selection
- Error bounds for compressed sensing algorithms with group sparsity: A unified approach
- The degrees of freedom of partly smooth regularizers
- A unified penalized method for sparse additive quantile models: an RKHS approach
- Structure learning of sparse directed acyclic graphs incorporating the scale-free property
- Graph-based sparse linear discriminant analysis for high-dimensional classification
- Minimax sparse principal subspace estimation in high dimensions
- Strong oracle optimality of folded concave penalized estimation
- On solutions of sparsity constrained optimization
- Penalised robust estimators for sparse and high-dimensional linear models
- High-dimensional robust regression with \(L_q\)-loss functions
- Sparse estimation via lower-order penalty optimization methods in high-dimensional linear regression
- Worst possible sub-directions in high-dimensional models
- High-dimensional VARs with common factors
- Estimation in high dimensions: a geometric perspective
This page was built for publication: A unified framework for high-dimensional analysis of \(M\)-estimators with decomposable regularizers
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5965308)
