Regularized M-estimators with nonconvexity: statistical and algorithmic theory for local optima
From MaRDI portal
(Redirected from Publication:5502126)
Regularized \(M\)-estimators with nonconvexity: statistical and algorithmic theory for local optima
Regularized \(M\)-estimators with nonconvexity: statistical and algorithmic theory for local optima
Abstract: We provide novel theoretical results regarding local optima of regularized -estimators, allowing for nonconvexity in both loss and penalty functions. Under restricted strong convexity on the loss and suitable regularity conditions on the penalty, we prove that emph{any stationary point} of the composite objective function will lie within statistical precision of the underlying parameter vector. Our theory covers many nonconvex objective functions of interest, including the corrected Lasso for errors-in-variables linear models; regression for generalized linear models with nonconvex penalties such as SCAD, MCP, and capped-; and high-dimensional graphical model estimation. We quantify statistical accuracy by providing bounds on the -, -, and prediction error between stationary points and the population-level optimum. We also propose a simple modification of composite gradient descent that may be used to obtain a near-global optimum within statistical precision in steps, which is the fastest possible rate of any first-order method. We provide simulation studies illustrating the sharpness of our theoretical results.
Recommendations
- Sparse recovery via nonconvex regularized \(M\)-estimators over \(\ell_q\)-balls
- Optimal computational and statistical rates of convergence for sparse nonconvex learning problems
- Statistical consistency and asymptotic normality for high-dimensional robust \(M\)-estimators
- Support recovery without incoherence: a case for nonconvex regularization
- I-LAMM for sparse learning: simultaneous control of algorithmic complexity and statistical error
Cited in
(only showing first 100 items - show all)- A general family of trimmed estimators for robust high-dimensional data analysis
- Distributed testing and estimation under sparse high dimensional models
- Folded concave penalized sparse linear regression: sparsity, statistical performance, and algorithmic theory for local solutions
- Pathwise coordinate optimization for sparse learning: algorithm and theory
- I-LAMM for sparse learning: simultaneous control of algorithmic complexity and statistical error
- Broken adaptive ridge regression and its asymptotic properties
- ROCKET: robust confidence intervals via Kendall's tau for transelliptical graphical models
- Restricted strong convexity implies weak submodularity
- The landscape of empirical risk for nonconvex losses
- Rate optimal estimation and confidence intervals for high-dimensional regression with missing covariates
- Statistical consistency and asymptotic normality for high-dimensional robust \(M\)-estimators
- Consistency bounds and support recovery of d-stationary solutions of sparse sample average approximations
- Asymptotic properties on high-dimensional multivariate regression M-estimation
- Minimum average variance estimation with group Lasso for the multivariate response central mean subspace
- A unified primal dual active set algorithm for nonconvex sparse recovery
- Bi-selection in the high-dimensional additive hazards regression model
- Graphical-model based high dimensional generalized linear models
- Iteratively reweighted _1-penalized robust regression
- An outer-inner linearization method for non-convex and nondifferentiable composite regularization problems
- The cost of privacy: optimal rates of convergence for parameter estimation with differential privacy
- Wavelet-based robust estimation and variable selection in nonparametric additive models
- Sparse classification: a scalable discrete optimization perspective
- Analysis of generalized Bregman surrogate algorithms for nonsmooth nonconvex statistical learning
- The finite sample properties of sparse M-estimators with pseudo-observations
- Asymptotic linear expansion of regularized M-estimators
- Nonregular and minimax estimation of individualized thresholds in high dimension with binary responses
- Penalized wavelet estimation and robust denoising for irregular spaced data
- High-dimensional linear regression with hard thresholding regularization: theory and algorithm
- On an extension of the promotion time cure model
- Robust estimation and shrinkage in ultrahigh dimensional expectile regression with heavy tails and variance heterogeneity
- A high-dimensional M-estimator framework for bi-level variable selection
- GSDAR: a fast Newton algorithm for \(\ell_0\) regularized generalized linear models with statistical guarantee
- On two recent nonconvex penalties for regularization in machine learning
- Bias versus non-convexity in compressed sensing
- Nonconvex regularization for sparse neural networks
- Matrix completion with nonconvex regularization: spectral operators and scalable algorithms
- Statistical analysis of sparse approximate factor models
- Lower bounds for finding stationary points I
- Finite-sample analysis of \(M\)-estimators using self-concordance
- Sorted concave penalized regression
- Perspective maximum likelihood-type estimation via proximal decomposition
- Non-concave penalization in linear mixed-effect models and regularized selection of fixed effects
- Computational and statistical analyses for robust non-convex sparse regularized regression problem
- Sample average approximation with sparsity-inducing penalty for high-dimensional stochastic programming
- Optimality condition and complexity analysis for linearly-constrained optimization without differentiability on the boundary
- Misspecified nonconvex statistical optimization for sparse phase retrieval
- Stability of the minimizers of least squares with a non-convex regularization. II: Global behavior
- Endogeneity in high dimensions
- Penalised robust estimators for sparse and high-dimensional linear models
- Sparse estimation via lower-order penalty optimization methods in high-dimensional linear regression
- A primal and dual active set algorithm for truncated \(L_1\) regularized logistic regression
- Global solutions to folded concave penalized nonconvex learning
- Minimum distance Lasso for robust high-dimensional regression
- Non-local estimators: a new class of multigrid convergent length estimators
- Building a telescope to look into high-dimensional image spaces
- scientific article; zbMATH DE number 4176228 (Why is no real title available?)
- Oracle inequalities for local and global empirical risk minimizers
- Adaptive Huber Regression
- scientific article; zbMATH DE number 6982301 (Why is no real title available?)
- Efficient learning with a family of nonconvex regularizers by redistributing nonconvexity
- A tight bound of hard thresholding
- Composite difference-MAX programs for modern statistical estimation problems
- Accelerated methods for nonconvex optimization
- On semiparametric exponential family graphical models
- Hard thresholding regression
- Non-convex projected gradient descent for generalized low-rank tensor regression
- Estimation of a low-rank topic-based model for information cascades
- Hypothesis testing in large-scale functional linear regression
- A New Principle for Tuning-Free Huber Regression
- Bayesian Estimation of Gaussian Conditional Random Fields
- Variable Selection With Second-Generation P-Values
- High-Dimensional Learning Under Approximate Sparsity with Applications to Nonsmooth Estimation and Regularized Neural Networks
- Hard thresholding regularised logistic regression: theory and algorithms
- An improved algorithm for high-dimensional continuous threshold expectile model with variance heterogeneity
- Robustness and Tractability for Non-convex M-estimators
- Learning Markov models via low-rank optimization
- Targeted random projection for prediction from high-dimensional features
- Convex and non-convex approaches for statistical inference with class-conditional noisy labels
- Optimal prediction for sparse linear models? Lower bounds for coordinate-separable M-estimators
- Model-free nonconvex matrix completion: local minima analysis and applications in memory-efficient kernel PCA
- Bayesian regularization for graphical models with unequal shrinkage
- Multiparameter Regularization for Construction of Extrapolating Estimators in Statistical Learning Theory
- The factor-Lasso and \(k\)-step bootstrap approach for inference in high-dimensional economic applications
- Asymptotic Properties of Stationary Solutions of Coupled Nonconvex Nonsmooth Empirical Risk Minimization
- The robust desparsified lasso and the focused information criterion for high-dimensional generalized linear models
- On the finite-sample analysis of -estimators
- On the finite-sample analysis of \(\Theta\)-estimators
- Rejoinder
- High-dimensional rank-based graphical models for non-Gaussian functional data
- On high-dimensional Poisson models with measurement error: hypothesis testing for nonlinear nonconvex optimization
- Zero-norm regularized problems: equivalent surrogates, proximal MM method and statistical error bound
- Byzantine-robust distributed sparse learning for M-estimation
- Penalized Estimation of Frailty-Based Illness–Death Models for Semi-Competing Risks
- Statistical Inference, Learning and Models in Big Data
- Penalized wavelet nonparametric univariate logistic regression for irregular spaced data
- Sparse estimation in high-dimensional linear errors-in-variables regression via a covariate relaxation method
- Sparse Laplacian shrinkage for nonparametric transformation survival model
- Model-Assisted Uniformly Honest Inference for Optimal Treatment Regimes in High Dimension
- Sparse precision matrix estimation with missing observations
- A convex-Nonconvex strategy for grouped variable selection
This page was built for publication: Regularized \(M\)-estimators with nonconvexity: statistical and algorithmic theory for local optima
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5502126)
