Strong oracle optimality of folded concave penalized estimation
From MaRDI portal
Abstract: Folded concave penalization methods have been shown to enjoy the strong oracle property for high-dimensional sparse estimation. However, a folded concave penalization problem usually has multiple local solutions and the oracle property is established only for one of the unknown local solutions. A challenging fundamental issue still remains that it is not clear whether the local optimum computed by a given optimization algorithm possesses those nice theoretical properties. To close this important theoretical gap in over a decade, we provide a unified theory to show explicitly how to obtain the oracle solution via the local linear approximation algorithm. For a folded concave penalized estimation problem, we show that as long as the problem is localizable and the oracle estimator is well behaved, we can obtain the oracle estimator by using the one-step local linear approximation. In addition, once the oracle estimator is obtained, the local linear approximation algorithm converges, namely it produces the same estimator in the next iteration. The general theory is demonstrated by using four classical sparse estimation problems, that is, sparse linear regression, sparse logistic regression, sparse precision matrix estimation and sparse quantile regression.
Recommendations
- Correction: Strong oracle optimality of folded concave penalized estimation
- On the strong oracle property of concave penalized estimators with infinite penalty derivative at the origin
- Sharp Oracle Inequalities for Stationary Points of Nonconvex Penalized M-Estimators
- An oracle inequality for regularized risk minimizers with strongly mixing observations
- Global optimality of nonconvex penalized estimators
- Oracle estimation of parametric models under boundary constraints
- Oracle inequalities and adaptive estimation in the convolution structure density model
- Estimation bounds and sharp oracle inequalities of regularized procedures with Lipschitz loss functions
- Asymptotic oracle properties of SCAD-penalized least squares estimators
- On the oracle complexity of smooth strongly convex minimization
Cites work
- \(\ell_1\)-penalized quantile regression in high-dimensional sparse models
- A constrained \(\ell _{1}\) minimization approach to sparse precision matrix estimation
- A general theory of concave regularization for high-dimensional sparse estimation problems
- A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers
- Adaptive robust variable selection
- Analysis of multi-stage convex relaxation for sparse regularization
- Composite quantile regression and the oracle model selection theory
- Estimation and selection via absolute penalized convex minimization and its multistage adaptive applications
- High-dimensional covariance estimation by minimizing \(\ell _{1}\)-penalized log-determinant divergence
- High-dimensional graphs and variable selection with the Lasso
- scientific article; zbMATH DE number 5957408 (Why is no real title available?)
- scientific article; zbMATH DE number 51763 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- Least angle regression. (With discussion)
- Limiting distributions for \(L_1\) regression estimators under general conditions
- Model selection and estimation in the Gaussian graphical model
- Multi-stage convex relaxation for feature selection
- Nearly unbiased variable selection under minimax concave penalty
- Nonconcave penalized composite conditional likelihood estimation of sparse Ising models
- Nonconcave penalized likelihood with a diverging number of parameters.
- Nonconcave Penalized Likelihood With NP-Dimensionality
- On the conditions used to prove oracle results for the Lasso
- One-Step Huber Estimates in the Linear Model
- One-step sparse estimates in nonconcave penalized likelihood models
- Quantile Regression for Analyzing Heterogeneity in Ultra-High Dimension
- Quantile regression.
- Rate minimaxity of the Lasso and Dantzig selector for the \(l_{q}\) loss in \(l_{r}\) balls
- Regression Quantiles
- Regularization for Cox's proportional hazards model with NP-dimensionality
- Regularization of Wavelet Approximations
- Regularized estimation of large covariance matrices
- Simultaneous analysis of Lasso and Dantzig selector
- Some sharp performance bounds for least squares regression with L₁ regularization
- Sparse inverse covariance estimation with the graphical lasso
- SparseNet: coordinate descent with nonconvex penalties
- Sparsistency and rates of convergence in large covariance matrix estimation
- Strong oracle optimality of folded concave penalized estimation
- The Adaptive Lasso and Its Oracle Properties
- The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
- Variable selection in quantile regression
- Variable selection using MM algorithms
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Cited in
(only showing first 100 items - show all)- Are discoveries spurious? Distributions of maximum spurious correlations and their applications
- Degrees of freedom for piecewise Lipschitz estimators
- Variable selection and parameter estimation with the Atan regularization method
- Balanced estimation for high-dimensional measurement error models
- Folded concave penalized sparse linear regression: sparsity, statistical performance, and algorithmic theory for local solutions
- Quantile regression for additive coefficient models in high dimensions
- Asymptotic normality of one-step M-estimators based on non-identically distributed observations
- Asymptotically honest confidence regions for high dimensional parameters by the desparsified conservative Lasso
- High-dimensional grouped folded concave penalized estimation via the LLA algorithm
- Variable selection via generalized SELO-penalized Cox regression models
- On the pervasiveness of difference-convexity in optimization and statistics
- Portal nodes screening for large scale social networks
- Pathwise coordinate optimization for sparse learning: algorithm and theory
- I-LAMM for sparse learning: simultaneous control of algorithmic complexity and statistical error
- Robust variable selection for finite mixture regression models
- High-dimensional inference: confidence intervals, \(p\)-values and R-software \texttt{hdi}
- Broken adaptive ridge regression and its asymptotic properties
- A cubic spline penalty for sparse approximation under tight frame balanced model
- Learning latent variable Gaussian graphical model for biomolecular network with low sample complexity
- Elastic net penalized quantile regression model
- Minimum average variance estimation with group Lasso for the multivariate response central mean subspace
- A unified primal dual active set algorithm for nonconvex sparse recovery
- The de-biased group Lasso estimation for varying coefficient models
- Analysis of generalized Bregman surrogate algorithms for nonsmooth nonconvex statistical learning
- Group penalized quantile regression
- Statistical inference for normal mixtures with unknown number of components
- A convex relaxation framework consisting of a primal-dual alternative algorithm for solving \(\ell_0\) sparsity-induced optimization problems with application to signal recovery based image restoration
- Robust estimation and shrinkage in ultrahigh dimensional expectile regression with heavy tails and variance heterogeneity
- On the strong oracle property of concave penalized estimators with infinite penalty derivative at the origin
- Distributed optimization and statistical learning for large-scale penalized expectile regression
- Estimating finite mixtures of ordinal graphical models
- Bias versus non-convexity in compressed sensing
- An interior stochastic gradient method for a class of non-Lipschitz optimization problems
- A unifying framework of high-dimensional sparse estimation with difference-of-convex (DC) regularizations
- Estimation and variable selection for partial functional linear regression
- Almost sure uniqueness of a global minimum without convexity
- Nonconcave penalized estimation in sparse vector autoregression model
- An efficient non-convex total variation approach for image deblurring and denoising
- Network tail risk estimation in the European banking system
- Sorted concave penalized regression
- On integer and MPCC representability of affine sparsity
- Adaptively weighted group Lasso for semiparametric quantile regression models
- Sample average approximation with sparsity-inducing penalty for high-dimensional stochastic programming
- Optimality condition and complexity analysis for linearly-constrained optimization without differentiability on the boundary
- Correction: Strong oracle optimality of folded concave penalized estimation
- Variable selection and structure identification for varying coefficient Cox models
- Alternating direction method of multipliers for nonconvex fused regression problems
- Strong oracle optimality of folded concave penalized estimation
- QUADRO: a supervised dimension reduction method via Rayleigh quotient optimization
- Robust low transformed multi-rank tensor methods for image alignment
- Linear-step solvability of some folded concave and singly-parametric sparse optimization problems
- Sparse estimation: an MMSE approach
- A necessary condition for the strong oracle property
- Global solutions to folded concave penalized nonconvex learning
- Oracle inequalities, variable selection and uniform inference in high-dimensional correlated random effects panel data models
- Nonconvex-TV based image restoration with impulse noise removal
- Constructing initial estimators in one-step estimation procedures of nonlinear regression
- scientific article; zbMATH DE number 6982301 (Why is no real title available?)
- On semiparametric exponential family graphical models
- Hard thresholding regression
- Approximation hardness for a class of sparse optimization problems
- Optimal computational and statistical rates of convergence for sparse nonconvex learning problems
- Covariate assisted screening and estimation
- Nonbifurcating Phylogenetic Tree Inference via the Adaptive LASSO
- Hypothesis testing in large-scale functional linear regression
- High-Dimensional Learning Under Approximate Sparsity with Applications to Nonsmooth Estimation and Regularized Neural Networks
- Diagonally Dominant Principal Component Analysis
- A proximal dual semismooth Newton method for zero-norm penalized quantile regression estimator
- Asymptotic properties of one-step \(M\)-estimators
- Rates of convergence of the adaptive elastic net and the post-selection procedure in ultra-high dimensional sparse models
- Penalized estimation in finite mixture of ultra-high dimensional regression models
- An unbiased approach to compressed sensing
- Stochastic correlation coefficient ensembles for variable selection
- Targeted random projection for prediction from high-dimensional features
- Mixed-effect time-varying network model and application in brain connectivity analysis
- Ultra-high dimensional single-index quantile regression
- Computation of second-order directional stationary points for group sparse optimization
- High-dimensional linear model selection motivated by multiple testing
- Difference-of-convex learning: directional stationarity, optimality, and sparsity
- Nonlocal robust tensor recovery with nonconvex regularization
- Robust tensor completion: equivalent surrogates, error bounds, and algorithms
- Achieving the oracle property of OEM with nonconvex penalties
- Discussion of ``Estimating structured high-dimensional covariance and precision matrices: optimal rates and adaptive estimation
- Rejoinder of ``Estimating structured high-dimensional covariance and precision matrices: optimal rates and adaptive estimation
- Zero-norm regularized problems: equivalent surrogates, proximal MM method and statistical error bound
- Smoothing neural network for L₀ regularized optimization problem with general convex constraints
- Simultaneous feature selection and outlier detection with optimality guarantees
- Functional Group Bridge for Simultaneous Regression and Support Estimation
- Estimation of banded time-varying precision matrix based on SCAD and group Lasso
- Neural network for a class of sparse optimization with L₀-regularization
- Time-varying forecast combination for high-dimensional data
- Sparse quantile regression
- Identification of microbial features in multivariate regression under false discovery rate control
- Impulse noise removal by using a nonconvex TGV regularizer and nonconvex fidelity
- Comparing solution paths of sparse quadratic minimization with a Stieltjes matrix
- scientific article; zbMATH DE number 7733439 (Why is no real title available?)
- A convex-Nonconvex strategy for grouped variable selection
- High-Dimensional Censored Regression via the Penalized Tobit Likelihood
- The nonparametric Box-Cox model for high-dimensional regression analysis
- Retire: robust expectile regression in high dimensions
This page was built for publication: Strong oracle optimality of folded concave penalized estimation
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2510819)
