Strong oracle optimality of folded concave penalized estimation
From MaRDI portal
Abstract: Folded concave penalization methods have been shown to enjoy the strong oracle property for high-dimensional sparse estimation. However, a folded concave penalization problem usually has multiple local solutions and the oracle property is established only for one of the unknown local solutions. A challenging fundamental issue still remains that it is not clear whether the local optimum computed by a given optimization algorithm possesses those nice theoretical properties. To close this important theoretical gap in over a decade, we provide a unified theory to show explicitly how to obtain the oracle solution via the local linear approximation algorithm. For a folded concave penalized estimation problem, we show that as long as the problem is localizable and the oracle estimator is well behaved, we can obtain the oracle estimator by using the one-step local linear approximation. In addition, once the oracle estimator is obtained, the local linear approximation algorithm converges, namely it produces the same estimator in the next iteration. The general theory is demonstrated by using four classical sparse estimation problems, that is, sparse linear regression, sparse logistic regression, sparse precision matrix estimation and sparse quantile regression.
Recommendations
- Correction: Strong oracle optimality of folded concave penalized estimation
- On the strong oracle property of concave penalized estimators with infinite penalty derivative at the origin
- Sharp Oracle Inequalities for Stationary Points of Nonconvex Penalized M-Estimators
- An oracle inequality for regularized risk minimizers with strongly mixing observations
- Global optimality of nonconvex penalized estimators
- Oracle estimation of parametric models under boundary constraints
- Oracle inequalities and adaptive estimation in the convolution structure density model
- Estimation bounds and sharp oracle inequalities of regularized procedures with Lipschitz loss functions
- Asymptotic oracle properties of SCAD-penalized least squares estimators
- On the oracle complexity of smooth strongly convex minimization
Cites work
- scientific article; zbMATH DE number 5957408 (Why is no real title available?)
- scientific article; zbMATH DE number 51763 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- A constrained \(\ell _{1}\) minimization approach to sparse precision matrix estimation
- A general theory of concave regularization for high-dimensional sparse estimation problems
- A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers
- Adaptive robust variable selection
- Analysis of multi-stage convex relaxation for sparse regularization
- Composite quantile regression and the oracle model selection theory
- Estimation and selection via absolute penalized convex minimization and its multistage adaptive applications
- High-dimensional covariance estimation by minimizing \(\ell _{1}\)-penalized log-determinant divergence
- High-dimensional graphs and variable selection with the Lasso
- Least angle regression. (With discussion)
- Limiting distributions for \(L_1\) regression estimators under general conditions
- Model selection and estimation in the Gaussian graphical model
- Multi-stage convex relaxation for feature selection
- Nearly unbiased variable selection under minimax concave penalty
- Nonconcave Penalized Likelihood With NP-Dimensionality
- Nonconcave penalized composite conditional likelihood estimation of sparse Ising models
- Nonconcave penalized likelihood with a diverging number of parameters.
- On the conditions used to prove oracle results for the Lasso
- One-Step Huber Estimates in the Linear Model
- One-step sparse estimates in nonconcave penalized likelihood models
- Quantile Regression for Analyzing Heterogeneity in Ultra-High Dimension
- Quantile regression.
- Rate minimaxity of the Lasso and Dantzig selector for the \(l_{q}\) loss in \(l_{r}\) balls
- Regression Quantiles
- Regularization for Cox's proportional hazards model with NP-dimensionality
- Regularization of Wavelet Approximations
- Regularized estimation of large covariance matrices
- Simultaneous analysis of Lasso and Dantzig selector
- Some sharp performance bounds for least squares regression with L₁ regularization
- Sparse inverse covariance estimation with the graphical lasso
- SparseNet: coordinate descent with nonconvex penalties
- Sparsistency and rates of convergence in large covariance matrix estimation
- Strong oracle optimality of folded concave penalized estimation
- The Adaptive Lasso and Its Oracle Properties
- The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
- Variable selection in quantile regression
- Variable selection using MM algorithms
- \(\ell_1\)-penalized quantile regression in high-dimensional sparse models
Cited in
(only showing first 100 items - show all)- Discussion of ``Estimating structured high-dimensional covariance and precision matrices: optimal rates and adaptive estimation
- A cubic spline penalty for sparse approximation under tight frame balanced model
- High-dimensional inference: confidence intervals, \(p\)-values and R-software \texttt{hdi}
- Are discoveries spurious? Distributions of maximum spurious correlations and their applications
- Fast optimization methods for high-dimensional row-sparse multivariate quantile linear regression
- Nonparametric augmented probability weighting with sparsity
- Sparse estimation: an MMSE approach
- High-dimensional grouped folded concave penalized estimation via the LLA algorithm
- A unified consensus-based parallel algorithm for high-dimensional regression with combined regularizations
- Parallel ADMM algorithm with Gaussian back substitution for high-dimensional quantile regression and classification
- ADMM for High-Dimensional Sparse Penalized Quantile Regression
- A scalable surrogate L₀ sparse regression method for generalized linear models with applications to large scale data
- Penalty free variable selection for high-dimensional linear models
- Covariate assisted screening and estimation
- Robust tensor completion: equivalent surrogates, error bounds, and algorithms
- An unbiased approach to compressed sensing
- Bias versus non-convexity in compressed sensing
- Sorted concave penalized regression
- Estimating time-varying networks for high-dimensional time series
- Feature-splitting algorithms for ultrahigh dimensional quantile regression
- Simultaneous feature selection and outlier detection with optimality guarantees
- Sparse Convoluted Rank Regression in High Dimensions
- An Interactive Greedy Approach to Group Sparsity in High Dimensions
- Variable selection via generalized SELO-penalized Cox regression models
- On the pervasiveness of difference-convexity in optimization and statistics
- Portal nodes screening for large scale social networks
- Semiparametric expectile regression for high-dimensional heavy-tailed and heterogeneous data
- Balanced estimation for high-dimensional measurement error models
- High-Dimensional Censored Regression via the Penalized Tobit Likelihood
- Retire: robust expectile regression in high dimensions
- The nonparametric Box-Cox model for high-dimensional regression analysis
- Multi-Task Learning with High-Dimensional Noisy Images
- Projection Test for Mean Vector in High Dimensions
- Robust High-Dimensional Regression with Coefficient Thresholding and Its Application to Imaging Data Analysis
- Statistical Learning for Individualized Asset Allocation
- Global solutions to folded concave penalized nonconvex learning
- Estimation and variable selection for partial functional linear regression
- Smoothing accelerated proximal gradient method with fast convergence rate for nonsmooth convex optimization beyond differentiability
- Sample average approximation with sparsity-inducing penalty for high-dimensional stochastic programming
- Difference-of-convex learning: directional stationarity, optimality, and sparsity
- Time-varying forecast combination for high-dimensional data
- Pathwise coordinate optimization for sparse learning: algorithm and theory
- On integer and MPCC representability of affine sparsity
- Learning latent variable Gaussian graphical model for biomolecular network with low sample complexity
- Robust estimation and shrinkage in ultrahigh dimensional expectile regression with heavy tails and variance heterogeneity
- An efficient non-convex total variation approach for image deblurring and denoising
- Folded concave penalized sparse linear regression: sparsity, statistical performance, and algorithmic theory for local solutions
- Broken adaptive ridge regression and its asymptotic properties
- On the strong oracle property of concave penalized estimators with infinite penalty derivative at the origin
- One-step sparse ridge estimation with folded concave penalty
- Robust variable selection of linear model under BAR penalty
- I-LAMM for sparse learning: simultaneous control of algorithmic complexity and statistical error
- Robust variable selection for finite mixture regression models
- Cross-Fitted Residual Regression for High-Dimensional Heteroscedasticity Pursuit
- Network tail risk estimation in the European banking system
- Elastic net penalized quantile regression model
- Penalized Mallow’s model averaging
- Zero-norm regularized problems: equivalent surrogates, proximal MM method and statistical error bound
- scientific article; zbMATH DE number 7306908 (Why is no real title available?)
- Identification of microbial features in multivariate regression under false discovery rate control
- Statistical inference for normal mixtures with unknown number of components
- Neural network for a class of sparse optimization with L₀-regularization
- Adaptively weighted group Lasso for semiparametric quantile regression models
- Nonbifurcating Phylogenetic Tree Inference via the Adaptive LASSO
- Nonconvex-TV based image restoration with impulse noise removal
- QUADRO: a supervised dimension reduction method via Rayleigh quotient optimization
- Robust statistics: a selective overview and new directions
- Semiparametric efficient estimation in high-dimensional partial linear regression models
- Group penalized expectile regression
- Functional Group Bridge for Simultaneous Regression and Support Estimation
- Optimality condition and complexity analysis for linearly-constrained optimization without differentiability on the boundary
- A penalized least product relative error loss function based on wavelet decomposition for non-parametric multiplicative additive models
- Oracle inequalities, variable selection and uniform inference in high-dimensional correlated random effects panel data models
- Simultaneous factors selection and fusion of their levels in penalized logistic regression
- Estimating finite mixtures of ordinal graphical models
- A varying coefficient model with matrix valued covariates
- Minimum average variance estimation with group Lasso for the multivariate response central mean subspace
- A unified primal dual active set algorithm for nonconvex sparse recovery
- Degrees of freedom for piecewise Lipschitz estimators
- Penalized estimation in finite mixture of ultra-high dimensional regression models
- Alternating direction method of multipliers for nonconvex fused regression problems
- Understanding Implicit Regularization in Over-Parameterized Single Index Model
- Regularized Linear Programming Discriminant Rule with Folded Concave Penalty for Ultrahigh-Dimensional Data
- Exact recovery in the double sparse model: sufficient and necessary signal conditions
- A convex-Nonconvex strategy for grouped variable selection
- Smoothing neural network for L₀ regularized optimization problem with general convex constraints
- Variable selection and parameter estimation with the Atan regularization method
- Sparse quantile regression
- Estimation of banded time-varying precision matrix based on SCAD and group Lasso
- Asset splitting algorithm for ultrahigh dimensional portfolio selection and its theoretical property
- A simple method for estimating Gaussian graphical models
- Nonparametric Finite Mixture of Gaussian Graphical Models
- Deep tobit model: an integrated framework for high-dimensional censored regression with variable selection
- Communication-efficient and distributed-oracle estimation for high-dimensional quantile regression
- An interior stochastic gradient method for a class of non-Lipschitz optimization problems
- A unifying framework of high-dimensional sparse estimation with difference-of-convex (DC) regularizations
- Impulse noise removal by using a nonconvex TGV regularizer and nonconvex fidelity
- Almost sure uniqueness of a global minimum without convexity
- Robust low transformed multi-rank tensor methods for image alignment
- Mixed-effect time-varying network model and application in brain connectivity analysis
This page was built for publication: Strong oracle optimality of folded concave penalized estimation
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2510819)
