Sparse estimation by exponential weighting

DOI10.1214/12-STS393zbMath1331.62351arXiv1108.5116MaRDI QIDQ5965309

Philippe Rigollet, Alexandre B. Tsybakov

Publication date: 3 March 2016

Published in: Statistical Science (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1108.5116

zbMATH Keywords

exponential weightssparsityhigh-dimensional regressionfused sparsitygroup sparsitysparse regressionsparsity oracle inequalitiessparsity pattern aggregationsparsity prior

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Ridge regression; shrinkage estimators (Lasso) (62J07) Bayesian inference (62F15)

Related Items

On the prediction loss of the Lasso in the partially labeled setting, Bayesian inference in high-dimensional linear models using an empirical correlation-adaptive prior, Entropic optimal transport is maximum-likelihood deconvolution, Solution of linear ill-posed problems by model selection and aggregation, Unnamed Item, Generalized aggregation of misspecified models: with an application to asset pricing, Ordered smoothers with exponential weighting, Exponential weights in multivariate regression and a low-rankness favoring prior, Bayesian linear regression with sparse priors, PAC-Bayesian risk bounds for group-analysis sparse regression by exponential weighting, Aggregation of predictors for nonstationary sub-linear processes and online adaptive forecasting of time varying autoregressive processes, An adaptive multiclass nearest neighbor classifier, On estimation of isotonic piecewise constant signals, Synthetic learner: model-free inference on treatments over time, Metamodel construction for sensitivity analysis, PAC-Bayesian estimation and prediction in sparse additive models, Optimal model selection in heteroscedastic regression using piecewise polynomial functions, Comparing and Weighting Imperfect Models Using D-Probabilities, A general framework for Bayes structured linear models, Concentration inequalities for the exponential weighting method, Aggregation of affine estimators, Estimation and variable selection with exponential weights, Statistical inference in compound functional models, Optimal learning with \textit{Q}-aggregation, Optimal bounds for aggregation of affine estimators, Introduction to the special issue on sparsity and regularization methods, On risk bounds in isotonic and other shape restricted regression problems, Global risk bounds and adaptation in univariate convex regression, Second-order Stein: SURE for SURE and other applications in high-dimensional inference, Sharp oracle inequalities for low-complexity priors, Non-parametric Poisson regression from independent and weakly dependent observations by model selection, Localized Gaussian width of \(M\)-convex hulls with applications to Lasso and convex aggregation, Multiscale Change Point Inference

Uses Software

• glmnet

Cites Work

• Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection
• Nearly unbiased variable selection under minimax concave penalty
• Nonlinear total variation based noise removal algorithms
• Sparse regression learning by aggregation and Langevin Monte-Carlo
• Mirror averaging with sparsity priors
• Kullback-Leibler aggregation and misspecified generalized linear models
• Exponential screening and optimal rates of sparse estimation
• Oracle inequalities and optimal inference under group sparsity
• Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion
• Linear and convex aggregation of density estimators
• Learning by mirror averaging
• The benefit of group sparsity
• Mixing least-squares estimators when the variance is unknown
• A simple proof of the restricted isometry property for random matrices
• Ordered linear smoothers
• Model selection in nonparametric regression
• Aggregating regression procedures to improve performance
• Complexity regularization via localized random penalties
• Statistical learning theory and stochastic optimization. Ecole d'Eté de Probabilitiés de Saint-Flour XXXI -- 2001.
• Sharp oracle inequalities for aggregation of affine estimators
• PAC-Bayesian bounds for sparse regression estimation with exponential weights
• Simultaneous analysis of Lasso and Dantzig selector
• Recursive aggregation of estimators by the mirror descent algorithm with averaging
• Empirical risk minimization is optimal for the convex aggregation problem
• Aggregation for Gaussian regression
• Simultaneous adaptation to the margin and to complexity in classification
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• Information Theory and Mixing Least-Squares Regressions
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• Sparsity and Smoothness Via the Fused Lasso
• Learning Theory and Kernel Machines
• Aggregation by Exponential Weighting and Sharp Oracle Inequalities
• Model Selection and Estimation in Regression with Grouped Variables
• Introduction to nonparametric estimation
• Gaussian model selection
• Model selection and error estimation
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item