Sparse estimation by exponential weighting
From MaRDI portal
Abstract: Consider a regression model with fixed design and Gaussian noise where the regression function can potentially be well approximated by a function that admits a sparse representation in a given dictionary. This paper resorts to exponential weights to exploit this underlying sparsity by implementing the principle of sparsity pattern aggregation. This model selection take on sparse estimation allows us to derive sparsity oracle inequalities in several popular frameworks, including ordinary sparsity, fused sparsity and group sparsity. One striking aspect of these theoretical results is that they hold under no condition in the dictionary. Moreover, we describe an efficient implementation of the sparsity pattern aggregation principle that compares favorably to state-of-the-art procedures on some basic numerical examples.
Recommendations
- Estimation and variable selection with exponential weights
- Aggregation by Exponential Weighting and Sharp Oracle Inequalities
- PAC-Bayesian bounds for sparse regression estimation with exponential weights
- Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity
- Exponential screening and optimal rates of sparse estimation
Cites work
- A simple proof of the restricted isometry property for random matrices
- Aggregating regression procedures to improve performance
- Aggregation by Exponential Weighting and Sharp Oracle Inequalities
- Aggregation for Gaussian regression
- Complexity regularization via localized random penalties
- Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection
- Empirical risk minimization is optimal for the convex aggregation problem
- Exponential screening and optimal rates of sparse estimation
- Gaussian model selection
- scientific article; zbMATH DE number 5544465 (Why is no real title available?)
- scientific article; zbMATH DE number 1522808 (Why is no real title available?)
- scientific article; zbMATH DE number 2117879 (Why is no real title available?)
- scientific article; zbMATH DE number 1405931 (Why is no real title available?)
- Information Theory and Mixing Least-Squares Regressions
- Introduction to nonparametric estimation
- Kullback-Leibler aggregation and misspecified generalized linear models
- Learning by mirror averaging
- Learning Theory and Kernel Machines
- Linear and convex aggregation of density estimators
- Mirror averaging with sparsity priors
- Mixing least-squares estimators when the variance is unknown
- Model selection and error estimation
- Model Selection and Estimation in Regression with Grouped Variables
- Model selection in nonparametric regression
- Nearly unbiased variable selection under minimax concave penalty
- Nonlinear total variation based noise removal algorithms
- Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion
- Oracle inequalities and optimal inference under group sparsity
- Ordered linear smoothers
- PAC-Bayesian bounds for sparse regression estimation with exponential weights
- Recursive aggregation of estimators by the mirror descent algorithm with averaging
- Sharp oracle inequalities for aggregation of affine estimators
- Simultaneous adaptation to the margin and to complexity in classification
- Simultaneous analysis of Lasso and Dantzig selector
- Sparse regression learning by aggregation and Langevin Monte-Carlo
- Sparsity and Smoothness Via the Fused Lasso
- Statistical learning theory and stochastic optimization. Ecole d'Eté de Probabilitiés de Saint-Flour XXXI -- 2001.
- The benefit of group sparsity
- The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Cited in
(44)- On the prediction loss of the Lasso in the partially labeled setting
- Entropic optimal transport is maximum-likelihood deconvolution
- Solution of linear ill-posed problems by model selection and aggregation
- Optimal bounds for aggregation of affine estimators
- PAC-Bayesian estimation and prediction in sparse additive models
- Optimal model selection in heteroscedastic regression using piecewise polynomial functions
- PAC-Bayesian bounds for sparse regression estimation with exponential weights
- Second-order Stein: SURE for SURE and other applications in high-dimensional inference
- Exponential weights in multivariate regression and a low-rankness favoring prior
- On estimation of isotonic piecewise constant signals
- A general framework for Bayes structured linear models
- Sharp oracle inequalities for low-complexity priors
- Non-parametric Poisson regression from independent and weakly dependent observations by model selection
- Localized Gaussian width of \(M\)-convex hulls with applications to Lasso and convex aggregation
- Sparse oracle inequalities for variable selection via regularized quantization
- PAC-Bayesian risk bounds for group-analysis sparse regression by exponential weighting
- Aggregation of affine estimators
- Estimation and variable selection with exponential weights
- Statistical inference in compound functional models
- Optimal learning with \textit{Q}-aggregation
- On risk bounds in isotonic and other shape restricted regression problems
- Generalized aggregation of misspecified models: with an application to asset pricing
- Ordered smoothers with exponential weighting
- Mirror averaging with sparsity priors
- Metamodel construction for sensitivity analysis
- Sparse matrix transform based weight updating in partial least squares regression
- Concentration inequalities for the exponential weighting method
- Chaining meets chain rule: multilevel entropic regularization and training of neural networks
- Bayesian inference in high-dimensional linear models using an empirical correlation-adaptive prior
- An adaptive multiclass nearest neighbor classifier
- Comparing and Weighting Imperfect Models Using D-Probabilities
- Exponential screening and optimal rates of sparse estimation
- Multiscale change point inference. With discussion and authors' reply
- Introduction to the special issue on sparsity and regularization methods
- Synthetic learner: model-free inference on treatments over time
- Model averaging for generalized linear models in diverging model spaces with effective model size
- Empirical Bayes inference in sparse high-dimensional generalized linear models
- Generalization bounds: perspectives from information theory and PAC-Bayes
- Kullback-Leibler excess risk bounds for exponential weighted aggregation in generalized linear models
- Consistent group selection using global-local shrinkage priors in sparse normal linear regression
- Exponentially weighted averaging of varying-coefficient partially linear models
- Global risk bounds and adaptation in univariate convex regression
- Bayesian linear regression with sparse priors
- Aggregation of predictors for nonstationary sub-linear processes and online adaptive forecasting of time varying autoregressive processes
This page was built for publication: Sparse estimation by exponential weighting
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5965309)
