Abstract: We consider the problem of aggregating the elements of a possibly infinite dictionary for building a decision procedure that aims at minimizing a given criterion. Along with the dictionary, an independent identically distributed training sample is available, on which the performance of a given procedure can be tested. In a fairly general set-up, we establish an oracle inequality for the Mirror Averaging aggregate with any prior distribution. By choosing an appropriate prior, we apply this oracle inequality in the context of prediction under sparsity assumption for the problems of regression with random design, density estimation and binary classification.
Recommendations
Cites work
- scientific article; zbMATH DE number 5957408 (Why is no real title available?)
- scientific article; zbMATH DE number 5544465 (Why is no real title available?)
- scientific article; zbMATH DE number 1158743 (Why is no real title available?)
- scientific article; zbMATH DE number 2015216 (Why is no real title available?)
- A universal procedure for aggregating estimators
- Adaptive Regression by Mixing
- Aggregating regression procedures to improve performance
- Aggregation and Sparsity Via ℓ1 Penalized Least Squares
- Aggregation by Exponential Weighting and Sharp Oracle Inequalities
- Aggregation for Gaussian regression
- Bayesian inference and optimal design for the sparse linear model
- Better Subset Regression Using the Nonnegative Garrote
- Compressive sampling
- Convexity, Classification, and Risk Bounds
- Empirical Bayes selection of wavelet thresholds
- Exponential screening and optimal rates of sparse estimation
- Fast learning rates in statistical inference through aggregation
- From -entropy to KL-entropy: analysis of minimum information complexity density estima\-tion
- Generalized mirror averaging and D-convex aggregation
- High-dimensional generalized linear models and the lasso
- High-dimensional graphs and variable selection with the Lasso
- Hyper-sparse optimal aggregation
- Information Theory and Mixing Least-Squares Regressions
- Learning Theory and Kernel Machines
- Learning by mirror averaging
- Mixing least-squares estimators when the variance is unknown
- Nonlinear estimation over weak Besov spaces and minimax Bayes
- On optimality of Bayesian testimation in the normal means problem
- On the Generalization Ability of On-Line Learning Algorithms
- On the optimality of the aggregate with exponential weights for low temperatures
- Optimal aggregation of classifiers in statistical learning.
- Optimal rates of aggregation in classification under low noise assumption
- PAC-Bayesian bounds for randomized empirical risk minimizers
- PAC-Bayesian stochastic model selection
- Prediction, Learning, and Games
- Recursive aggregation of estimators by the mirror descent algorithm with averaging
- Regularization and Variable Selection Via the Elastic Net
- Sequential Procedures for Aggregating Arbitrary Estimators of a Conditional Mean
- Sequential prediction of individual sequences under general loss functions
- Simultaneous analysis of Lasso and Dantzig selector
- Smoothing of Multivariate Data
- Some sharp performance bounds for least squares regression with L₁ regularization
- Sparse recovery in convex hulls via entropy penalization
- Sparse regression learning by aggregation and Langevin Monte-Carlo
- Sparsity in penalized empirical risk minimization
- Sparsity oracle inequalities for the Lasso
- Stable recovery of sparse overcomplete representations in the presence of noise
- Statistical behavior and consistency of classification methods based on convex risk minimization.
- Statistical learning theory and stochastic optimization. Ecole d'Eté de Probabilitiés de Saint-Flour XXXI -- 2001.
- The Adaptive Lasso and Its Oracle Properties
- The Dantzig selector and sparsity oracle inequalities
- The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
- The sparsity and bias of the LASSO selection in high-dimensional linear regression
Cited in
(16)- Sharp oracle inequalities for aggregation of affine estimators
- Exponential screening and optimal rates of sparse estimation
- Optimal Kullback-Leibler aggregation in mixture density estimation by maximum likelihood
- Optimal learning with \textit{Q}-aggregation
- Exponential weights in multivariate regression and a low-rankness favoring prior
- Sparse regression learning by aggregation and Langevin Monte-Carlo
- Entropic optimal transport is maximum-likelihood deconvolution
- Prediction error bounds for linear regression with the TREX
- PAC-Bayesian bounds for sparse regression estimation with exponential weights
- Adaptive Bayesian density regression for high-dimensional data
- On the exponentially weighted aggregate with the Laplace prior
- Learning by mirror averaging
- PAC-Bayesian risk bounds for group-analysis sparse regression by exponential weighting
- High-dimensional sparse classification using exponential weighting with empirical hinge loss
- A quasi-Bayesian perspective to online clustering
- Sparse estimation by exponential weighting
This page was built for publication: Mirror averaging with sparsity priors
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q442083)
