Mirror averaging with sparsity priors
From MaRDI portal
Publication:442083
DOI10.3150/11-BEJ361zbMATH Open1243.62008arXiv1003.1189MaRDI QIDQ442083FDOQ442083
Authors: Arnak S. Dalalyan, Alexandre B. Tsybakov 
Publication date: 9 August 2012
Published in: Bernoulli (Search for Journal in Brave)
Abstract: We consider the problem of aggregating the elements of a possibly infinite dictionary for building a decision procedure that aims at minimizing a given criterion. Along with the dictionary, an independent identically distributed training sample is available, on which the performance of a given procedure can be tested. In a fairly general set-up, we establish an oracle inequality for the Mirror Averaging aggregate with any prior distribution. By choosing an appropriate prior, we apply this oracle inequality in the context of prediction under sparsity assumption for the problems of regression with random design, density estimation and binary classification.
Full work available at URL: https://arxiv.org/abs/1003.1189
Recommendations
Bayesian inference (62F15) Density estimation (62G07) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Linear regression; mixed models (62J05) Statistical decision theory (62C99)
Cites Work
- The Adaptive Lasso and Its Oracle Properties
- Fast learning rates in statistical inference through aggregation
- Simultaneous analysis of Lasso and Dantzig selector
- High-dimensional generalized linear models and the lasso
- The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
- High-dimensional graphs and variable selection with the Lasso
- Title not available (Why is that?)
- Title not available (Why is that?)
- Sequential prediction of individual sequences under general loss functions
- Learning Theory and Kernel Machines
- Regularization and Variable Selection Via the Elastic Net
- Prediction, Learning, and Games
- Learning by mirror averaging
- Sparsity oracle inequalities for the Lasso
- Empirical Bayes selection of wavelet thresholds
- Compressive sampling
- The sparsity and bias of the LASSO selection in high-dimensional linear regression
- Better Subset Regression Using the Nonnegative Garrote
- Stable recovery of sparse overcomplete representations in the presence of noise
- A universal procedure for aggregating estimators
- Mixing least-squares estimators when the variance is unknown
- Statistical behavior and consistency of classification methods based on convex risk minimization.
- Aggregation for Gaussian regression
- Information Theory and Mixing Least-Squares Regressions
- Adaptive Regression by Mixing
- The Dantzig selector and sparsity oracle inequalities
- Some sharp performance bounds for least squares regression with \(L_1\) regularization
- Title not available (Why is that?)
- Aggregation and Sparsity Via ℓ1 Penalized Least Squares
- Convexity, Classification, and Risk Bounds
- Exponential screening and optimal rates of sparse estimation
- Aggregating regression procedures to improve performance
- Statistical learning theory and stochastic optimization. Ecole d'Eté de Probabilitiés de Saint-Flour XXXI -- 2001.
- Optimal aggregation of classifiers in statistical learning.
- On the Generalization Ability of On-Line Learning Algorithms
- On optimality of Bayesian testimation in the normal means problem
- Bayesian inference and optimal design for the sparse linear model
- Hyper-sparse optimal aggregation
- Sparsity in penalized empirical risk minimization
- Smoothing of Multivariate Data
- Sequential Procedures for Aggregating Arbitrary Estimators of a Conditional Mean
- PAC-Bayesian stochastic model selection
- Title not available (Why is that?)
- Recursive aggregation of estimators by the mirror descent algorithm with averaging
- Sparse regression learning by aggregation and Langevin Monte-Carlo
- Sparse recovery in convex hulls via entropy penalization
- On the optimality of the aggregate with exponential weights for low temperatures
- Aggregation by Exponential Weighting and Sharp Oracle Inequalities
- PAC-Bayesian bounds for randomized empirical risk minimizers
- Nonlinear estimation over weak Besov spaces and minimax Bayes
- Optimal rates of aggregation in classification under low noise assumption
- Generalized mirror averaging and \(D\)-convex aggregation
- From \(\varepsilon\)-entropy to KL-entropy: analysis of minimum information complexity density estima\-tion
Cited In (16)
- Sharp oracle inequalities for aggregation of affine estimators
- Exponential screening and optimal rates of sparse estimation
- Optimal Kullback-Leibler aggregation in mixture density estimation by maximum likelihood
- Optimal learning with \textit{Q}-aggregation
- Exponential weights in multivariate regression and a low-rankness favoring prior
- Sparse regression learning by aggregation and Langevin Monte-Carlo
- Entropic optimal transport is maximum-likelihood deconvolution
- Prediction error bounds for linear regression with the TREX
- Adaptive Bayesian density regression for high-dimensional data
- PAC-Bayesian bounds for sparse regression estimation with exponential weights
- On the exponentially weighted aggregate with the Laplace prior
- PAC-Bayesian risk bounds for group-analysis sparse regression by exponential weighting
- High-dimensional sparse classification using exponential weighting with empirical hinge loss
- Learning by mirror averaging
- A quasi-Bayesian perspective to online clustering
- Sparse estimation by exponential weighting
Uses Software
This page was built for publication: Mirror averaging with sparsity priors
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q442083)
