Learning by mirror averaging
From MaRDI portal
Abstract: Given a finite collection of estimators or classifiers, we study the problem of model selection type aggregation, that is, we construct a new estimator or classifier, called aggregate, which is nearly as good as the best among them with respect to a given risk criterion. We define our aggregate by a simple recursive procedure which solves an auxiliary stochastic linear programming problem related to the original nonlinear one and constitutes a special case of the mirror averaging algorithm. We show that the aggregate satisfies sharp oracle inequalities under some general assumptions. The results are applied to several problems including regression, classification and density estimation.
Recommendations
- Unifying mirror descent and dual averaging
- Learning from others: conditioning versus averaging
- Mirror averaging with sparsity priors
- Average behavior in learning models
- Ergodic mirror descent
- Recursive aggregation of estimators by the mirror descent algorithm with averaging
- Learning by matching
- Learning dynamics: a replica approach
- scientific article; zbMATH DE number 741372
Cites work
- scientific article; zbMATH DE number 3790208 (Why is no real title available?)
- scientific article; zbMATH DE number 1344902 (Why is no real title available?)
- scientific article; zbMATH DE number 741240 (Why is no real title available?)
- scientific article; zbMATH DE number 1522808 (Why is no real title available?)
- Aggregation for Gaussian regression
- Competitive On-line Statistics
- Complexity regularization via localized random penalties
- Concentration inequalities and model selection. Ecole d'Eté de Probabilités de Saint-Flour XXXIII -- 2003.
- Efficient agnostic learning of neural networks with bounded fan-in
- From -entropy to KL-entropy: analysis of minimum information complexity density estima\-tion
- Information Theory and Mixing Least-Squares Regressions
- Introduction to nonparametric estimation
- Learning Theory and Kernel Machines
- Mixing strategies for density estimation.
- Model selection and error estimation
- Model selection in nonparametric regression
- Prediction, Learning, and Games
- Recursive aggregation of estimators by the mirror descent algorithm with averaging
- Sequential Procedures for Aggregating Arbitrary Estimators of a Conditional Mean
- Sequential prediction of individual sequences under general loss functions
- Spatial aggregation of local likelihood estimates with applications to classification
- Statistical learning theory and stochastic optimization. Ecole d'Eté de Probabilitiés de Saint-Flour XXXI -- 2001.
- Theory of Classification: a Survey of Some Recent Advances
- Universal linear prediction by model order weighting
Cited in
(52)- On martingale extensions of Vapnik-Chervonenkis theory with applications to online learning
- Unifying mirror descent and dual averaging
- A universal procedure for aggregating estimators
- User-friendly Introduction to PAC-Bayes Bounds
- Aggregation of estimators and stochastic optimization
- Localized Gaussian width of \(M\)-convex hulls with applications to Lasso and convex aggregation
- An optimal method for stochastic composite optimization
- Suboptimality of constrained least squares and improvements via non-linear predictors
- Optimal Kullback-Leibler aggregation in mixture density estimation by maximum likelihood
- On variance reduction for stochastic smooth convex optimization with multiplicative noise
- Fast learning rates in statistical inference through aggregation
- Performance of empirical risk minimization in linear aggregation
- Optimal learning with \textit{Q}-aggregation
- Exponential weights in multivariate regression and a low-rankness favoring prior
- Sparse regression learning by aggregation and Langevin Monte-Carlo
- Estimation and variable selection with exponential weights
- Learning from others: conditioning versus averaging
- Entropic optimal transport is maximum-likelihood deconvolution
- Distribution-free robust linear regression
- Mirror averaging with sparsity priors
- Aggregation for Gaussian regression
- Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity
- PAC-Bayesian bounds for sparse regression estimation with exponential weights
- Generalized mirror averaging and D-convex aggregation
- scientific article; zbMATH DE number 7625184 (Why is no real title available?)
- Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization
- Penalty methods with stochastic approximation for stochastic nonlinear programming
- Simple proof of the risk bound for denoising by exponential weights for asymmetric noise distributions
- Stochastic approximation versus sample average approximation for Wasserstein barycenters
- On aggregation for heavy-tailed classes
- Stochastic online convex optimization. Application to probabilistic time series forecasting
- Optimal learning with Bernstein online aggregation
- On the exponentially weighted aggregate with the Laplace prior
- Fast rates for general unbounded loss functions: from ERM to generalized Bayes
- On the optimality of the aggregate with exponential weights for low temperatures
- Empirical risk minimization is optimal for the convex aggregation problem
- Some multivariate risk indicators: minimization by using a Kiefer-Wolfowitz approach to the mirror stochastic algorithm
- Model selection for density estimation with \(\mathbb L_2\)-loss
- Kullback-Leibler aggregation and misspecified generalized linear models
- Noisy independent factor analysis model for density estimation and classification
- General oracle inequalities for model selection
- Prediction of time series by statistical learning: general losses and fast rates
- Deviation optimal learning using greedy \(Q\)-aggregation
- Aggregation via empirical risk minimization
- Optimal rates for estimation of two-dimensional totally positive distributions
- Optimal rates of aggregation in classification under low noise assumption
- An adaptive multiclass nearest neighbor classifier
- Exponentially concave functions and a new information geometry
- Parameter tuning in pointwise adaptation using a propagation approach
- Sparse estimation by exponential weighting
- Estimation of Monge matrices
- Aggregating estimates by convex optimization
This page was built for publication: Learning by mirror averaging
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q955138)
