Learning by mirror averaging
From MaRDI portal
Abstract: Given a finite collection of estimators or classifiers, we study the problem of model selection type aggregation, that is, we construct a new estimator or classifier, called aggregate, which is nearly as good as the best among them with respect to a given risk criterion. We define our aggregate by a simple recursive procedure which solves an auxiliary stochastic linear programming problem related to the original nonlinear one and constitutes a special case of the mirror averaging algorithm. We show that the aggregate satisfies sharp oracle inequalities under some general assumptions. The results are applied to several problems including regression, classification and density estimation.
Recommendations
- Unifying mirror descent and dual averaging
- Learning from others: conditioning versus averaging
- Mirror averaging with sparsity priors
- Average behavior in learning models
- Ergodic mirror descent
- Recursive aggregation of estimators by the mirror descent algorithm with averaging
- Learning by matching
- Learning dynamics: a replica approach
- scientific article; zbMATH DE number 741372
Cites work
- Aggregation for Gaussian regression
- Competitive On-line Statistics
- Complexity regularization via localized random penalties
- Concentration inequalities and model selection. Ecole d'Eté de Probabilités de Saint-Flour XXXIII -- 2003.
- Efficient agnostic learning of neural networks with bounded fan-in
- From -entropy to KL-entropy: analysis of minimum information complexity density estima\-tion
- scientific article; zbMATH DE number 3790208 (Why is no real title available?)
- scientific article; zbMATH DE number 1344902 (Why is no real title available?)
- scientific article; zbMATH DE number 741240 (Why is no real title available?)
- scientific article; zbMATH DE number 1522808 (Why is no real title available?)
- Information Theory and Mixing Least-Squares Regressions
- Introduction to nonparametric estimation
- Learning Theory and Kernel Machines
- Mixing strategies for density estimation.
- Model selection and error estimation
- Model selection in nonparametric regression
- Prediction, Learning, and Games
- Recursive aggregation of estimators by the mirror descent algorithm with averaging
- Sequential prediction of individual sequences under general loss functions
- Sequential Procedures for Aggregating Arbitrary Estimators of a Conditional Mean
- Spatial aggregation of local likelihood estimates with applications to classification
- Statistical learning theory and stochastic optimization. Ecole d'Eté de Probabilitiés de Saint-Flour XXXI -- 2001.
- Theory of Classification: a Survey of Some Recent Advances
- Universal linear prediction by model order weighting
Cited in
(55)- A universal procedure for aggregating estimators
- Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity
- Entropic optimal transport is maximum-likelihood deconvolution
- Optimal Kullback-Leibler aggregation in mixture density estimation by maximum likelihood
- On variance reduction for stochastic smooth convex optimization with multiplicative noise
- Exponentially concave functions and a new information geometry
- On the exponentially weighted aggregate with the Laplace prior
- General oracle inequalities for model selection
- Noisy independent factor analysis model for density estimation and classification
- PAC-Bayesian bounds for sparse regression estimation with exponential weights
- On the optimality of the aggregate with exponential weights for low temperatures
- Aggregating estimates by convex optimization
- Suboptimality of constrained least squares and improvements via non-linear predictors
- Distribution-free robust linear regression
- Exponential weights in multivariate regression and a low-rankness favoring prior
- Optimal rates for estimation of two-dimensional totally positive distributions
- Aggregation of estimators and stochastic optimization
- Estimation of Monge matrices
- Localized Gaussian width of \(M\)-convex hulls with applications to Lasso and convex aggregation
- On aggregation for heavy-tailed classes
- Fast learning rates in statistical inference through aggregation
- Empirical risk minimization is optimal for the convex aggregation problem
- Estimation and variable selection with exponential weights
- Model selection for density estimation with \(\mathbb L_2\)-loss
- Optimal learning with \textit{Q}-aggregation
- Aggregation for Gaussian regression
- Optimal rates of aggregation in classification under low noise assumption
- On martingale extensions of Vapnik-Chervonenkis theory with applications to online learning
- Performance of empirical risk minimization in linear aggregation
- Penalty methods with stochastic approximation for stochastic nonlinear programming
- Some multivariate risk indicators: minimization by using a Kiefer-Wolfowitz approach to the mirror stochastic algorithm
- An optimal method for stochastic composite optimization
- Sparse regression learning by aggregation and Langevin Monte-Carlo
- Mirror averaging with sparsity priors
- Kullback-Leibler aggregation and misspecified generalized linear models
- Fast rates for general unbounded loss functions: from ERM to generalized Bayes
- Aggregated hold-out
- scientific article; zbMATH DE number 7625184 (Why is no real title available?)
- Stochastic approximation versus sample average approximation for Wasserstein barycenters
- An adaptive multiclass nearest neighbor classifier
- Prediction of time series by statistical learning: general losses and fast rates
- Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization
- Sparse estimation by exponential weighting
- Unifying mirror descent and dual averaging
- User-friendly Introduction to PAC-Bayes Bounds
- Simple proof of the risk bound for denoising by exponential weights for asymmetric noise distributions
- Stochastic online convex optimization. Application to probabilistic time series forecasting
- Cross-validation approaches for multi-study predictions
- Deviation optimal learning using greedy \(Q\)-aggregation
- A rank-based sequential test of independence
- Learning from others: conditioning versus averaging
- Optimal learning with Bernstein online aggregation
- Generalized mirror averaging and D-convex aggregation
- Parameter tuning in pointwise adaptation using a propagation approach
- Aggregation via empirical risk minimization
This page was built for publication: Learning by mirror averaging
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q955138)
