Deviation optimal learning using greedy Q-aggregation
From MaRDI portal
Abstract: Given a finite family of functions, the goal of model selection aggregation is to construct a procedure that mimics the function from this family that is the closest to an unknown regression function. More precisely, we consider a general regression model with fixed design and measure the distance between functions by the mean squared error at the design points. While procedures based on exponential weights are known to solve the problem of model selection aggregation in expectation, they are, surprisingly, sub-optimal in deviation. We propose a new formulation called Q-aggregation that addresses this limitation; namely, its solution leads to sharp oracle inequalities that are optimal in a minimax sense. Moreover, based on the new formulation, we design greedy Q-aggregation procedures that produce sparse aggregation models achieving the optimal rate. The convergence and performance of these greedy procedures are illustrated and compared with other standard methods on simulated examples.
Recommendations
Cites work
- scientific article; zbMATH DE number 5764862 (Why is no real title available?)
- scientific article; zbMATH DE number 1321826 (Why is no real title available?)
- scientific article; zbMATH DE number 1522808 (Why is no real title available?)
- scientific article; zbMATH DE number 2107836 (Why is no real title available?)
- scientific article; zbMATH DE number 6176343 (Why is no real title available?)
- scientific article; zbMATH DE number 1405931 (Why is no real title available?)
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
- A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training
- Adaptive estimation of a quadratic functional by model selection.
- Aggregated estimators and empirical complexity for least square regression
- Aggregation by Exponential Weighting and Sharp Oracle Inequalities
- Aggregation via empirical risk minimization
- Exponential screening and optimal rates of sparse estimation
- Functional aggregation for nonparametric regression.
- Hyper-sparse optimal aggregation
- Kullback-Leibler aggregation and misspecified generalized linear models
- Learning Theory and Kernel Machines
- Learning by mirror averaging
- On the optimality of the aggregate with exponential weights for low temperatures
- Sharp oracle inequalities for aggregation of affine estimators
- Trading accuracy for sparsity in optimization problems with sparsity constraints
- Universal approximation bounds for superpositions of a sigmoidal function
Cited in
(22)- Aggregation of affine estimators
- User-friendly Introduction to PAC-Bayes Bounds
- Sharp oracle inequalities for aggregation of affine estimators
- Localized Gaussian width of \(M\)-convex hulls with applications to Lasso and convex aggregation
- Hyper-sparse optimal aggregation
- Optimal Kullback-Leibler aggregation in mixture density estimation by maximum likelihood
- Histopathological imaging‐based cancer heterogeneity analysis via penalized fusion with model averaging
- Optimal learning with \textit{Q}-aggregation
- Rank-Based Greedy Model Averaging for High-Dimensional Survival Data
- Model aggregation for doubly divided data with large size and large dimension
- Solution of linear ill-posed problems by model selection and aggregation
- Optimal bounds for aggregation of affine estimators
- Statistical inference in compound functional models
- Martingale-residual-based greedy model averaging for high-dimensional current status data
- Second-order Stein: SURE for SURE and other applications in high-dimensional inference
- scientific article; zbMATH DE number 7306896 (Why is no real title available?)
- Optimal learning with Bernstein online aggregation
- Optimal and Safe Estimation for High-Dimensional Semi-Supervised Learning
- PAC-Bayesian risk bounds for group-analysis sparse regression by exponential weighting
- An adaptive multiclass nearest neighbor classifier
- Aggregating estimates by convex optimization
- Estimation and Inference for High-Dimensional Generalized Linear Models with Knowledge Transfer
This page was built for publication: Deviation optimal learning using greedy \(Q\)-aggregation
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q693750)
