Aggregation of affine estimators
From MaRDI portal
Abstract: We consider the problem of aggregating a general collection of affine estimators for fixed design regression. Relevant examples include some commonly used statistical estimators such as least squares, ridge and robust least squares estimators. Dalalyan and Salmon (2012) have established that, for this problem, exponentially weighted (EW) model selection aggregation leads to sharp oracle inequalities in expectation, but similar bounds in deviation were not previously known. While results indicate that the same aggregation scheme may not satisfy sharp oracle inequalities with high probability, we prove that a weaker notion of oracle inequality for EW that holds with high probability. Moreover, using a generalization of the newly introduced -aggregation scheme we also prove sharp oracle inequalities that hold with high probability. Finally, we apply our results to universal aggregation and show that our proposed estimator leads simultaneously to all the best known bounds for aggregation, including -aggregation, , with high probability.
Recommendations
Cites work
- scientific article; zbMATH DE number 3122730 (Why is no real title available?)
- scientific article; zbMATH DE number 3772326 (Why is no real title available?)
- scientific article; zbMATH DE number 1239310 (Why is no real title available?)
- scientific article; zbMATH DE number 1321826 (Why is no real title available?)
- scientific article; zbMATH DE number 1522808 (Why is no real title available?)
- Adaptive Minimax Estimation over Sparse $\ell_q$-Hulls
- Adaptive estimation of a quadratic functional by model selection.
- Adaptive spline estimators in nonparametric regression models
- Aggregating regression procedures to improve performance
- Aggregation by Exponential Weighting and Sharp Oracle Inequalities
- Aggregation via empirical risk minimization
- All Admissible Linear Estimates of the Mean Vector
- Deviation optimal learning using greedy \(Q\)-aggregation
- Exponential screening and optimal rates of sparse estimation
- Functional aggregation for nonparametric regression.
- General nonexact oracle inequalities for classes with a subexponential envelope
- Generalized mirror averaging and \(D\)-convex aggregation
- Information Theory and Mixing Least-Squares Regressions
- Introduction to nonparametric estimation
- Kullback-Leibler aggregation and misspecified generalized linear models
- Learning Theory and Kernel Machines
- Linear and convex aggregation of density estimators
- Minimax Rates of Estimation for High-Dimensional Linear Regression Over $\ell_q$-Balls
- Mixing least-squares estimators when the variance is unknown
- Optimal filtering of square-integrable signals in Gaussian noise
- Optimal learning with \textit{Q}-aggregation
- Optimal rates of aggregation in classification under low noise assumption
- PAC-Bayesian bounds for sparse regression estimation with exponential weights
- Penalized blockwise Stein's method, monotone oracles and sharp adaptive estimation
- Sharp oracle inequalities for aggregation of affine estimators
- Simultaneous analysis of Lasso and Dantzig selector
- Sparse estimation by exponential weighting
- Sparsity oracle inequalities for the Lasso
- Statistical learning theory and stochastic optimization. Ecole d'Eté de Probabilitiés de Saint-Flour XXXI -- 2001.
- The Gelfand widths of \(\ell_p\)-balls for \(0 < p \leq 1\)
Cited in
(17)- On the exponentially weighted aggregate with the Laplace prior
- Solution of linear ill-posed problems by model selection and aggregation
- Aggregation of spectral density estimators
- scientific article; zbMATH DE number 1164005 (Why is no real title available?)
- A bias aggregation theorem
- Common Knowledge of a Multivariate Aggregate Statistic
- Exponential weights in multivariate regression and a low-rankness favoring prior
- Recursive aggregation of estimators by the mirror descent algorithm with averaging
- scientific article; zbMATH DE number 2217376 (Why is no real title available?)
- Second-order Stein: SURE for SURE and other applications in high-dimensional inference
- Sharp oracle inequalities for aggregation of affine estimators
- User-friendly Introduction to PAC-Bayes Bounds
- Optimal bounds for aggregation of affine estimators
- Model averaging: a shrinkage perspective
- PAC-Bayesian aggregation of affine estimators
- A unified scheme for generalizing cardinality estimators to sum aggregation
- Aggregating estimates by convex optimization
This page was built for publication: Aggregation of affine estimators
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2447090)
