L₂ boosting in kernel regression

DOI10.3150/08-BEJ160MaRDI QIDQ605014zbMATH OpenFDO

Authors Sumit K. Garg

Publication date 12 November 2010

Published in Bernoulli (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0909.0833

zbMATH Keywords

bias reductionboostingkernel regressionNadaraya-Watson smoothertwicing

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Asymptotic properties of nonparametric inference (62G20)

Abstract: In this paper, we investigate the theoretical and empirical properties of $L_2$ boosting with kernel regression estimates as weak learners. We show that each step of $L_2$ boosting reduces the bias of the estimate by two orders of magnitude, while it does not deteriorate the order of the variance. We illustrate the theoretical findings by some simulated examples. Also, we demonstrate that $L_2$ boosting is superior to the use of higher-order kernels, which is a well-known method of reducing the bias of the kernel estimate.

Recommendations

• On boosting kernel regression
• Boosting With theL₂Loss
• Boosting as a kernel-based method
• Boosting kernel density estimates: A bias reduction technique?
• Dimension reduction boosting

Cites work

• A decision-theoretic generalization of on-line learning and an application to boosting
• A distribution-free theory of nonparametric regression
• Additive logistic regression: a statistical view of boosting. (With discussion and a rejoinder by the authors)
• An Effective Bandwidth Selector for Local Least Squares Regression
• Arcing classifiers. (With discussion)
• Boosting a weak learning algorithm by majority
• Boosting the margin: a new explanation for the effectiveness of voting methods
• Boosting With theL₂Loss
• Greedy function approximation: A gradient boosting machine.
• scientific article; zbMATH DE number 991833 (Why is no real title available?)
• scientific article; zbMATH DE number 3655168 (Why is no real title available?)
• scientific article; zbMATH DE number 3637090 (Why is no real title available?)
• Improved boosting algorithms using confidence-rated predictions
• Some theory for generalized boosting algorithms

Cited in

(10)

• On boosting kernel regression
• Optimal rates for coefficient-based regularized regression
• Boosting as a kernel-based method
• Boosting With theL₂Loss
• Boosted kernel ridge regression: optimal learning rates and early stopping
• An $L_{2}$-Boosting Algorithm for Estimation of a Regression Function
• Boosting kernel density estimates: A bias reduction technique?
• Algorithmic Learning Theory
• A new kernel regression approach for robustified L ₂ boosting
• Encompassing tests for nonparametric regressions

This page was built for publication: \(L_{2}\) boosting in kernel regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q605014)