L₂ boosting in kernel regression

DOI10.3150/08-BEJ160zbMATH Open1200.62040arXiv0909.0833MaRDI QIDQ605014FDOQ605014

Sumit K. Garg

Publication date: 12 November 2010

Published in: Bernoulli (Search for Journal in Brave)

Abstract: In this paper, we investigate the theoretical and empirical properties of $L_2$ boosting with kernel regression estimates as weak learners. We show that each step of $L_2$ boosting reduces the bias of the estimate by two orders of magnitude, while it does not deteriorate the order of the variance. We illustrate the theoretical findings by some simulated examples. Also, we demonstrate that $L_2$ boosting is superior to the use of higher-order kernels, which is a well-known method of reducing the bias of the kernel estimate.

Full work available at URL: https://arxiv.org/abs/0909.0833

Recommendations

• On boosting kernel regression
• Boosting With theL₂Loss
• Boosting as a kernel-based method
• Boosting kernel density estimates: A bias reduction technique?
• Dimension reduction boosting

zbMATH Keywords

boostingkernel regressionbias reductionNadaraya-Watson smoothertwicing

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Asymptotic properties of nonparametric inference (62G20)

Cites Work

• A decision-theoretic generalization of on-line learning and an application to boosting
• Greedy function approximation: A gradient boosting machine.
• Title not available (Why is that?)
• Title not available (Why is that?)
• Boosting the margin: a new explanation for the effectiveness of voting methods
• Additive logistic regression: a statistical view of boosting. (With discussion and a rejoinder by the authors)
• An Effective Bandwidth Selector for Local Least Squares Regression
• Arcing classifiers. (With discussion)
• Boosting With theL₂Loss
• A distribution-free theory of nonparametric regression
• Improved boosting algorithms using confidence-rated predictions
• Boosting a weak learning algorithm by majority
• Title not available (Why is that?)
• Title not available (Why is that?)

Cited In (8)

• An $L_{2}$-Boosting Algorithm for Estimation of a Regression Function
• Title not available (Why is that?)
• On boosting kernel regression
• A new kernel regression approach for robustified L ₂ boosting
• Boosting With theL₂Loss
• Optimal rates for coefficient-based regularized regression
• Boosting kernel density estimates: A bias reduction technique?
• Algorithmic Learning Theory

This page was built for publication: \(L_{2}\) boosting in kernel regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q605014)