Tree-structured regression and the differentiation of integrals

DOI10.1214/009053606000001000zbMATH Open1122.62027arXiv0708.0385OpenAlexW3099307660MaRDI QIDQ997366FDOQ997366

Authors: Richard A. Olshen 

Publication date: 23 July 2007

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: This paper provides answers to questions regarding the almost sure limiting behavior of rooted, binary tree-structured rules for regression. Examples show that questions raised by Gordon and Olshen in 1984 have negative answers. For these examples of regression functions and sequences of their associated binary tree-structured approximations, for all regression functions except those in a set of the first category, almost sure consistency fails dramatically on events of full probability. One consequence is that almost sure consistency of binary tree-structured rules such as CART requires conditions beyond requiring that (1) the regression function be in ${mathcalL}^1$, (2) partitions of a Euclidean feature space be into polytopes with sides parallel to coordinate axes, (3) the mesh of the partitions becomes arbitrarily fine almost surely and (4) the empirical learning sample content of each polytope be ``large enough. The material in this paper includes the solution to a problem raised by Dudley in discussions. The main results have a corollary regarding the lack of almost sure consistency of certain Bayes-risk consistent rules for classification.

Full work available at URL: https://arxiv.org/abs/0708.0385

Recommendations

• Risk Bounds for CART Regression Trees
• Almost surely consistent nonparametric regression from recursive partitioning schemes
• Almost everywhere convergence of a recursive regression function estimate and classification (Corresp.)
• Asymptotic consistency of median regression trees
• Histogram regression estimation using data-dependent partitions

zbMATH Keywords

maximal functionsbinary tree-structured partition

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Empirical decision procedures; empirical Bayes procedures (62C12) Strong limit theorems (60F15) Continuity and differentiation questions (26B05) Abstract differentiation theory, differentiation of set functions (28A15)

Cites Work

• The elements of statistical learning. Data mining, inference, and prediction
• Title not available (Why is that?)
• Consistent nonparametric regression. Discussion
• Title not available (Why is that?)
• Title not available (Why is that?)
• Recursive partitioning in the health sciences
• Histogram regression estimation using data-dependent partitions
• Differentiation of integrals in \(\mathbb{R}^n\)
• Title not available (Why is that?)
• Consistency of data-driven histogram methods for density estimation and classification
• CART and best-ortho-basis: a connection
• New multivariate product density estimators
• Remark on the differentiability of the Lebesgue indefinite integral
• Almost surely consistent nonparametric regression from recursive partitioning schemes
• Zur Differentiation der Lebesgueschen Integrale

Cited In (3)

• <b>CONSTRUCTION OF REGRESSION TREES ON INTERVAL-VALUED SYMBOLIC VARIABLES </b>
• Interactive Tree-Structured Regression via Principal Hessian Directions
• Remembering Leo Breiman

Uses Software

• ElemStatLearn

This page was built for publication: Tree-structured regression and the differentiation of integrals

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q997366)