Consistent regression using data-dependent coverings

DOI10.1214/21-EJS1806MaRDI QIDQ2044358zbMATH OpenOpenAlexFDO

Authors Vincent Margot, Jean-Patrick Baudry, Olivier Wintenberger, Frédéric Guilloux

Publication date 9 August 2021

Published in Electronic Journal of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1907.02306

zbMATH Keywords

consistencydata-dependent coveringinterpretable learningnonparametric regressionrule-based algorithm

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Nonparametric regression and quantile regression (62G08) Asymptotic properties of nonparametric inference (62G20)

Abstract: In this paper, we introduce a novel method to generate interpretable regression function estimators. The idea is based on called data-dependent coverings. The aim is to extract from the data a covering of the feature space instead of a partition. The estimator predicts the empirical conditional expectation over the cells of the partitions generated from the coverings. Thus, such estimator has the same form as those issued from data-dependent partitioning algorithms. We give sufficient conditions to ensure the consistency, avoiding the sufficient condition of shrinkage of the cells that appears in the former literature. Doing so, we reduce the number of covering elements. We show that such coverings are interpretable and each element of the covering is tagged as significant or insignificant. The proof of the consistency is based on a control of the error of the empirical estimation of conditional expectations which is interesting on its own.

Recommendations

• Consistent regression estimation with fixed design points under dependence conditions
• Consistent least squares nonparametric regression
• On consistent statistical procedures in regression
• Consistent estimation of linear regression models using matched~data
• scientific article; zbMATH DE number 3211331
• A Consistent Estimator for Linear Models with Dependent Observations
• Strongly consistent estimation in regression models with contaminated data
• A consistent estimator for nonlinear regression models
• Robust regression with high coverage.

Cites work

• A distribution-free theory of nonparametric regression
• Asymptotic Statistics
• Consistency of random forests
• Consistent estimation of residual variance with random forest out-of-bag errors
• First order regression
• Greedy function approximation: A gradient boosting machine.
• Hedonic housing prices and the demand for clean air
• Histogram regression estimation using data-dependent partitions
• scientific article; zbMATH DE number 3860199 (Why is no real title available?)
• scientific article; zbMATH DE number 823069 (Why is no real title available?)
• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• Least angle regression. (With discussion)
• Node harvest
• Predictive learning via rule ensembles
• Random forests
• SIRUS: stable and interpretable RUle set for classification
• Some special Vapnik-Chervonenkis classes
• Stochastic gradient boosting.
• The elements of statistical learning. Data mining, inference, and prediction

Describes a project that uses

Uses Software

• UCI
• FORS
• shap
• C4.5

This page was built for publication: Consistent regression using data-dependent coverings

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2044358)