Node harvest
From MaRDI portal
Abstract: When choosing a suitable technique for regression and classification with multivariate predictor variables, one is often faced with a tradeoff between interpretability and high predictive accuracy. To give a classical example, classification and regression trees are easy to understand and interpret. Tree ensembles like Random Forests provide usually more accurate predictions. Yet tree ensembles are also more difficult to analyze than single trees and are often criticized, perhaps unfairly, as `black box' predictors. Node harvest is trying to reconcile the two aims of interpretability and predictive accuracy by combining positive aspects of trees and tree ensembles. Results are very sparse and interpretable and predictive accuracy is extremely competitive, especially for low signal-to-noise data. The procedure is simple: an initial set of a few thousand nodes is generated randomly. If a new observation falls into just a single node, its prediction is the mean response of all training observation within this node, identical to a tree-like prediction. A new observation falls typically into several nodes and its prediction is then the weighted average of the mean responses across all these nodes. The only role of node harvest is to `pick' the right nodes from the initial large ensemble of nodes by choosing node weights, which amounts in the proposed algorithm to a quadratic programming problem with linear inequality constraints. The solution is sparse in the sense that only very few nodes are selected with a nonzero weight. This sparsity is not explicitly enforced. Maybe surprisingly, it is not necessary to select a tuning parameter for optimal predictive accuracy. Node harvest can handle mixed data and missing values and is shown to be simple to interpret and competitive in predictive accuracy on a variety of data sets.
Recommendations
Cites work
- scientific article; zbMATH DE number 3860199 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- A numerically stable dual method for solving strictly convex quadratic programs
- Additive logistic regression: a statistical view of boosting. (With discussion and a rejoinder by the authors)
- Atomic decomposition by basis pursuit
- Bagging predictors
- Boosting With theL2Loss
- Breast Cancer Diagnosis and Prognosis Via Linear Programming
- Convexity, Classification, and Risk Bounds
- Forest Garrote
- Least angle regression. (With discussion)
- Predictive learning via rule ensembles
- Probabilistic Sensitivity Analysis of Complex Models: A Bayesian Approach
- Random Forests and Adaptive Nearest Neighbors
- Random forests
- Random survival forests
- Stacked regressions
- The elements of statistical learning. Data mining, inference, and prediction
Cited in
(17)- Mathematical optimization in classification and regression trees
- The Delaunay triangulation learner and its ensembles
- Supervised classification and mathematical optimization
- SIRUS: stable and interpretable RUle set for classification
- Selective harvesting over networks
- SUBiNN: a stacked uni- and bivariate \(k\)NN sparse ensemble
- On optimal regression trees to detect critical intervals for multivariate functional data
- Consistent regression using data-dependent coverings
- Ensemble of optimal trees, random forest and random projection ensemble classification
- Conclusive local interpretation rules for random forests
- Model transparency and interpretability: survey and application to the insurance industry
- Neural-symbolic temporal decision trees for multivariate time series classification
- Imputation Scores
- Disjunctive Rule Lists
- A decision-theoretic approach for model interpretability in Bayesian framework
- Learning customized and optimized lists of rules with mathematical programming
- Interpretable classifiers using rules and Bayesian analysis: building a better stroke prediction model
This page was built for publication: Node harvest
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q542973)
