Sparsity in optimal randomized classification trees
From MaRDI portal
Abstract: Decision trees are popular Classification and Regression tools and, when small-sized, easy to interpret. Traditionally, a greedy approach has been used to build the trees, yielding a very fast training process; however, controlling sparsity (a proxy for interpretability) is challenging. In recent studies, optimal decision trees, where all decisions are optimized simultaneously, have shown a better learning performance, especially when oblique cuts are implemented. In this paper, we propose a continuous optimization approach to build sparse optimal classification trees, based on oblique cuts, with the aim of using fewer predictor variables in the cuts as well as along the whole tree. Both types of sparsity, namely local and global, are modeled by means of regularizations with polyhedral norms. The computational experience reported supports the usefulness of our methodology. In all our data sets, local and global sparsity can be improved without harming classification accuracy. Unlike greedy approaches, our ability to easily trade in some of our classification accuracy for a gain in global sparsity is shown.
Recommendations
Cites work
- scientific article; zbMATH DE number 3860199 (Why is no real title available?)
- scientific article; zbMATH DE number 6438182 (Why is no real title available?)
- A random forest guided tour
- Auction optimization using regression trees and linear models as integer programs
- Comprehensible credit scoring models using rule extraction from support vector machines
- Constructing optimal binary decision trees is NP-complete
- Do we need hundreds of classifiers to solve real world classification problems?
- Learning decision trees with flexible constraints and objectives using integer optimization
- On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming
- Operations research and data mining
- Optimal classification trees
- Optimal decision trees for categorical data via integer programming
- Optimal randomized classification trees
- Optimization approaches to supervised classification
- Pyomo -- optimization modeling in Python
- Random forests
- Supersparse linear integer models for optimized medical scoring systems
- Supervised classification and mathematical optimization
- The \(F_{\infty}\)-norm support vector machine
- Using neural network rule extraction and decision tables for credit-risk evaluation
Cited in
(23)- Sparse projection oblique randomer forests
- Proximal variable metric method with spectral diagonal update for large scale sparse optimization
- Mathematical optimization in classification and regression trees
- On mathematical optimization for clustering categories in contingency tables
- On sparse ensemble methods: an application to short-term predictions of the evolution of COVID-19
- A pivot-based simulated annealing algorithm to determine oblique splits for decision tree induction
- scientific article; zbMATH DE number 7625179 (Why is no real title available?)
- Treelets -- an adaptive multi-scale basis for sparse unordered data
- Optimal randomized classification trees
- On multivariate randomized classification trees: \(l_0\)-based sparsity, VC dimension and decomposition methods
- Loss-optimal classification trees: a generalized framework and the logistic case
- On optimal regression trees to detect critical intervals for multivariate functional data
- Sparse regression over clusters: SparClur
- Optimization problems for machine learning: a survey
- Convergence Rates of Oblique Regression Trees for Flexible Function Libraries
- Margin optimal classification trees
- Discussion of: Treelets -- an adaptive multi-scale basis for sparse unordered data
- The backbone method for ultra-high dimensional sparse machine learning
- Interpretable clustering via soft clustering trees
- Optimal multivariate decision trees
- On constrained smoothing and out-of-range prediction using \(P\)-splines: a conic optimization approach
- On sparse optimal regression trees
- Oblique decision tree induction by cross-entropy optimization based on the von Mises-Fisher distribution
This page was built for publication: Sparsity in optimal randomized classification trees
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2301963)
