Sparsity in optimal randomized classification trees
From MaRDI portal
Abstract: Decision trees are popular Classification and Regression tools and, when small-sized, easy to interpret. Traditionally, a greedy approach has been used to build the trees, yielding a very fast training process; however, controlling sparsity (a proxy for interpretability) is challenging. In recent studies, optimal decision trees, where all decisions are optimized simultaneously, have shown a better learning performance, especially when oblique cuts are implemented. In this paper, we propose a continuous optimization approach to build sparse optimal classification trees, based on oblique cuts, with the aim of using fewer predictor variables in the cuts as well as along the whole tree. Both types of sparsity, namely local and global, are modeled by means of regularizations with polyhedral norms. The computational experience reported supports the usefulness of our methodology. In all our data sets, local and global sparsity can be improved without harming classification accuracy. Unlike greedy approaches, our ability to easily trade in some of our classification accuracy for a gain in global sparsity is shown.
Recommendations
Cites work
- A random forest guided tour
- Auction optimization using regression trees and linear models as integer programs
- Comprehensible credit scoring models using rule extraction from support vector machines
- Constructing optimal binary decision trees is NP-complete
- Do we need hundreds of classifiers to solve real world classification problems?
- scientific article; zbMATH DE number 3860199 (Why is no real title available?)
- scientific article; zbMATH DE number 6438182 (Why is no real title available?)
- Learning decision trees with flexible constraints and objectives using integer optimization
- On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming
- Operations research and data mining
- Optimal classification trees
- Optimal decision trees for categorical data via integer programming
- Optimal randomized classification trees
- Optimization approaches to supervised classification
- Pyomo -- optimization modeling in Python
- Random forests
- Supersparse linear integer models for optimized medical scoring systems
- Supervised classification and mathematical optimization
- The \(F_{\infty}\)-norm support vector machine
- Using neural network rule extraction and decision tables for credit-risk evaluation
Cited in
(30)- Optimization problems for machine learning: a survey
- Oblique decision tree induction by cross-entropy optimization based on the von Mises-Fisher distribution
- On constrained smoothing and out-of-range prediction using P-splines: a conic optimization approach
- Sparse regression over clusters: SparClur
- The backbone method for ultra-high dimensional sparse machine learning
- On sparse ensemble methods: an application to short-term predictions of the evolution of COVID-19
- Treelets -- an adaptive multi-scale basis for sparse unordered data
- Optimal randomized classification trees
- On sparse optimal regression trees
- Sparse projection oblique randomer forests
- scientific article; zbMATH DE number 7625179 (Why is no real title available?)
- Discussion of: Treelets -- an adaptive multi-scale basis for sparse unordered data
- Margin optimal classification trees
- Interpretable clustering via soft clustering trees
- Proximal variable metric method with spectral diagonal update for large scale sparse optimization
- On mathematical optimization for clustering categories in contingency tables
- On multivariate randomized classification trees: \(l_0\)-based sparsity, VC dimension and decomposition methods
- On optimal regression trees to detect critical intervals for multivariate functional data
- Convergence Rates of Oblique Regression Trees for Flexible Function Libraries
- Optimal multivariate decision trees
- A pivot-based simulated annealing algorithm to determine oblique splits for decision tree induction
- Loss-optimal classification trees: a generalized framework and the logistic case
- Soft regression trees: a model variant and a decomposition training algorithm
- Rule generation for classification: scalability, interpretability, and fairness
- k-tree: crossing sharp boundaries in regression trees to find neighbors
- On enhancing the explainability and fairness of tree ensembles
- Interpretable optimisation-based approach for hyper-box classification
- Soft decision trees for survival analysis
- On contextual inverse multiobjective problems
- Mathematical optimization in classification and regression trees
This page was built for publication: Sparsity in optimal randomized classification trees
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2301963)
