The AdaBoost flow

DOI10.1002/CPA.21555MaRDI QIDQ5247411zbMATH OpenOpenAlexFDO

Authors Stepan Muzychka, Kirill Vaninsky, A. A. Lykov

Publication date 24 April 2015

Published in Communications on Pure and Applied Mathematics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1110.6228

Mathematics Subject Classification ID

Dynamics induced by flows and semiflows (37C10) Learning and adaptive systems in artificial intelligence (68T05)

Abstract: We introduce a dynamical system which we call the AdaBoost flow. The flow is defined by a system of ODEs with control. We show that three algorithms of the AdaBoost family (i) the AdaBoost algorithm of Schapire and Freund (ii) the arc-gv algorithm of Breiman (iii) the confidence rated prediction of Schapire and Singer can be can be embedded in the AdaBoost flow. The nontrivial part of the AdaBoost flow equations coincides with the equations of dynamics of nonperiodic Toda system written in terms of spectral variables. We provide a novel invariant geometrical description of the AdaBoost algorithm as a gradient flow on a foliation defined by level sets of the potential function. We propose a new approach for constructing boosting algorithms as a continuous time gradient flow on measures defined by various metrics and potential functions. Finally we explain similarity of the AdaBoost algorithm with the Perelman's construction for the Ricci flow.

Recommendations

• The dynamics of adaboost: cyclic behavior and convergence of margins
• scientific article; zbMATH DE number 5251641
• An adaptive version of the boost by majority algorithm
• Parallelizing AdaBoost by weights dynamics
• Logistic regression, AdaBoost and Bregman distances

Cites work

• A decision-theoretic generalization of on-line learning and an application to boosting
• Finitely many mass points on the line under the influence of an exponential potential -- an integrable system
• Greedy function approximation: A gradient boosting machine.
• scientific article; zbMATH DE number 67633 (Why is no real title available?)
• scientific article; zbMATH DE number 1560711 (Why is no real title available?)
• scientific article; zbMATH DE number 3998602 (Why is no real title available?)
• scientific article; zbMATH DE number 3103174 (Why is no real title available?)
• Improved boosting algorithms using confidence-rated predictions
• The Atiyah-Hitchin bracket and the open Toda lattice.
• The entropy formula for the Ricci flow and its geometric applications
• Theory of nonlinear lattices.

Cited in

(2)

• Process consistency for AdaBoost.
• A large-sample theory for infinitesimal gradient boosting

This page was built for publication: The AdaBoost flow

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5247411)