Concept drift detection and adaptation with hierarchical hypothesis testing
From MaRDI portal
Publication:1738599
Abstract: A fundamental issue for statistical classification models in a streaming environment is that the joint distribution between predictor and response variables changes over time (a phenomenon also known as concept drifts), such that their classification performance deteriorates dramatically. In this paper, we first present a hierarchical hypothesis testing (HHT) framework that can detect and also adapt to various concept drift types (e.g., recurrent or irregular, gradual or abrupt), even in the presence of imbalanced data labels. A novel concept drift detector, namely Hierarchical Linear Four Rates (HLFR), is implemented under the HHT framework thereafter. By substituting a widely-acknowledged retraining scheme with an adaptive training strategy, we further demonstrate that the concept drift adaptation capability of HLFR can be significantly boosted. The theoretical analysis on the Type-I and Type-II errors of HLFR is also performed. Experiments on both simulated and real-world datasets illustrate that our methods outperform state-of-the-art methods in terms of detection precision, detection delay as well as the adaptability across different concept drift types.
Recommendations
- Data stream ensemble classification based on concept drift detection
- Tracking concept drift using a constrained penalized regression combiner
- Detecting concept change in dynamic data streams
- An efficient and sensitive decision tree approach to mining concept-drifting data streams
- Handling concept drift via model reuse
Cites work
- 10.1162/153244302760200704
- \(\lambda \)-perceptron: an adaptive classifier for data streams
- A survey on concept drift adaptation
- Advances in Artificial Intelligence – SBIA 2004
- Detection of abrupt changes: theory and application
- Evaluation methods and decision theory for classification of streaming data with temporal dependence
- scientific article; zbMATH DE number 3187232 (Why is no real title available?)
- Information theoretic learning. Renyi's entropy and kernel perspectives
- On evaluating stream learning algorithms
- Sequential analysis. Tests and confidence intervals
- Temporally adaptive estimation of logistic classifiers on data streams
Cited in
(10)- Tracking concept drift using a constrained penalized regression combiner
- Characterizing concept drift
- Detecting virtual concept drift of regressors without ground truth values
- CURIE: a cellular automaton for concept drift detection
- Concept drift detection via competence models
- A survey on concept drift adaptation
- Data stream ensemble classification based on concept drift detection
- Adversarial concept drift detection under poisoning attacks for robust data stream mining
- Unsupervised concept drift detection for time series on Riemannian manifolds
- A dynamic similarity weighted evolving fuzzy system for concept drift of data streams
This page was built for publication: Concept drift detection and adaptation with hierarchical hypothesis testing
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1738599)
