Learning entropy: multiscale measure for incremental learning

DOI10.3390/E15104159MaRDI QIDQ280617zbMATH OpenOpenAlexFDO

Authors Ivo Bukovsky

Publication date 10 May 2016

Published in Entropy (Search for Journal in Brave)

Full work available at URL https://doi.org/10.3390/e15104159

zbMATH Keywords

adaptation plotapproximate learning entropychaosECGHRVincremental learningindividual sample learning entropylearning entropylearning entropy of a modelmultiscalenon-Shannon entropynovelty detectionorder of learning entropytime series

Mathematics Subject Classification ID

Time series analysis of dynamical systems (37M10) Learning and adaptive systems in artificial intelligence (68T05) Measures of information, entropy (94A17)

Review text

Summary: First, this paper recalls a recently introduced method of adaptive monitoring of dynamical systems and presents the most recent extension with a multiscale-enhanced approach. Then, it is shown that this concept of real-time data monitoring establishes a novel non-Shannon and non-probabilistic concept of novelty quantification, i.e., Entropy of Learning, or in short the Learning Entropy. This novel cognitive measure can be used for evaluation of each newly measured sample of data, or even of whole intervals. The Learning Entropy is quantified in respect to the inconsistency of data to the temporary governing law of system behavior that is incrementally learned by adaptive models such as linear or polynomial adaptive filters or neural networks. The paper presents this novel concept on the example of gradient descent learning technique with normalized learning rate.

Recommendations

• Teleonomic entropy: Measuring the phase-space of end-directed systems
• Beyond second-order statistics for learning: a pairwise interaction model for entropy estima\-tion
• Active and passive learning in control processes application of the entropy concept
• scientific article; zbMATH DE number 1928635
• What we learn from the learning rate

Cites work

• A survey of design methods for failure detection in dynamic systems
• Approximate entropy as a measure of system complexity.
• Fault diagnosis in dynamic systems using analytical and knowledge-based redundancy - a survey and some new results
• Function based fault detection for uncertain multivariate nonlinear non-Gaussian stochastic systems using entropy optimization principle
• Hyperchaotic behaviour of two bi-directionally coupled Chua's circuits
• Incipient fault diagnosis of dynamical systems using online approximators
• Learning approach to nonlinear fault diagnosis: detectability analysis
• Novelty detection: a review. I. Statistical approaches
• Novelty detection: a review. II. Neural network based approaches
• Oscillation and chaos in physiological control systems
• Process fault detection based on modeling and estimation methods - a survey
• Time series analysis using composite multiscale entropy

Cited in

(5)

• Teleonomic entropy: Measuring the phase-space of end-directed systems
• AECID: asymmetric entropy for classifying imbalanced data
• A metric entropy bound is not sufficient for learnability
• Entropy-Based Learning of Compositional Models from Data
• Novelty detection based on learning entropy

This page was built for publication: Learning entropy: multiscale measure for incremental learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q280617)