Active online learning in the binary perceptron problem

DOI10.1088/0253-6102/71/2/243MaRDI QIDQ3387706zbMATH OpenOpenAlexWikidataFDO

Authors Haijun Zhou

Publication date 13 January 2021

Published in Communications in Theoretical Physics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1902.08043

zbMATH Keywords

neural networkstatistical inferenceonline learningperceptronbelief propagation

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Neural networks for/in biological studies, artificial life and related topics (92B20)

Abstract: The binary perceptron is the simplest artificial neural network formed by $N$ input units and one output unit, with the neural states and the synaptic weights all restricted to $pm1$ values. The task in the teacher--student scenario is to infer the hidden weight vector by training on a set of labeled patterns. Previous efforts on the passive learning mode have shown that learning from independent random patterns is quite inefficient. Here we consider the active online learning mode in which the student designs every new Ising training pattern. We demonstrate that it is mathematically possible to achieve perfect (error-free) inference using only $N$ designed training patterns, but this is computationally unfeasible for large systems. We then investigate two Bayesian statistical designing protocols, which require $2.3N$ and $1.9N$ training patterns, respectively, to achieve error-free inference. If the training patterns are instead designed through deductive reasoning, perfect inference is achieved using $N!+!lo{g}_2!N$ samples. The performance gap between Bayesian and deductive designing strategies may be shortened in future work by taking into account the possibility of ergodicity breaking in the version space of the binary perceptron.

Recommendations

• scientific article; zbMATH DE number 1569117
• Learning strategy for the binary perceptron
• On-line learning in the Ising perceptron.
• Learning and generalization errors for the 2D binary perceptron.
• Learning binary perceptrons perfectly efficiently.

Cites work

• scientific article; zbMATH DE number 1569117 (Why is no real title available?)
• scientific article; zbMATH DE number 3420728 (Why is no real title available?)
• Active inference, curiosity and insight
• Deep learning
• Entropy landscape of solutions in the binary perceptron problem
• Generalization errors of the simple perceptron
• Generalization learning in a perceptron with binary synapses
• Generalization performance of Bayes optimal classification algorithm for learning a perceptron
• Learning from correlated patterns by simple perceptrons
• On-line learning in the Ising perceptron.
• Optimal generalization in perceptions
• Optimization by simulated annealing
• Queries and concept learning
• Science and Statistics
• Some aspects of the sequential design of experiments
• Statistical mechanics approach to 1-bit compressed sensing
• Statistical mechanics of learning

Cited in

(3)

• scientific article; zbMATH DE number 1569117 (Why is no real title available?)
• Online active learning of decision trees with evidential data
• Learning Theory

This page was built for publication: Active online learning in the binary perceptron problem

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3387706)