What we learn from the learning rate
From MaRDI portal
Abstract: The learning rate is an information-theoretical quantity for bipartite Markov chains describing two coupled subsystems. It is defined as the rate at which transitions in the downstream subsystem tend to increase the mutual information between the two subsystems, and is bounded by the dissipation arising from these transitions. Its physical interpretation, however, is unclear, although it has been used as a metric for the sensing performance of the downstream subsystem. In this paper, we explore the behaviour of the learning rate for a number of simple model systems, establishing when and how its behaviour is distinct from the instantaneous mutual information between subsystems. In the simplest case, the two are almost equivalent. In more complex steady-state systems, the mutual information and the learning rate behave qualitatively distinctly, with the learning rate clearly now reflecting the rate at which the downstream system must update its information in response to changes in the upstream system. It is not clear whether this quantity is the most natural measure for sensor performance, and, indeed, we provide an example in which optimising the learning rate over a region of parameter space of the downstream system yields an apparently sub-optimal sensor.
Recommendations
- Rate of mutual information between coarse-grained non-Markovian variables
- Critical behavior of entropy production and learning rate: Ising model with an oscillating field
- Efficiency of cellular information processing
- Predictability, complexity, and learning
- Fluctuation-Dissipation Theorem and Models of Learning
Cites work
- A Mathematical Theory of Communication
- Critical behavior of entropy production and learning rate: Ising model with an oscillating field
- Efficiency of a Brownian information machine
- Efficiency of cellular information processing
- Elements of Information Theory
- Fundamental limits to cellular sensing
- Irreversibility and Heat Generation in the Computing Process
- Multipartite information flow for multiple Maxwell demons
- Notes on Landauer's principle, reversible computation, and Maxwell's demon
- Stochastic thermodynamics of bipartite systems: transfer entropy inequalities and a Maxwell’s demon interpretation
- Thermodynamic limits to information harvesting by sensory systems
- Trade-offs in delayed information transmission in biochemical networks
Cited in
(5)- Prediction and dissipation in nonequilibrium molecular sensors: conditionally Markovian channels driven by memoryful environments
- Learning entropy: multiscale measure for incremental learning
- Critical behavior of entropy production and learning rate: Ising model with an oscillating field
- Efficiency of cellular information processing
- A fractional rate model of learning
This page was built for publication: What we learn from the learning rate
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3303100)
