Sophisticated inference
From MaRDI portal
Abstract: Active inference offers a first principle account of sentient behaviour, from which special and important cases can be derived, e.g., reinforcement learning, active learning, Bayes optimal inference, Bayes optimal design, etc. Active inference resolves the exploitation-exploration dilemma in relation to prior preferences, by placing information gain on the same footing as reward or value. In brief, active inference replaces value functions with functionals of (Bayesian) beliefs, in the form of an expected (variational) free energy. In this paper, we consider a sophisticated kind of active inference, using a recursive form of expected free energy. Sophistication describes the degree to which an agent has beliefs about beliefs. We consider agents with beliefs about the counterfactual consequences of action for states of affairs and beliefs about those latent states. In other words, we move from simply considering beliefs about 'what would happen if I did that' to 'what would I believe about what would happen if I did that'. The recursive form of the free energy functional effectively implements a deep tree search over actions and outcomes in the future. Crucially, this search is over sequences of belief states, as opposed to states per se. We illustrate the competence of this scheme, using numerical simulations of deep decision problems.
Recommendations
Cites work
- A Cognitive Hierarchy Model of Games
- A minimum relative entropy principle for learning and acting
- Active inference on discrete state-spaces: a synthesis
- Active inference, curiosity and insight
- Active inference: a process theory
- Bayes Factors
- Bayesian reinforcement learning: a survey
- Constructing Free-Energy Approximations and Generalized Belief Propagation Algorithms
- Deep active inference
- Deep active inference as variational policy gradients
- Deep exploration via randomized value functions
- Factorial hidden Markov models
- Generalised free energy and active inference
- scientific article; zbMATH DE number 2061729 (Why is no real title available?)
- scientific article; zbMATH DE number 3106184 (Why is no real title available?)
- Information geometry on complexity and stochastic interaction
- Information Theory and Statistical Mechanics
- Markov blankets, information geometry and stochastic thermodynamics
- On a Measure of the Information Provided by an Experiment
- On the Theory of Dynamic Programming
- Online planning algorithms for POMDPS
- Optimal control of Markov processes with incomplete state information
- Planning and navigation as active inference
- Predictability, complexity, and learning
- Prospect Theory: An Analysis of Decision under Risk
- Reinforcement learning. An introduction
- Risk-sensitive control and an optimal investment model. II.
- Stochastic Reasoning, Free Energy, and Information Geometry
- The basics of information geometry
- Variational message passing
Cited in
(11)- Deep active inference
- Retrospective surprise: a computational component for active inference
- Deeply felt affect: the emergence of valence in deep active inference
- Whence the expected free energy?
- Active inference, curiosity and insight
- Active sensing with artificial neural networks
- The free energy principle made simpler but not too simple
- Reward Maximization Through Discrete Active Inference
- Branching time active inference: the theory and its generality
- A Bayesian theory of mind approach to modeling cooperation and communication
- Quantifying imperfect cognition via achieved information gain
This page was built for publication: Sophisticated inference
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5004320)
