Rationally inattentive control of Markov processes

DOI10.1137/15M1008476zbMATH Open1360.93785arXiv1502.03762MaRDI QIDQ2802080FDOQ2802080

Authors: Ehsan Shafieepoorfard, Maxim Raginsky, Sean P. Meyn 

Publication date: 25 April 2016

Published in: SIAM Journal on Control and Optimization (Search for Journal in Brave)

Abstract: The article poses a general model for optimal control subject to information constraints, motivated in part by recent work of Sims and others on information-constrained decision-making by economic agents. In the average-cost optimal control framework, the general model introduced in this paper reduces to a variant of the linear-programming representation of the average-cost optimal control problem, subject to an additional mutual information constraint on the randomized stationary policy. The resulting optimization problem is convex and admits a decomposition based on the Bellman error, which is the object of study in approximate dynamic programming. The theory is illustrated through the example of information-constrained linear-quadratic-Gaussian (LQG) control problem. Some results on the infinite-horizon discounted-cost criterion are also presented.

Full work available at URL: https://arxiv.org/abs/1502.03762

Recommendations

• Multivariate rational inattention
• Optimal information acquisition for a linear quadratic control problem
• Markov control with rare state observation: average optimality
• A general linear quadratic stochastic control and information value
• A linear-quadratic Gaussian approach to dynamic information acquisition

zbMATH Keywords

optimizationinformation theorystochastic controlMarkov decision processesobservation channels

Mathematics Subject Classification ID

Dynamic programming (90C39) Markov and semi-Markov decision processes (90C40) Stochastic systems in control theory (general) (93E03) Optimal stochastic control (93E20)

Cites Work

• Elements of Information Theory
• Markov Chains and Stochastic Stability
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• On general minimax theorems
• Title not available (Why is that?)
• Information acquisition and under-diversification
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Control Techniques for Complex Networks
• Title not available (Why is that?)
• Equivalent stochastic control problems
• Causal coding and control for Markov chains
• A convex analytic approach to Markov decision processes
• Markov chains and invariant probabilities
• Linear programming and sequential decisions
• Control Over Noisy Channels
• Stochastic Linear Control Over a Communication Channel
• Simultaneous design of measurement and control strategies for stochastic systems with feedback
• On the Structure of Real-Time Source Coders
• Linear Programming and Average Optimality of Markov Control Processes on Borel Spaces—Unbounded Costs
• Dual effect, certainty equivalence, and separation in stochastic control
• Title not available (Why is that?)
• Optimum design of measurement channels and control policies for linear- quadratic stochastic systems
• Passage to the Limit under the Information and Entropy Signs
• Markov control problems under communication constraints
• Functional Properties of Minimum Mean-Square Error and Mutual Information
• An Optimizer's Approach to Stochastic Control Problems With Nonclassical Information Structures
• Optimization and convergence of observation channels in stochastic control

Cited In (9)

• Ordinary differential equation methods for Markov decision processes and application to Kullback-Leibler control cost
• Optimal Control and Signaling Strategies of Control-Coding Capacity of General Decision Models: Applications to Gaussian Models and Decentralized Strategies
• Multivariate rational inattention
• Concordant informational control
• A linear-quadratic Gaussian approach to dynamic information acquisition
• Simultaneous perception-action design via invariant finite belief sets
• Bounded rationality and control
• Bounded rationality and control
• From infinite to finite programs: explicit error bounds with applications to approximate dynamic programming

This page was built for publication: Rationally inattentive control of Markov processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2802080)