Optimal control with learning on the fly: a toy problem

DOI10.4171/RMI/1275MaRDI QIDQ832436zbMATH OpenOpenAlexFDO

Authors Bernat Guillén Pegueroles, Clarence W. Rowley, Melanie Weber, Charles Fefferman

Publication date 25 March 2022

Published in Revista Matemática Iberoamericana (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2002.11578

zbMATH Keywords

adaptive controlagnostic controlcompetitive ratiofuel tax regretregret

Mathematics Subject Classification ID

Optimal stochastic control (93E20)

Abstract: We exhibit optimal control strategies for a simple toy problem in which the underlying dynamics depend on a parameter that is initially unknown and must be learned. We consider a cost function posed over a finite time interval, in contrast to much previous work that considers asymptotics as the time horizon tends to infinity. We study several different versions of the problem, including Bayesian control, in which we assume a prior distribution on the unknown parameter; and "agnostic" control, in which we assume nothing about the unknown parameter. For the agnostic problems, we compare our performance with that of an opponent who knows the value of the parameter. This comparison gives rise to several notions of "regret," and we obtain strategies that minimize the "worst-case regret" arising from the most unfavorable choice of the unknown parameter. In every case, the optimal strategy turns out to be a Bayesian strategy or a limit of Bayesian strategies.

Recommendations

• Controlling unknown linear dynamics with bounded multiplicative regret
• Optimal Control of an Unknown Linear Process with Learning
• Learning and control in a changing economic environment.
• Bayes' learning of unknown parameters
• scientific article; zbMATH DE number 17342

Cites work

• Approximate dynamic programming. Solving the curses of dimensionality
• Asymptotically efficient adaptive allocation rules
• Dynamic programming and optimal control. Vol. 1.
• Finite-time analysis of the multiarmed bandit problem
• Prediction, Learning, and Games
• Regret analysis of stochastic and nonstochastic multi-armed bandit problems

Cited in

(13)

• Learning and control in a changing economic environment.
• Controlling unknown linear dynamics with bounded multiplicative regret
• Bayes' learning of unknown parameters
• Optimal Control of an Unknown Linear Process with Learning
• scientific article; zbMATH DE number 17342 (Why is no real title available?)
• Learning optimal control in deterministic systems
• THE OPTIMAL SOLUTION OF THE LOB-PASS PROBLEM WITH KNOWN REACTION CURVES
• Optimal control under uncertainty and Bayesian parameters adjustments
• Optimal Learning with Endogenous Data
• On incomplete learning and certainty-equivalence control
• Controlling unknown linear dynamics with almost optimal regret
• Optimal agnostic control of unknown linear dynamics in a bounded parameter range
• Optimal policies for passive learning controllers

This page was built for publication: Optimal control with learning on the fly: a toy problem

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q832436)