Optimal control with learning on the fly: a toy problem

DOI10.4171/RMI/1275MaRDI QIDQ832436zbMATH OpenOpenAlexFDO

Authors Bernat Guillén Pegueroles, Clarence W. Rowley, Melanie Weber, Charles Fefferman

Publication date 25 March 2022

Published in Revista Matemática Iberoamericana (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2002.11578

zbMATH Keywords

adaptive controlcompetitive ratioregretagnostic controlfuel tax regret

Mathematics Subject Classification ID

Optimal stochastic control (93E20)

Abstract: We exhibit optimal control strategies for a simple toy problem in which the underlying dynamics depend on a parameter that is initially unknown and must be learned. We consider a cost function posed over a finite time interval, in contrast to much previous work that considers asymptotics as the time horizon tends to infinity. We study several different versions of the problem, including Bayesian control, in which we assume a prior distribution on the unknown parameter; and "agnostic" control, in which we assume nothing about the unknown parameter. For the agnostic problems, we compare our performance with that of an opponent who knows the value of the parameter. This comparison gives rise to several notions of "regret," and we obtain strategies that minimize the "worst-case regret" arising from the most unfavorable choice of the unknown parameter. In every case, the optimal strategy turns out to be a Bayesian strategy or a limit of Bayesian strategies.

Recommendations

• Controlling unknown linear dynamics with bounded multiplicative regret
• Optimal Control of an Unknown Linear Process with Learning
• Learning and control in a changing economic environment.
• Bayes' learning of unknown parameters
• scientific article; zbMATH DE number 17342

Cites work

• Approximate dynamic programming. Solving the curses of dimensionality
• Asymptotically efficient adaptive allocation rules
• Dynamic programming and optimal control. Vol. 1.
• Finite-time analysis of the multiarmed bandit problem
• Prediction, Learning, and Games
• Regret analysis of stochastic and nonstochastic multi-armed bandit problems

Cited in

(11)

• Controlling unknown linear dynamics with bounded multiplicative regret
• Optimal Learning with Endogenous Data
• Bayes' learning of unknown parameters
• Optimal control under uncertainty and Bayesian parameters adjustments
• On incomplete learning and certainty-equivalence control
• Optimal policies for passive learning controllers
• Optimal Control of an Unknown Linear Process with Learning
• Learning and control in a changing economic environment.
• Learning optimal control in deterministic systems
• THE OPTIMAL SOLUTION OF THE LOB-PASS PROBLEM WITH KNOWN REACTION CURVES
• scientific article; zbMATH DE number 17342 (Why is no real title available?)

This page was built for publication: Optimal control with learning on the fly: a toy problem

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q832436)