Optimal control with learning on the fly: a toy problem

DOI10.4171/RMI/1275zbMATH Open1485.93628arXiv2002.11578OpenAlexW3177406382MaRDI QIDQ832436FDOQ832436

Authors: Bernat Guillén Pegueroles, Clarence W. Rowley, Melanie Weber, Charles Fefferman 

Publication date: 25 March 2022

Published in: Revista Matemática Iberoamericana (Search for Journal in Brave)

Abstract: We exhibit optimal control strategies for a simple toy problem in which the underlying dynamics depend on a parameter that is initially unknown and must be learned. We consider a cost function posed over a finite time interval, in contrast to much previous work that considers asymptotics as the time horizon tends to infinity. We study several different versions of the problem, including Bayesian control, in which we assume a prior distribution on the unknown parameter; and "agnostic" control, in which we assume nothing about the unknown parameter. For the agnostic problems, we compare our performance with that of an opponent who knows the value of the parameter. This comparison gives rise to several notions of "regret," and we obtain strategies that minimize the "worst-case regret" arising from the most unfavorable choice of the unknown parameter. In every case, the optimal strategy turns out to be a Bayesian strategy or a limit of Bayesian strategies.

Full work available at URL: https://arxiv.org/abs/2002.11578

Recommendations

• Controlling unknown linear dynamics with bounded multiplicative regret
• Optimal Control of an Unknown Linear Process with Learning
• Learning and control in a changing economic environment.
• Bayes' learning of unknown parameters
• scientific article; zbMATH DE number 17342

zbMATH Keywords

adaptive controlcompetitive ratioregretagnostic controlfuel tax regret

Mathematics Subject Classification ID

Optimal stochastic control (93E20)

Cites Work

• Prediction, Learning, and Games
• Asymptotically efficient adaptive allocation rules
• Finite-time analysis of the multiarmed bandit problem
• Approximate dynamic programming. Solving the curses of dimensionality
• Dynamic programming and optimal control. Vol. 1.
• Regret analysis of stochastic and nonstochastic multi-armed bandit problems

Cited In (11)

• Optimal Learning with Endogenous Data
• On incomplete learning and certainty-equivalence control
• Learning optimal control in deterministic systems
• Title not available (Why is that?)
• Learning and control in a changing economic environment.
• THE OPTIMAL SOLUTION OF THE LOB-PASS PROBLEM WITH KNOWN REACTION CURVES
• Controlling unknown linear dynamics with bounded multiplicative regret
• Optimal control under uncertainty and Bayesian parameters adjustments
• Optimal policies for passive learning controllers
• Bayes' learning of unknown parameters
• Optimal Control of an Unknown Linear Process with Learning

This page was built for publication: Optimal control with learning on the fly: a toy problem

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q832436)