Controlling a Stochastic Process with Unknown Parameters
discountinginfinite horizonlearning control problemslimit distributionsequence of beliefsstochastic process with unknown parameters
Asymptotic distribution theory in statistics (62E20) Applications of statistics to economics (62P20) Learning and adaptive systems in artificial intelligence (68T05) Management decision making, including multiple objectives (90B50) Decision theory (91B06) Discrete-time control/observation systems (93C55) Optimal stochastic control (93E20)
The problem of controlling a stochastic process with unknown parameters over an infinite horizon with discounting is considered. The derivations given are mathematically rigorous. The decisionmaker or agent faces a tradeoff between current reward and information which may be useful in the future. The emphasis is on the behaviour of the sequence of beliefs, i.e., how much learning occurs when the agent acts optionally. The paper starts with a brief overview of some existing literature on estimation and learning control problems; the bandit problem is a related problem in that the action space (the arms) is discrete, whereas in the current paper it may be continuous. Section 2 describes the decision problem in mathematical terms; it is discrete time. The underlying distribution function \(\mu\) which governs the evolution in time depends on a parameter \(\theta\) which is initially not known to the decisionmaker. The section concludes with a number of technical assumptions. In section 3 the functional equation for the value function is derived, using the dynamic programming approach, with beliefs (the prior distributions) as state variables. In section 4 the limit between behaviour of the sequence of beliefs \((\mu_ i\), \(i=1,2,...)\) is investigated. Convergence to a limit distribution is almost always assured. The problem is, however, that the limit distribution is not necessarily the true distribution, which is clearly indicated in sections 5 and 6. Section 5 deals with special cases such as one parameter problems and low discount rates. For a low discount rate certain convexity results can be proved. Section 6 deals with an example which clearly shows the features described above. For certain decisions the beliefs turn out to be invariant.
- Optimal growth and uncertainty: Learning
- A value function arising in the economics of information
- On the convergence of Bayesian posterior processes in linear economic models. Counting equations and unknowns
- Learning in mis-specified models and the possibility of cycles
- Stochastic dominance under Bayesian learning
- Bayesian learning in normal form games
- Evolution and market behavior
- Active learning. Monte Carlo results
- Risk, uncertainty, and complexity
- Computability, complexity and economics
- Experimentation and learning with network effects
- On the complexity of linear quadratic control
- On the generic nonconvergence of Bayesian actions and beliefs
- Bayesian learning leads to correlated equilibria in normal form games
- Dynamic duopoly with learning through market experimentation
- Learning from others: A welfare analysis
- Experimentation and competition
- Entry and exit under demand uncertainty
- Learning and control in a changing economic environment.
- Learning by doing and the value of optimal experimentation
- Herding with costly information
- Entry and vertical differentiation
- Another look at the Radner--Stiglitz nonconcavity in the value of information.
- Convergence of Bayesian learning to general equilibrium in mis-specified models.
- Optimal learning with costly adjustment
- Bayesian learning behaviour and the stability of equilibrium forecasts
- Asymptotic behavior of Bayesian learners with misspecified models
- Cultural transmission with incomplete information
- A framework for the analysis of self-confirming policies
- Learning and self-confirming long-run biases
- Bayesian economists \dots Bayesian agents. An alternative approach to optimal learning
- Equilibrium learning in simple contests
- On learning and growth
- On the potential use of adaptive control methods for improving adaptive natural resource management
- Bayes' learning of unknown parameters
- THE ECONOMIC VALUE OF ENVIRONMENTAL RESEARCH IN UNDERSTANDING THE RELATIVE CONTRIBUTIONS OF SOURCES OF NUTRIENTS TO COASTAL WATERS
- Maximum Likelihood Estimation of Discrete Control Processes
- Optimal Control of an Unknown Linear Process with Learning
- Solutions to a class of nonstandard stochastic control problems with active learning
- scientific article; zbMATH DE number 17342 (Why is no real title available?)
- Optimal control under uncertainty and Bayesian parameters adjustments
- Learning in society
- Computation of expected shortfall by fast detection of worst scenarios
- Dynamic learning and decision making via basis weight vectors
- Selling a lemon under demand uncertainty
- A dynamic migration model with uncertainty
- A dynamic migration model with uncertainty
- Imperfect information, Bayesian learning, and capital accumulation
- Robust monetary policy with misspecified models: Does model uncertainty always call for attenuated policy?
- Managing disinflation under uncertainty
- Nonparametric learning for impulse control problems -- exploration vs. exploitation
- The dynamics of efficient asset trading with heterogeneous beliefs
- Learning about the arrival of sales
- Optimal control under uncertainty: application to the issue of CAT bonds
- Learning about variable demand in the long run
- Nonconcave stochastic optimal control in finite discrete time under model uncertainty
- Dynamic concern for misspecification
- Periodic learning about a hidden state variable
- Learning underspecified models
- Experimentation with accumulation
- Optimal experimentation and the perturbation method in the neighborhood of the augmented linear regulator problem
- Stochastic control for economic models: past, present and the paths ahead
- Good news and bad news in two-armed bandits
This page was built for publication: Controlling a Stochastic Process with Unknown Parameters
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3808001)