Numerical comparison of controls and verification of optimality for stochastic control problems
The purpose of this paper is to provide a numerical method for comparing the efficiency of various controls and to present a verification approach to the solution to stochastic control problems. A Linear Programming (LP) formulation involving the moments of the stationary distribution of the process is used. Two approaches were adopted. The first was to specify the control for the process and run the LP to obtain various characteristics of the controlled process including the cost associated with the process. Doing this with several controls allows a comparison of the costs. The second approach was to specify a candidate for the optimal control and derive a modified LP which all controlled processes satisfied and which the candidate control satisfied with tightness for particular constraints. This differs from dynamic programming in which one guesses the value function, verifies that it is a solution to the Hamilton-Jacobi-Bellman equation, and as a consequence determines the optimal control. For numerical computation, the problem is reduced to finite dimensions by considering only a finite number of moments. The proposed approach deals directly with the stochastic process of the control problem rather than with some approximating process.
- Determining the Optimal Control of Singular Stochastic Processes Using Linear Programming
- Numerical Methods for Stochastic Control Problems in Continuous Time
- Approximation of Infinite-Dimensional Linear Programming Problems which Arise in Stochastic Control
- Some Analytic Aspects of the Linear-Programming Approach to the Numerical Solution of Singular Stochastic Control Problems
- scientific article; zbMATH DE number 21025
- A Linear Programming Problem Connected with Optimal Stationary Control in a Dynamic Decision Problem
- Approximation of Infinite-Dimensional Linear Programming Problems which Arise in Stochastic Control
- Average cost Markov decision processes: Optimality conditions
- Existence of Markov Controls and Characterization of Optimal Markov Controls
- scientific article; zbMATH DE number 3951715 (Why is no real title available?)
- scientific article; zbMATH DE number 158461 (Why is no real title available?)
- scientific article; zbMATH DE number 3505708 (Why is no real title available?)
- scientific article; zbMATH DE number 765034 (Why is no real title available?)
- scientific article; zbMATH DE number 3059214 (Why is no real title available?)
- Linear programming and sequential decisions
- Linear Programming in a Markov Chain
- Occupation measures for controlled Markov processes: Characterization and optimality
- On Linear Programming in a Markov Decision Problem
- On sequential decisions and Markov chains
- Some solvable stochastic control problemst†
- Time-average control of martingale problems: A linear programming formulation
- Time-average control of martingale problems: Existence of a stationary solution
- The problem of moments on polytopes and other bounded regions.
- Numerical solution of a long-term average control problem for singular stochastic processes
- Bounding stationary averages of polynomial diffusions via semidefinite programming
- Dynamic programming or direct comparison?
- Analysis of production decisions under budget limitations
- Solving the drift control problem
- Determining the Optimal Control of Singular Stochastic Processes Using Linear Programming
- Extension Of Dale's Moment Conditions With Application To The Wright–fisher Model
This page was built for publication: Numerical comparison of controls and verification of optimality for stochastic control problems
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1586818)
