Continuity of cost in Borkar control topology and implications on discrete space and time approximations for controlled diffusions under several criteria
From MaRDI portal
(Redirected from Publication:6126973)
Abstract: We first show that the discounted cost, cost up to an exit time, and ergodic cost involving controlled non-degenerate diffusions are continuous on the space of stationary control policies when the policies are given a topology introduced by Borkar [V. S. Borkar, A topology for Markov controls, Applied Mathematics and Optimization 20 (1989), 55-62]. The same applies for finite horizon problems when the control policies are markov and the topology is revised to include time also as a parameter. We then establish that finite action/piecewise constant stationary policies are dense in the space of stationary Markov policies under this topology. Using the above mentioned continuity and denseness results we establish that finite action/piecewise constant policies approximate optimal stationary policies with arbitrary precision. This gives rise to the applicability of many numerical methods such as policy iteration and stochastic learning methods for discounted cost, cost up to an exit time, and ergodic cost optimal control problems in continuous-time. For the finite-horizon setup, we establish additionally near optimality of time-discretized policies by an analogous argument. We thus present a unified and concise approach for approximations directly applicable under several commonly adopted cost criteria.
Recommendations
- Near optimality of Lipschitz and smooth policies in controlled diffusions
- Nonstationary continuous-time Markov control processes with discounted costs on infinite horizon
- Approximation of infinite horizon discounted cost Markov decision processes
- Numerical Methods for Stochastic Control Problems in Continuous Time
- On Borkar and Young relaxed control topologies and continuous dependence of invariant measures on control policy
Cites work
- A partial history of the early development of continuous-time nonlinear stochastic systems theory
- A remark on the attainable distributions of controlled diffusions
- A topology for Markov controls
- An extension of Tietze's theorem
- Approximate Q Learning for Controlled Diffusion Processes and Its Near Optimality
- Approximating value functions for controlled degenerate diffusion processes by using piece-wise constant policies.
- Asymptotic Optimality of Finite Model Approximations for Partially Observed Markov Decision Processes With Discounted Cost
- Controlled diffusion processes
- Controlled diffusions with constraints
- Convergence of discretization procedures in dynamic programming
- Convex analytic method revisited: further optimality results and performance of deterministic policies in average cost stochastic control
- Ergodic control of diffusion processes.
- Ergodic Control of Multidimensional Diffusions I: The Existence Results
- Ergodic control of multidimensional diffusions. II: Adaptive control
- Error Bounds for Monotone Approximation Schemes for Hamilton--Jacobi--Bellman Equations
- Finite approximations in discrete-time stochastic control. Quantized models and asymptotic optimality
- Finite linear programming approximations of constrained discounted Markov decision processes
- scientific article; zbMATH DE number 1577097 (Why is no real title available?)
- scientific article; zbMATH DE number 1631027 (Why is no real title available?)
- scientific article; zbMATH DE number 3167340 (Why is no real title available?)
- scientific article; zbMATH DE number 42423 (Why is no real title available?)
- scientific article; zbMATH DE number 3491650 (Why is no real title available?)
- scientific article; zbMATH DE number 1166343 (Why is no real title available?)
- scientific article; zbMATH DE number 7625164 (Why is no real title available?)
- scientific article; zbMATH DE number 3895476 (Why is no real title available?)
- scientific article; zbMATH DE number 3262637 (Why is no real title available?)
- Improved order 1/4 convergence for piecewise constant policy approximation of stochastic control problems
- Mean value theorems for stochastic integrals
- Multidimensional diffusion processes.
- Near optimality of quantized policies in stochastic control under weak continuity conditions
- Numerical Approximations for Stochastic Differential Games
- Occupation measures for controlled Markov processes: Characterization and optimality
- On the asymptotic optimality of finite approximations to Markov decision processes with Borel spaces
- On the convergence rate of approximation schemes for Hamilton-Jacobi-Bellman Equations
- On the rate of convergence of finite-difference approximations for Bellman's equations with variable coefficients
- Optimal control of diffusion processes and hamilton–jacobi–bellman equations part 2 : viscosity solutions and uniqueness
- Optimal control of diffustion processes and hamilton-jacobi-bellman equations part I: the dynamic programming principle and application
- Performance Loss Bounds for Approximate Value Iteration with State Aggregation
- Piecewise constant policy approximations to Hamilton-Jacobi-Bellman equations
- Probability methods for approximations in stochastic control and for elliptic equations
- Real Analysis and Probability
- Uniform recurrence properties of controlled diffusions and applications to optimal control
Cited in
(5)- On Borkar and Young relaxed control topologies and continuous dependence of invariant measures on control policy
- Robust decentralized control of coupled systems via risk sensitive control of decoupled or simple models with measure change
- Near optimality of Lipschitz and smooth policies in controlled diffusions
- Towards an analytical framework for dynamic potential games
- Robustness of optimal controlled diffusions with near-Brownian noise via rough paths theory
This page was built for publication: Continuity of cost in Borkar control topology and implications on discrete space and time approximations for controlled diffusions under several criteria
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6126973)
