Existence of Markov Controls and Characterization of Optimal Markov Controls
From MaRDI portal
Consider a controlled martingale problem under the requirement that \[ f(X(t))- \int^t_0 Af(X(s), u(s)) ds \] be a martingale for \(f\in{\mathcal D}(A)\), where \(u\) is the control process. The existenc of an optimal Markov control has been proved by using the Krylov selection theorem, which, however, does not characterize the control [cf. \textit{U. G. Haussmann} and \textit{J. P. Lepeltier}, SIAM J. Control Optimization 28, 851-902 (1990; Zbl 0712.49013)]. In this paper, under general conditions for the generator \(A\), it is shown that there exists a so-called relaxed solution having Markov controls which has the same cost as the original solution. It is then shown that the original control problem can be reformulated as a linear programming (LP) problem over an appropriate space of measures. With this LP approach, characterization as well as existence of optimal Markov controls can directly follow. The equivalent LP formulation (and expressions for minimization) under a variety of cost criteria is developed in detail.
Recommendations
- Time-average control of martingale problems: A linear programming formulation
- On the existence of strict optimal controls for constrained, controlled Markov processes in continuous time
- Publication:3199307
- Occupation measures for controlled Markov processes: Characterization and optimality
- scientific article; zbMATH DE number 4047561
Cited in
(76)- A separation principle for partially observed control of singular stochastic processes
- Variance minimization and the overtaking optimality approach to continuous-time controlled Markov chains
- Genealogical processes for Fleming-Viot models with selection and recombination
- Numerical comparison of controls and verification of optimality for stochastic control problems
- Controlled equilibrium selection in stochastically perturbed dynamics
- On uniqueness of solutions for the stochastic differential equations of nonlinear filtering
- Some applications of linear programming formulations in stochastic control
- Non-explosivity of stochastically modeled reaction networks that are complex balanced
- Linear programming approach to optimal impulse control problems with functional constraints
- On average control generating families for singularly perturbed optimal control problems with long run average optimality criteria
- On the Ambrosio-Figalli-Trevisan superposition principle for probability solutions to Fokker-Planck-Kolmogorov equations
- Control and optimal stopping mean field games: a linear programming approach
- Mean field games with controlled jump-diffusion dynamics: existence results and an illiquid interbank market model
- Mean field games via controlled martingale problems: existence of Markovian equilibria
- Averaging and linear programming in some singularly perturbed problems of optimal control
- Linear programming formulations of deterministic infinite horizon optimal control problems in discrete time
- Linear programming formulation of long-run average optimal control problem
- Discussion of dynamic programming and linear programming approaches to stochastic control and optimal stopping in continuous time
- Numerical solution of a long-term average control problem for singular stochastic processes
- On characterisation of Markov processes via martingale problems
- Risk aggregation and stochastic claims reserving in disability insurance
- Occupation measures for controlled Markov processes: Characterization and optimality
- A partial history of the early development of continuous-time nonlinear stochastic systems theory
- Linear programming based optimality conditions and approximate solution of a deterministic infinite horizon discounted optimal control problem in discrete time
- Superposition principle for the Fokker-Planck-Kolmogorov equations with unbounded coefficients
- Use of approximations of Hamilton-Jacobi-Bellman inequality for solving periodic optimization problems
- Analysis of production decisions under budget limitations
- Impulsive control for continuous-time Markov decision processes: a linear programming approach
- SDP vs. LP Relaxations for the Moment Approach in Some Performance Evaluation Problems
- scientific article; zbMATH DE number 4174815 (Why is no real title available?)
- scientific article; zbMATH DE number 3882326 (Why is no real title available?)
- Linear programming approach to the optimal stopping of singular stochastic processes
- scientific article; zbMATH DE number 5290174 (Why is no real title available?)
- Controlled Heterogeneous Collection: The Role of Occupation Numbers
- Existence of optimal Markov solutions for ergodic control of Markov processes
- scientific article; zbMATH DE number 3940369 (Why is no real title available?)
- scientific article; zbMATH DE number 3945001 (Why is no real title available?)
- scientific article; zbMATH DE number 3954798 (Why is no real title available?)
- scientific article; zbMATH DE number 4047561 (Why is no real title available?)
- On sets of occupational measures generated by a deterministic control system on an infinite time horizon
- Markov marginal problems and their applications to Markov optimal control
- Large deviations for multiscale diffusion via weak convergence methods
- Convergence of finite element methods for singular stochastic control
- On the existence of strict optimal controls for constrained, controlled Markov processes in continuous time
- Infection time in multistable gene networks. A backward stochastic variational inequality with nonconvex switch-dependent reflection approach
- Representation formulas for limit values of long run stochastic optimal controls
- On the solution structure of infinite-dimensional linear problems stemming from singular stochastic control problems
- Computable Primal and Dual Bounds for Stochastic Control
- scientific article; zbMATH DE number 7307478 (Why is no real title available?)
- Asymptotically optimal dynamic pricing for network revenue management
- Two-armed restless bandits with imperfect information: stochastic control and indexability
- LP Formulations of Discrete Time Long-Run Average Optimal Control Problems: The NonErgodic Case
- Limit theory for controlled McKean-Vlasov dynamics
- Stochastic optimal control and linear programming approach
- Mimicking finite dimensional marginals of a controlled diffusion with jumps
- PRICING A CLASS OF EXOTIC OPTIONS VIA MOMENTS AND SDP RELAXATIONS
- Sequential entry and exit decisions with an ergodic performance criterion
- Stationary solutions and forward equations for controlled and singular martingale problems
- Linear programming fictitious play algorithm for mean field games with optimal stopping and absorption
- A dynamic analytic method for risk-aware controlled martingale problems
- Superposition and mimicking theorems for conditional McKean-Vlasov equations
- MF-OMO: An Optimization Formulation of Mean-Field Games
- Thinning and harvesting in stochastic forest models
- Utility indifference valuation for jump risky assets
- Theoretical guarantees for satisfaction of terminal state constraints for nonlinear stochastic systems
- Controlled martingale problems and their Markov mimics
- Energy transition under scenario uncertainty: a mean-field game of stopping with common noise
- Linear programming approach to optimal control problems with unbounded state constraint
- Moment-SOS hierarchy and exit location of stochastic processes
- A graphon mean field convex duality approach to shaping the terminal probability distribution of a network of nonlinear multi-agent systems
- Existence of optimal controls for stochastic Volterra equations
- Finite element approximations for stochastic control problems with unbounded state space
- Accuracy of discretely sampled stochastic policies in continuous-time reinforcement learning
- Existence of asymptotic values for nonexpansive stochastic control systems
- On the LP formulation in measure spaces of optimal control problems for jump-diffusions
- Time-average control of martingale problems: A linear programming formulation
This page was built for publication: Existence of Markov Controls and Characterization of Optimal Markov Controls
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4388926)
