Controlled Markov processes and viscosity solutions
The aim of this book is to give a reasonably self-contained introduction to optimal stochastic control for continuous time Markov processes, from the point of view of dynamic programming, and to the theory of viscosity solutions. Particular attention is given to controlled Markov diffusion processes, whose dynamic programming equation becomes a nonlinear partial differential equation of second order, called Hamilton-Jacobi-Bellman (HJB) equation. Typically, the value function is not smooth enough to satisfy the HJB equation in a classical sense, but is the unique viscosity solution of it under quite general assumptions. Thus, the theory of viscosity solutions provides a powerful method for studying control problems. A special feature of the book is the presentation of control problems in the viscosity solution framework, the first detailed presentation in book form, and the authors played a key role in the development of this field. The book consists of nine chapters and an appendix. The first part (Chapters 1-2) presents deterministic optimal control and its corresponding viscosity solution. Main topics are (1) dynamic programming approach to deterministic control problems and connection with Pontryagin's principle, (2) calculus of variations and (3) viscosity solutions for nonlinear partial differential equations of first order. The second part (Chapters 3-5) deals with the stochastic case. Chapter 3 gives an introduction to abstract dynamic programming for controlled Markov processes. Here, the dynamic programming equation is described using the infinitesimal generator of semigroups. Chapter 4 is concerned with the control of diffusion processes governed by stochastic differential equations. Some regularity properties of value functions are presented, by using probability arguments. Chapter 5 is devoted to study viscosity solutions of nonlinear differential equations of second order. A comparison result is proved in detail. Here the value function turns out to be the unique viscosity solution of the HJB equation by the dynamic programming principle. For readers interested in viscosity solutions but not in control theory, this chapter together with Chapter 2 may be interesting. The third part (Chapters 6-9) consists of four topics. Chapter 6 (logarithmic transformation) and Chapter 7 (singular perturbations) deal with asymptotic problems by stochastic control method. Chapter 8 presents singular stochastic control and Chapter 9 is concerned with numerical methods for solving HJB equations approximately. Each chapter contains historical remarks. The appendix consists of five subjects: (A) duality relationships; (B) Dynkin's formula for random evolutions with Markov chain parameters; (C) extension of Lipschitz continuous functions, smoothing; (D) stochastic differential equations: random coefficients; (E) a result of Alexandrov. This book may be interesting for mathematicians and engineers working with control problems and the theory of viscosity solutions.
- Mean field games
- A regularity theorem for a Volterra integral equation of the second kind
- Large investor trading impacts on volatility
- Optimal investment strategy to minimize the ruin probability of an insurance company under borrowing constraints
- Optimal financing and dividend control of the insurance company with fixed and proportional transaction costs
- On the numerical solution of nonlinear Black-Scholes equations
- An optimal investment strategy with maximal risk aversion and its ruin probability
- Dynamic mean-variance problem with constrained risk control for the insurers
- Some flows in shape optimization
- A comparison principle for Hamilton-Jacobi equations related to controlled gradient flows in infinite dimensions
- A PDE approach to large deviations in Hilbert spaces
- Killed Brownian motion and inequalities among solutions of the Schrödinger equation
- Impulse control problem on finite horizon with execution delay
- Controlled stochastic differential equations under Poisson uncertainty and with unbounded utility
- Beale-Kato-Majda type condition for Burgers equation
- Optimal proportional reinsurance and investment with transaction costs. I: Maximizing the terminal wealth
- Optimal dividends in the Brownian motion risk model with interest
- Dynamic portfolio selection with fixed and/or proportional transaction costs using non-singular stochastic optimal control theory
- Penalty approach to the HJB equation arising in European stock option pricing with proportional transaction costs
- Estimating error of the optimal grid design in the problems of nonlinear optimal control of prescribed duration
- Optimal asset allocation for aggregated defined benefit pension funds with stochastic interest rates
- Stochastic optimal bounded control of MDOF quasi nonintegrable-Hamiltonian systems with actuator saturation
- Approximate solutions to the time-invariant Hamilton-Jacobi-Bellman equation
- Hedging point policy improvement
- On piecewise deterministic Markov control processes: Control of jumps and of risk processes in insurance
- Dissipative control systems synthesis with full state feedback
- Nonlinear Neumann boundary conditions for quasilinear degenerate elliptic equations and applications
- Nash equilibria of risk-sensitive nonlinear stochastic differential games
- Singular optimal strategies for investment with transaction costs
- Ginzburg-Landau equation and motion by mean curvature. I: Convergence
- On asymptotic optimization of a class of nonlinear stochastic hybrid systems
- Shape-from-shading, viscosity solutions and edges
- Viscosity solutions of Hamilton-Jacobi equations
- Computational aspects in applied stochastic control
- Convex viscosity solutions and state constraints
- On the role of computation in economic theory
- Optimal times for constrained nonlinear control problems without local controllability
- Optimal production planning in a stochastic manufacturing system with long-run average cost
- Linear-quadratic tracking of coupled slow and fast targets
- Controlled diffusion models for optimal dividend pay-out
- Cauchy-Dirichlet problem for first order nonlinear systems
- Asymptotic for the principal eigenvalue and eigenfunction of a nearly first-order operator with large potential
- General existence theorems for Hamilton-Jacobi equations in the scalar and vectorial cases
- Optimal consumption and portfolio choice with borrowing constraints
- Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation
- Hedging in incomplete markets with HARA utility
- Quadratic functionals and small ball probabilities for the \(m\)-fold integrated Brownian motion
- The first exit time of a Brownian motion from an unbounded convex domain
- A transformation approach for solving the Hamilton-Jacobi-Bellman equation in \({\mathcal H}_2\) deterministic and stochastic optimal control of affine nonlinear systems.
- Singular stochastic control in the presence of a state-dependent yield structure
- Monopolists and viscous demand.
- Mean-variance hedging for pricing European-type contingent claims with transaction costs.
- Pricing equity-linked pure endowments via the principle of equivalent utility.
- On the properties of \(r\)-excessive mappings for a class of diffusions
- On the generalized Dirichlet problem for viscous Hamilton--Jacobi equations.
- A stochastic representation for mean curvature type geometric flows
- Optimal consumption/investment policies with undiversifiable income risk and liquidity constraints
- On the dynamics of stochastic diffusion of manufacturing technology
- Optimal long term growth rate of expected utility of wealth
- Computation of distorted probabilities for diffusion processes via stochastic control methods.
- Numerical comparison of controls and verification of optimality for stochastic control problems
- Remarks regarding the gap between continuous, Lipschitz, and differentiable storage functions for dissipation inequalities appearing in \(H_{\infty}\) control
- Oligopoly equilibria in nonrenewable resource markets
- Incentive compatibility constraints and dynamic programming in continuous time
- Ergodic control problems for optimal stochastic production planning with production constraints.
- Sufficient conditions for relative minima of broken extremals in optimal control theory
- Optimal consumption of a divisible durable good
- Regularity properties of viscosity solutions of a non-Hörmander degenerate equation
- Risk-sensitive control and differential games in infinite dimensions
- Optimal singular control strategies for controlling a process to a goal.
- Rectifiability results for singular and conjugate points of optimal exit time problems
- Turnpike property and convergence rate for an investment model with general utility functions
- Nonstandard local discontinuous Galerkin methods for fully nonlinear second order elliptic and parabolic equations in high dimensions
- Optimal excess-of-loss reinsurance and investment problem with delay and jump-diffusion risk process under the CEV model
- Dynamics of concentration in a population model structured by age and a phenotypical trait
- Hiring, firing, and relocation under employment protection
- Optimal dividend and capital injection strategies in the Cramér-Lundberg risk model
- Continuous viscosity solutions for nonlocal Dirichlet problems with coercive gradient terms
- Optimal entry to an irreversible investment plan with non convex costs
- Optimal harvesting strategies for stochastic competitive Lotka-Volterra ecosystems
- A stochastic maximum principle for mixed regular-singular control problems via Malliavin calculus
- Optimal stopping with a probabilistic constraint
- A semi-Lagrangian scheme for a modified version of the Hughes' model for Pedestrian flow
- Variance-optimal martingale measures for diffusion processes with stochastic coefficients
- Optimal risk control and dividend strategies in the presence of two reinsurers: variance premium principle
- Optimal investment and reinsurance for insurers with uncertain time-horizon
- Optimal investment for insurers with the extended CIR interest rate model
- Optimal control of investment-reinsurance problem for an insurer with jump-diffusion risk process: independence of Brownian motions
- Optimal reinsurance-investment problem for an insurer and a reinsurer with jump-diffusion process
- Optimal learning before choice
- Method for constructing piecewise quadratic value functions in a control problem for a switched system
- Singular optimal controls of stochastic recursive systems and Hamilton-Jacobi-Bellman inequality
- Large time behavior of solutions of local and nonlocal nondegenerate Hamilton-Jacobi equations with Ornstein-Uhlenbeck operator
- Asymptotic behaviour for operators of Grushin type: invariant measure and singular perturbations
- Martingale problem under nonlinear expectations
- The perspective of a bank in granting credits: an optimization model
- An example of a stochastic equilibrium with incomplete markets
- Singular risk-neutral valuation equations
- Interplay between dividend rate and business constraints for a financial corporation
- The right time to sell a stock whose price is driven by Markovian noise
This page was built for publication: Controlled Markov processes and viscosity solutions
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4032143)
