Optimal control of Markov chains admitting strong and weak interactions

DOI10.1016/0005-1098(81)90047-9zbMath0467.49020OpenAlexW2058141673MaRDI QIDQ1155852

Jean-Pierre Quadrat, Francois Delebecque

Publication date: 1981

Published in: Automatica (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1016/0005-1098(81)90047-9

zbMATH Keywords

perturbation techniquesdynamic programmingMarkov chainsdiscount factornonlinear problemdiscrete time Markov chainaggregate chainsdecentralized chainsergodic control problems

Mathematics Subject Classification ID

Dynamic programming in optimal control and differential games (49L20) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10) Large-scale systems (93A15) Asymptotic expansions of solutions to ordinary differential equations (34E05) Limit theorems in probability theory (60F99)

Related Items

A unified perturbation analysis framework for countable Markov chains ⋮ Simultaneous impulse and continuous control of a Markov chain in continuous time ⋮ Asymptotic properties of a singularly perturbed Markov chain with inclusion of transient states. ⋮ Joint queue-perturbed and weakly coupled power control for wireless backbone networks ⋮ Time scale decomposition in production planning for unreliable flexible manufacturing systems ⋮ Sequences of random matrices modulated by a discrete-time Markov chain* ⋮ Asymptotic Expansions for Stationary Distributions of Perturbed Semi-Markov Processes ⋮ Subsystems, time scales and multimodeling ⋮ Singulary perturbed Markov control problem: Limiting average cost ⋮ On some problems arising in asymptotic analysis of Markov processes with singularly perturbed generators ⋮ Singularly perturbed Markov chains: Convergence and aggregation ⋮ Towards the optimal control of Markov chains with constraints ⋮ A two-factor stochastic production model with two time scales ⋮ Singularly perturbed Markov chains with two small parameters: A matched asymptotic expansion ⋮ Control of singularly perturbed Markov chains: A numerical study ⋮ Occupation measures of singularly perturbed Markov chains with absorbing states ⋮ Hierarchical aggregation of singularly perturbed finite state markov processes ⋮ On asymptotic optimization of a class of nonlinear stochastic hybrid systems ⋮ Singular perturbations and time-scale methods in control theory: Survey 1976-1983 ⋮ Optimal and near-optimal incentive strategies in the hierarchical control of Markov chains ⋮ Estimating equilibrium probabilities for band diagonal Markov chains using aggregation and disaggregation techniques ⋮ A unified view of aggregation and coherency in networks and Markov chains† ⋮ Multi-grid methods for Hamilton-Jacobi-Bellman equations

Cites Work

• Perturbations singulières dans les problèmes aux limites et en contrôle optimal
• Finite state Markovian decision processes
• A decomposition of near-optimum regulators for systems with slow and fast modes
• A Controlled Finite Markov Chain with an Arbitrary Set of Decisions
• Contribution of stochastic control singular perturbation averaging and team theories to an example of large-scale systems: Management of hydropower production
• Stability and Control of Stochastic Systems with Wide-band Noise Disturbances. I
• Discrete Dynamic Programming with a Small Interest Rate
• Discrete Dynamic Programming with Sensitive Discount Optimality Criteria
• Perturbation theory and finite Markov chains
• Aggregation of Variables in Dynamic Systems
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item