A singular perturbation approach to modeling and control of Markov chains

DOI10.1109/TAC.1981.1102780zbMath0475.93079OpenAlexW1989080638MaRDI QIDQ3931155

Randolph G. Phillips, Petar V. Kokotovic

Publication date: 1981

Published in: IEEE Transactions on Automatic Control (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1109/tac.1981.1102780

zbMATH Keywords

discounted costMarkov chain modelsdiscrete time Markov chaindecentralized algorithmnear optimal controlaggregate statesgroup algorithminfinite horizon cost function

Mathematics Subject Classification ID

Continuous-time Markov processes on general state spaces (60J25) Interacting random processes; statistical mechanics type models; percolation theory (60K35) Optimal stochastic control (93E20) Singular perturbations of ordinary differential equations (34D15) Decomposition methods (49M27)

Related Items

Singular perturbations in optimal control problems, Asymptotic properties of a singularly perturbed Markov chain with inclusion of transient states., Approximation and decomposition of singularly perturbed stochastic hybrid systems, Control of dynamic systems under the influence of singularly perturbed Markov chains, Asymptotic expansions of solutions of integro-differential equations for transition densities of singularly perturbed switching diffusions: Rapid switchings, Optimal stopping of two-time scale Markovian systems: analysis, numerical methods, and applications, Time scale decomposition in production planning for unreliable flexible manufacturing systems, Asymptotic Expansions for Stationary Distributions of Perturbed Semi-Markov Processes, Singulary perturbed Markov control problem: Limiting average cost, Singularly perturbed Markov chains: Convergence and aggregation, NEARLY OPTIMAL CONTROL OF NONLINEAR MARKOVIAN SYSTEMS SUBJECT TO WEAK AND STRONG INTERACTIONS, A stochastic multiscale model for electricity generation capacity expansion, A two-factor stochastic production model with two time scales, Honey-pot constrained searching with local sensory information, Singularly perturbed Markov chains with two small parameters: A matched asymptotic expansion, Singularly perturbed diffusisons: Rapid switchings and fast diffusions., Discrete-time singularly perturbed Markov chains: aggregation, occupation measures, and switching diffusion limit, Control of singularly perturbed Markov chains: A numerical study, Approximate optimal adaptive control for weakly coupled nonlinear systems: A neuro-inspired approach, Singularly perturbed multidimensional switching diffusions with fast and slow switchings, Occupation measures of singularly perturbed Markov chains with absorbing states, Exponential bounds for discrete-time singularly perturbed Markov chains, Asymptotic expansions of backward equations for two-time-scale Markov chains in continuous time, 11 Reduced-order modeling of large-scale network systems, On asymptotic optimization of a class of nonlinear stochastic hybrid systems, On coherency-based decomposition algorithms, Singular perturbations and time-scale methods in control theory: Survey 1976-1983, Optimal and near-optimal incentive strategies in the hierarchical control of Markov chains, Estimating equilibrium probabilities for band diagonal Markov chains using aggregation and disaggregation techniques, A unified view of aggregation and coherency in networks and Markov chains†, Asymptotic expansions of solutions of systems of Kolmogorov backward equations for two-time-scale switching diffusions, Near-optimal controls of random-switching LQ problems with indefinite control weight costs