scientific article; zbMATH DE number 1461223
The book presents approaches to the adaptive control of finite Markov chains. The structure of the Markov chains is assumed to be given but the transition probabilities are considered unknown a priori. The goal of the control process is to optimize a linear cost function subject to a set of linear inequality constraints. The time horizon of the control process is infinite and the problem is to optimize the asymptotic estimated cost function over time assuming the limit exists. The book consists of eight chapters divided into two parts. The first four chapters introduce basic definitions and address the unconstrained optimization problem. The other four chapters of the second part deal with the theoretical aspects of the constrained cases and include one chapter containing simulation results for the various algorithms covered in the book. The approach adopted throughout the book falls in the category of direct approaches to adaptive control where the control actions are directly estimated using the available information as in the learning automata literature. There appears to be no comparison with the indirect approaches relying on the so-called certainty equivalence principle in adaptive control. All the algorithms presented in the book are based on well-known stochastic approximation and gradient optimization techniques. The book is rich with theorems and mathematical descriptions but falls short in presenting viable practical applications. The content of the book is mostly not new and the chapters seem to be loosely coupled with each chapter containing a list of references with some redundancy. The book may serve as a textbook for graduate study for engineers and mathematicians in the stochastic control area.
- Adaptive control of constrained finite Markov chains
- Constructing the Pareto front for multi-objective Markov chains handling a strong Pareto policy approach
- Computing the strong Nash equilibrium for Markov chains games
- Adapting attackers and defenders patrolling strategies: a reinforcement learning approach for Stackelberg security games
- Recursive estimation of high-order Markov chains: approximation by finite mixtures
- Sparse mean-variance customer Markowitz portfolio optimization for Markov chains: a Tikhonov's regularization penalty approach
- Solving the cost to go with time penalization using the Lagrange optimization approach
- Learning Machiavellian strategies for manipulation in Stackelberg security games
- Using the extraproximal method for computing the shortest-path mixed Lyapunov equilibrium in Stackelberg security games
- Observer and control design in partially observable finite Markov chains
- Computing the strong \(L_p\)-Nash equilibrium for Markov chains games: convergence and uniqueness
- Controller exploitation-exploration reinforcement learning architecture for computing near-optimal policies
- A Tikhonov regularized penalty function approach for solving polylinear programming problems
- Optimization problems in chemical reactions using continuous-time Markov chains
- Oja's algorithm for graph clustering, Markov spectral decomposition, and risk sensitive control
- Minimizing the learning loss in adaptive control of Markov chains under the weak accessibility condition
- A continuous-time Markov Stackelberg security game approach for reasoning about real patrol strategies
- Learning Control of Dynamical Systems Based on Markov Decision Processes: Research Frontiers and Outlooks
- Using the Manhattan distance for computing the multiobjective Markov chains problem
- A Tikhonov regularization parameter approach for solving Lagrange constrained optimization problems
- Handling a Kullback--Leibler divergence random walk for scheduling effective patrol strategies in Stackelberg security games
- Setting Nash versus Kalai-Smorodinsky bargaining approach: computing the continuous-time controllable Markov game
- Adaptive policy for two finite Markov chains zero-sum stochastic game with unknown transition matrices and average payoffs
- Optimization based on a team of automata with binary outputs
- Computing the Stackelberg/Nash equilibria using the extraproximal method: convergence analysis and implementation details for Markov chains games
- Saddle-point calculation for constrained finite Markov chains
This page was built for publication:
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4485809)
