Approximate iterative algorithms

DOI10.1201/B16613MaRDI QIDQ2873813zbMATH OpenOpenAlexFDO

Authors Anthony Almudevar

Publication date 27 January 2014

Full work available at URL https://doi.org/10.1201/b16613

Mathematics Subject Classification ID

Research exposition (monographs, survey articles) pertaining to numerical analysis (65-02) Numerical mathematical programming methods (65K05) Dynamic programming (90C39) Markov and semi-Markov decision processes (90C40)

Review text

This is an excellent book on dynamic programming and Markov decision processes. Dynamic programming, invented by the late Richard Bellman, has created a new field of optimality and approximation theory.NEWLINENEWLINEThe author has divided his book into three parts: I: Mathematical background with 8 chapters, II: General theory of approximate iterative algorithms with 3 chapters, and III: Application to Markov decision processes with 6 chapters.NEWLINENEWLINEPart I covers real analysis and linear algebra, measure theory, probability theory, stochastic processes and functional analysis including fixed-point equations and the distribution of a maximum. Part II deals with linear convergence, the general theory of approximate iterative systems, and selection of approximate schedules. Part III contains chapters on Markov decision processes, value iteration, approximation in dynamic programming, sampling, iteration by truncation, grid approximation and adaptive control.NEWLINENEWLINEThe author has elaborated the theory in the application to online parameter estimation and exploration schedule.

Recommendations

• Approximate Dynamic Programming
• Approximate dynamic programming. Solving the curses of dimensionality
• scientific article; zbMATH DE number 3148886
• scientific article; zbMATH DE number 4061056
• Dynamic programming and optimal control. Vol. 1.

Cited in

(12)

• The Arrow-Hurwicz algorithm with alternating iterative parameters
• A consumption and investment problem via a Markov decision processes approach with random horizon
• Convergence of Markov decision processes with constraints and state-action dependent discount factors
• Computable approximations for continuous-time Markov decision processes on Borel spaces based on empirical measures
• scientific article; zbMATH DE number 4086860 (Why is no real title available?)
• scientific article; zbMATH DE number 1995854 (Why is no real title available?)
• Some limit properties of Markov chains induced by recursive stochastic algorithms
• Convergence results for stochastic convex feasibility problem using random Mann and simultaneous projection iterative algorithms in Hilbert space
• A stochastic contraction mapping theorem
• Bayesian optimal experimental design for inferring causal structure
• Non-stationary value iteration for adaptive average control of piecewise deterministic Markov processes
• Adaptive nonstationary value iteration for discounted control of piecewise deterministic Markov processes

This page was built for publication: Approximate iterative algorithms

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2873813)