Markovian models and algorithms (Q701228)

This book is devoted to all those, mathematicians or not, who want to acquire a practical probabilistic tool for their current applications. Elaboration and validation of a probabilistic model can not avoid the process of writing efficient simulation programs, analyzing their computational efficiency and numerical performance; this is one of the most important goals of the present book. Another practical purpose followed by this volume is to provide a deeper understanding of the considered Markov (chain) algorithms, their running, the simulation of the algorithms with statistical verification, all these based on the powerful (and freeware) computational environment called Scilab (Scientific Laboratory). Thus book is also an application-oriented guide devoted to undergraduate and graduate students, with a minimal knowledge on probability theory and statistics. It encloses six (6) chapters whose contents is, briefly, the following: Chapter 1 (Introduction) provides a gentle introduction to the Monte Carlo random algorithms and methods. Chapter 2 (Independent drawings) studies the generation of random samples, with applications to integral calculus and expectation random variables. Chapter 3 (Finite-time Markov methods) is using the law of large numbers to compute expectations, dealing also with linear system resolution, differential and particular methods, Fokker-Planck and Feynman-Kac equations, and Boltzmann equations). Chapter 4 (Markov-chain investigation) considers either a homogeneous state space to approach a given equilibrium measure (Monte Carlo Markov chains), or the search of a function extremum point (e.g. when solving the simulated annealing problem). Experiences with stochastic optimization algorithms such as genetic algorithms and MOSES algorithms are exposed. Chapter 5 (Jump Markov processes) explores continuous-time and discrete statespace Markov models, involving delayed algorithms, network simulation, and the convergence of particle systems. Chapter 6 (Scilab simulation) presents practical experience with programs that implement effectively the considered algorithms. Numerical simulation within the powerful Scilab framework is provided. It is important to mention that each chapter is ended with interesting proposed problems having the role of clearing up, fixing, and extending the comprehension of the issues discussed within the chapter body.