Modeling, analysis, design, and control of stochastic systems (Q4255597)

This book is a classical one on the modelling of stochastic systems. Chapters 1, 2, 3 and 4 are an elementary course in probability: law of total probability, Bayes rules, univariate and multivariate random variables, expectation, independence, conditional probability and expectation. Chapter 5 is devoted to discrete time Markov models: discrete time Markov chain, transient distribution, occupancy times, limiting behaviour, first passage time. Chapter 6 is the same as Chapter 5 but for continuous time models. The generalized Markov models are presented in Chapter 7: renewal processes, cumulative processes, semi-Markov processes. Chapter 8 applies the results of Chapters 5, 6 and 7 to queueing systems. Chapter 9 is an application to ``designing'' systems. Chapter 10 is also an application to control a dynamical system: it gives an optimal operating policy using the method of linear programming. There are two principal contributions of this book: 1) The author gives many examples of using Markov models in a real application: machine reliability, weather model, inventory system, manufacturing, manpower planning, stock market, telecommunication\dots 2) A software package is available for use with this book.