Adapative importance sampling on discrete Markov chains (Q1305417)

In modelling particle transport through the medium, the path of a particle behaves as a transient Markov chain. The authors are interested in characteristics of the particle's movement depending on its starting state, which take the form of a ``score'' accumulated with each transition. The main purpose of this work is to prove that under certain conditions adaptive importance sampling for discrete Markov chains with scoring converges exponentially. Examples presented show that this exponential convergence can occur with a reasonably small number of simulation runs. These assumptions include that the state space is finite, the vector of expected scores conform a linear model and that there are sufficiently many replications of the initial states in the simulation.