Combining the stochastic counterpart and stochastic approximation methods (Q679024)

Let \(\ell(v, \theta)=E_v\{L(Y,\theta)\}\) be the expected performance of a discrete event system (DES), where \(L\) is the sample performance driven by an input vector \(Y\) with a probability density function \(f(y, v)\) and \(\theta\) is a parameter of the sample performance. This paper considers how the score function (SF) and crude Monte Carlo (CMC) methods are used to estimate \(\ell(v, \theta), \nabla_v \ell(v, \theta)\) and \(\nabla_\theta(v, \theta)\) simultaneously for several combinations of \((v, \theta)\), and how to combine the stochastic counterpart (SC) and stochastic approximation (SA) to optimize \(\ell(v, \theta)\) with respect to two parameter sets. After a brief review of SF, CMC, SC and SA methods, the problem concerned in this paper is described in Section 1. Section 2 explains how to combine SF and CMC in order to estimate \(\ell(v, \theta)\) and its gradient simultaneously for several values of \(v\) and \(\theta\). In Section 3, three algorithms combining SC and SA methods, one sequential and two parallel, are presented for the minimization of \(\ell(v, \theta)\) with respect to both \(v\) and \(\theta\), and a convergence proof for the first sequential algorithm is provided. An M/D/1 queue example illustrates the behavior of the algorithms.