On the method of multipliers for mathematical programming problems (Q2552449)

In this paper, the numerical solution of the basic problem of mathematical programming is considered. This is the problem of minimizing a function \(f(x)\) subject to a constraint \(\varphi(x) = 0\). Here, \(f\) is a scalar, \(x\) an \(n\)-vector, and \(\varphi\) a \(q\)-vector, with \(q<n\). The approach employed is based on the introduction of a special function which allows one to view the vector \(x\) as unconstrained. Specifically, the function \(f(x)\) is replaced by the augmented penalty function \(W(x,\lambda,k) = f(x) + \lambda^T \varphi(x) + \varphi^T\varphi(x)\). Here, the \(q\)-vector \(\lambda\) is an approximation to the Lagrange multiplier and the scalar \(k>0\) is the penalty constant. Previously, the augmented penalty function \(W(x,\lambda,k)\) was used by Hestenes in his method of multipliers. In Hestenes' version, the method of multipliers involves cycles, in each of which the multiplier and the penalty constant are held constant. After the minimum of the augmented penalty function is achieved in any given cycle, the multiplier \(\lambda\) is updated, while the penalty constant \(k\) is held unchanged. In this paper, two modifications of the method of multipliers are presented in order to improve its convergence characteristics. The improved convergence is achieved by (i) increasing the updating frequency so that the number of iterations in a cycle is \(\Delta N=1\) for the ordinary-gradient algorithm and the modified-quasilinearization algorithm and \(\Delta N=n\) for the conju