Weighted averaging and stochastic approximation
The authors investigate the relations between weighted averaging and stochastic approximation (SA) algorithms. First, they define the weighted averaging operator and introduce two properties of the operator: regularity and effectiveness. They establish necessary and sufficient conditions on a sequence for convergence of its average. The equivalence between weighted averaging and SA in terms of convergence is proved, i.e., it is shown that the convergence of a weighted average of the noise sequence is necessary and sufficient for the convergence of standard SA algorithms. Based on this, they present necessary and sufficient noise conditions for convergence of averaged SA algorithms in the linear case. The established noise conditions for convergence of the averaged SA algorithms are substantially weaker than the conditions for convergence of the standard SA algorithms without averaging. The analysis is deterministic by studying the sample-path behavior of the algorithms.
- Weighted Means in Stochastic Approximation of Minima
- Weighted means of processes in stochastic approximation
- scientific article; zbMATH DE number 272681
- On stochastic approximation procedures with averaging
- Weighted approximation of random functions
- Weighted averages of random variables and exact weights
- Stochastic Averaging: Some Methods and Applications
- Acceleration of Stochastic Approximation by Averaging
- Almost sure approximation of the Robbins-Monro process by sums of independent random variables
- Almost sure approximations to the Robbins-Monro and Kiefer-Wolfowitz processes with dependent noise
- An alternative proof for convergence of stochastic approximation algorithms
- Applications of a Kushner and Clark lemma to general classes of stochastic algorithms
- Asymptotically optimal rate of convergence of smoothed stochastic recursive algorithms
- Convergence and robustness of the Robbins-Monro algorithm truncated at randomly varying bounds
- Convergence of the Robbins-Monro method for linear problems in a Banach space
- Equivalent necessary and sufficient conditions on noise sequences for stochastic approximation algorithms
- General results on the convergence of stochastic algorithms
- scientific article; zbMATH DE number 48363 (Why is no real title available?)
- scientific article; zbMATH DE number 53271 (Why is no real title available?)
- MINISYMPOSIUM 3
- Necessary and sufficient conditions for the Robbins-Monro method
- New method of stochastic approximation type
- On a stochastic approximation procedure based on averaging
- On extensions of Polyak's averaging approach to stochastic approximation
- On the Averaged Stochastic Approximation for Linear Regression
- Some Remarks on The Paper by Theodorescu and Wolff: “Sequential Estimation of Expectations in the Presence of Trend”
- Stochastic algorithms with Armijo stepsizes for minimization of functions
- Stochastic approximation methods for constrained and unconstrained systems
- Stochastic approximation with averaging and feedback: rapidly convergent "on-line" algorithms
- Stochastic Approximation with Averaging of the Iterates: Optimal Asymptotic Rate of Convergence for General Processes
- Strong convergence of a stochastic approximation algorithm
- Strong representation of an adaptive stochastic approximation procedure
- Weighted means of processes in stochastic approximation
- Stochastic global stability and bifurcation of a hydro-turbine generator
- Generalization and Robustness of Batched Weighted Average Algorithm with V-Geometrically Ergodic Markov Data
- Recurrent partial averaging in the theory of weighted Monte Carlo methods
- Weighted Means in Stochastic Approximation of Minima
- Trajectory averaging for stochastic approximation MCMC algorithms
This page was built for publication: Weighted averaging and stochastic approximation
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1367100)
