Rates of convergence of adaptive step-size of stochastic approximation algorithms
The stochastic approximation of the model \(y_n=x_n^T\) \(\theta +v_n\) is considered, where \(y_n\), \(x_n\), \(v_n\) are the observations, the random inputs, the noise, respectively, and \(\theta\) is the estimated parameter. The constant step-size \(\varepsilon\) of the classical adaptive algorithm \(\widehat{\theta}_{n+1}=\widehat{\theta}_n+\varepsilon x_n(y_n-x_n^T\widehat{\theta}_n)\) is replaced by an adaptive step-size \(\overline{\varepsilon}_n\), \(\widehat{\theta}_{n+1}=\widehat{\theta}_n+\overline{\varepsilon}_nx_n(y_n-x_n^T\widehat{\theta}_n)\), defined recursively by \(\overline{\varepsilon}_n=\overline{\varepsilon}_{n-1}+\frac 1n(\varepsilon_n-\overline{\varepsilon}_ n)=\frac 1n\sum_{i=1}^n \varepsilon_i\) with \(\varepsilon_{n+1}=\varepsilon_n+\gamma_nx_n(y_n-x_n^T\widehat{\theta}_n)\), \(\sum\varepsilon_n<\infty\) and \(\sum\gamma_n=\infty\) and \(\gamma_n\to 0\) slower than \(\frac 1n\). The convergence of \(\widehat{\theta}_n\) and \(\overline{\varepsilon}_n\) are proven and the rate of the convergence is analyzed by using the Gaussian approximation theorem.
- Adaptive stochastic approximation algorithm
- A stochastic approximation algorithm with step-size adaptation
- Rate of Convergence for Constrained Stochastic Approximation Algorithms
- Noise conditions for prespecified convergence rates of stochastic approximation algorithms
- Stabilization of stochastic approximation by step size adaptation
- A Stochastic Approximation Method
- Acceleration of Stochastic Approximation by Averaging
- Analysis of recursive stochastic algorithms
- Analysis of stochastic gradient algorithms for linear regression problems
- Convergence of an adaptive linear estimation algorithm
- scientific article; zbMATH DE number 3683489 (Why is no real title available?)
- scientific article; zbMATH DE number 48727 (Why is no real title available?)
- On positive real transfer functions and the convergence of some recursive schemes
- Rates of Convergence for Stochastic Approximation Type Algorithms
- Stochastic Approximation with Averaging of the Iterates: Optimal Asymptotic Rate of Convergence for General Processes
- Sharp convergence rates of stochastic approximation for degenerate roots
- Stochastic approximation algorithms: Nonasymptotic estimation of their convergence rates
- A stochastic approximation algorithm with step-size adaptation
- New stochastic approximation algorithms with adaptive step sizes
- An ODE method to prove the geometric convergence of adaptive stochastic algorithms
- Stochastic approximation with adaptive step sizes for optimization in noisy environment and its application in regression models
- scientific article; zbMATH DE number 440564 (Why is no real title available?)
- Linear Convergence of Comparison-based Step-size Adaptive Randomized Search via Stability of Markov Chains
- scientific article; zbMATH DE number 53162 (Why is no real title available?)
- On 'global convergence' of Steiglitz-McBride adaptive algorithm
- Stabilization of stochastic approximation by step size adaptation
- Almost sure rate of convergence of the parameter estimates in stochastic approximation algorithm
- Rate of Convergence for Constrained Stochastic Approximation Algorithms
- Noise conditions for prespecified convergence rates of stochastic approximation algorithms
- scientific article; zbMATH DE number 4120105 (Why is no real title available?)
- Decreasing step Stochastic algorithms: a.s. behaviour of weighted empirical measures
- Convergence properties and step size estimation for MLMS algorithm
- scientific article; zbMATH DE number 778098 (Why is no real title available?)
- Analysis of adaptive step-size SA algorithms for parameter tracking
- Analysis of practical step size selection in stochastic approximation algorithms
- Convergence rates and decoupling in linear stochastic approximation algorithms
- Adaptive step size rules for stochastic optimization in large-scale learning
- Adaptive stepsizes for recursive estimation with applications in approximate dynamic programming
This page was built for publication: Rates of convergence of adaptive step-size of stochastic approximation algorithms
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1977809)
