Scalable estimation strategies based on stochastic approximations: classical results and new insights

DOI10.1007/S11222-015-9560-YzbMATH Open1332.62291OpenAlexW644527247WikidataQ38542365 ScholiaQ38542365MaRDI QIDQ5963780FDOQ5963780

Authors: Panos Toulis, E. M. Airoldi 

Publication date: 23 February 2016

Published in: Statistics and Computing (Search for Journal in Brave)

Full work available at URL: http://europepmc.org/articles/pmc4484776

Recommendations

• Asymptotic and finite-sample properties of estimators based on stochastic gradients
• Optimal survey schemes for stochastic gradient descent with applications to \(M\)-estimation
• Large-scale machine learning with stochastic gradient descent
• Scalable statistical inference for averaged implicit stochastic gradient descent
• Statistical inference for model parameters in stochastic gradient descent

zbMATH Keywords

maximum likelihoodrecursive estimationbig dataasymptotic analysisoptimal learning rateimplicit stochastic gradient descent methods

Mathematics Subject Classification ID

Markov processes: estimation; hidden Markov models (62M05) Neural nets and related approaches to inference from stochastic processes (62M45)

Cites Work

• The elements of statistical learning. Data mining, inference, and prediction
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Learning deep architectures for AI
• Title not available (Why is that?)
• Robust Estimation of a Location Parameter
• Title not available (Why is that?)
• Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
• Title not available (Why is that?)
• Acceleration of Stochastic Approximation by Averaging
• A Stochastic Approximation Method
• Mirror descent and nonlinear projected subgradient methods for convex optimization.
• SGD-QN: careful quasi-Newton stochastic gradient descent
• Robust Stochastic Approximation Approach to Stochastic Programming
• Title not available (Why is that?)
• Title not available (Why is that?)
• Spectrum estimation for large dimensional covariance matrices using random matrix theory
• On-line expectation-maximization algorithm for latent data models
• Title not available (Why is that?)
• Numerical analysis for statisticians
• Riemann Manifold Langevin and Hamiltonian Monte Carlo Methods
• Title not available (Why is that?)
• Adaptive design and stochastic approximation
• Monotone Operators and the Proximal Point Algorithm
• Title not available (Why is that?)
• A Class of Methods for Solving Nonlinear Simultaneous Equations
• On Asymptotic Normality in Stochastic Approximation
• Title not available (Why is that?)
• On the convergence of markovian stochastic algorithms with rapidly decreasing ergodicity rates
• Convergence of stochastic proximal gradient algorithm
• On‐line learning for very large data sets
• Multivariate adaptive stochastic approximation
• Training Products of Experts by Minimizing Contrastive Divergence
• On sampling controlled stochastic approximation
• On a Stochastic Approximation Method
• Adaptive stepsizes for recursive estimation with applications in approximate dynamic programming
• Reinforcement learning: a tutorial survey and recent advances
• Large-Scale Machine Learning with Stochastic Gradient Descent
• Efficient recursive estimation. Application to estimating the parameters of a covariance function
• Justifying and Generalizing Contrastive Divergence
• An Extension of the Robbins-Monro Procedure
• Asymptotic Distribution of Stochastic Approximation Procedures
• On the convergence behavior of the LMS and the normalized LMS algorithms
• The p-norm generalization of the LMS algorithm for adaptive filtering
• Asymptotically efficient stochastic approximation; the RM case
• Robust estimation via stochastic approximation
• Stabilization of Stochastic Iterative Methods for Singular and Nearly Singular Linear Systems
• On-line EM Algorithm for the Normalized Gaussian Network
• Title not available (Why is that?)
• Online EM algorithm for mixture with application to Internet traffic modeling
• Title not available (Why is that?)
• Title not available (Why is that?)
• On the worst-case analysis of temporal-difference learning algorithms

Cited In (12)

• Sublinear Convergence of a Tamed Stochastic Gradient Descent Method in Hilbert Space
• Dynamical behavior of a stochastic forward-backward algorithm using random monotone operators
• Multivariate online regression analysis with heterogeneous streaming data
• Online causal inference with application to near real-time post-market vaccine safety surveillance
• Renewable risk assessment of heterogeneous streaming time-to-event cohorts
• Title not available (Why is that?)
• Sub-linear convergence of a stochastic proximal iteration method in Hilbert space
• Distributed simultaneous inference in generalized linear models via confidence distribution
• Online inference with multi-modal likelihood functions
• Adaptive step size rules for stochastic optimization in large-scale learning
• Title not available (Why is that?)
• Parallel-and-stream accelerator for computationally fast supervised learning

Uses Software

• ElemStatLearn
• SGD-QN
• AdaGrad

This page was built for publication: Scalable estimation strategies based on stochastic approximations: classical results and new insights

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5963780)