A modular analysis of adaptive (non-)convex optimization: optimism, composite objectives, variance reduction, and variational bounds

DOI10.1016/J.TCS.2019.11.015MaRDI QIDQ2290691zbMATH OpenOpenAlexFDO

Authors Pooria Joulani, András György, Csaba Szepesvári

Publication date 29 January 2020

Published in Theoretical Computer Science (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1016/j.tcs.2019.11.015

zbMATH Keywords

stochastic optimizationadaptive algorithmsonline learningmirror descentregret boundfollow the regularized leader

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Online algorithms; streaming algorithms (68W27) Stochastic programming (90C15)

Recommendations

• A modular analysis of adaptive (non-)convex optimization: optimism, composite objectives, and variational bounds
• A survey of algorithms and analysis for adaptive online learning
• Regret bounded by gradual variation for online convex optimization
• Analysis of Online Composite Mirror Descent Algorithm
• Scale-free online learning

Cites work

• scientific article; zbMATH DE number 3790208 (Why is no real title available?)
• scientific article; zbMATH DE number 3313108 (Why is no real title available?)
• scientific article; zbMATH DE number 3371284 (Why is no real title available?)
• A generalized online mirror descent with applications to classification and regression
• A survey of algorithms and analysis for adaptive online learning
• Adaptive subgradient methods for online learning and stochastic optimization
• An optimal method for stochastic composite optimization
• Beyond the regret minimization barrier: optimal algorithms for stochastic strongly-convex optimization
• Convex analysis and monotone operator theory in Hilbert spaces
• Cubic regularization of Newton method and its global performance
• Deep learning
• Exponentiated gradient versus gradient descent for linear predictors
• Gradient descent learns linear dynamical systems
• Logarithmic regret algorithms for online convex optimization
• Mirror descent and nonlinear projected subgradient methods for convex optimization.
• Online learning and online convex optimization
• Optimal distributed online prediction using mini-batches
• Prediction, Learning, and Games
• Primal-dual subgradient methods for convex problems
• Proximal Minimization Methods with Generalized Bregman Functions
• Solving variational inequalities with stochastic mirror-prox algorithm
• Stochastic optimal control. The discrete time case

Cited in

(4)

• Online composite optimization with time-varying regularizers
• A modular analysis of adaptive (non-)convex optimization: optimism, composite objectives, and variational bounds
• A survey of algorithms and analysis for adaptive online learning
• Optimistic optimisation of composite objective with exponentiated update

Describes a project that uses

Uses Software

• AdaGrad

This page was built for publication: A modular analysis of adaptive (non-)convex optimization: optimism, composite objectives, variance reduction, and variational bounds

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2290691)