Actor-critic algorithms with online feature adaptation

DOI10.1145/2868723MaRDI QIDQ5270681zbMATH OpenOpenAlexFDO

Authors K. J. Prabuchandran, Shalabh Bhatnagar, Vivek Borkar

Publication date 30 June 2017

Published in ACM Transactions on Modeling and Computer Simulation (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1145/2868723

zbMATH Keywords

actor-critic algorithmsfeature adaptationfunction approximationGrassmann manifoldMarkov decision processesonline learningpolicy gradientsresidual gradient schemeSPSAstochastic approximationtemporal difference learning

Mathematics Subject Classification ID

Markov and semi-Markov decision processes (90C40)

Recommendations

• OnActor-Critic Algorithms
• Natural actor-critic algorithms
• An actor-critic algorithm with function approximation for discounted cost constrained Markov decision processes
• Actor-Critic--Type Learning Algorithms for Markov Decision Processes
• An online actor-critic algorithm with function approximation for constrained Markov decision processes

Cites work

• 10.1162/1532443041827907
• 10.1162/1532443041827934
• An analysis of temporal-difference learning with function approximation
• Average cost temporal-difference learning
• Basis function adaptation in temporal difference reinforcement learning
• Convergence rate of linear two-time-scale stochastic approximation.
• scientific article; zbMATH DE number 5957492 (Why is no real title available?)
• scientific article; zbMATH DE number 5223994 (Why is no real title available?)
• Multivariate stochastic approximation using a simultaneous perturbation gradient approximation
• Natural actor-critic algorithms
• OnActor-Critic Algorithms
• Regression on fixed-rank positive semidefinite matrices: a Riemannian approach
• Simulation-based optimization of Markov reward processes
• Stochastic approximation methods for constrained and unconstrained systems
• Stochastic approximation with two time scales
• Stochastic approximation. A dynamical systems viewpoint.
• Stochastic recursive algorithms for optimization. Simultaneous perturbation methods
• The Geometry of Algorithms with Orthogonality Constraints

Cited in

(13)

• Natural actor-critic algorithms
• Two-phase iteration for value function approximation and hyperparameter optimization in Gaussian-kernel-based adaptive critic design
• Reinforcement learning based algorithms for average cost Markov decision processes
• A convergent online single time scale actor critic algorithm
• Actor-critic algorithms based on symmetric perturbation sampling
• Natural actor-critic based on batch recursive least-squares
• An online actor-critic algorithm with function approximation for constrained Markov decision processes
• OnActor-Critic Algorithms
• Artificial Intelligence and Soft Computing - ICAISC 2004
• Full gradient DQN reinforcement learning: a provably convergent scheme
• Variational actor-critic algorithms,
• Error controlled actor-critic
• Stochastic approximation and reinforcement learning: the interface and a little beyond

This page was built for publication: Actor-critic algorithms with online feature adaptation

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5270681)