Kalman temporal differences
From MaRDI portal
Abstract: Because reinforcement learning suffers from a lack of scalability, online value (and Q-) function approximation has received increasing interest this last decade. This contribution introduces a novel approximation scheme, namely the Kalman Temporal Differences (KTD) framework, that exhibits the following features: sample-efficiency, non-linear approximation, non-stationarity handling and uncertainty management. A first KTD-based algorithm is provided for deterministic Markov Decision Processes (MDP) which produces biased estimates in the case of stochastic transitions. Than the eXtended KTD framework (XKTD), solving stochastic MDP, is described. Convergence is analyzed for special cases for both deterministic and stochastic transitions. Related algorithms are experimented on classical benchmarks. They compare favorably to the state of the art while exhibiting the announced features.
Recommendations
- A generalized Kalman filter for fixed point approximation and efficient temporal-difference learning
- Kernel-based reinforcement learning
- Practical kernel-based reinforcement learning
- Linear least-squares algorithms for temporal difference learning
- Linear least-squares algorithms for temporal difference learning
Cited in
(6)- A generalized Kalman filter for fixed point approximation and efficient temporal-difference learning
- Practical kernel-based reinforcement learning
- Reinforcement learning
- Distributed nonlinear Kalman filter with communication protocol
- Dopamine, inference, and uncertainty
- Temporal difference learning with incremental nearest neighbors in continuous spaces
This page was built for publication: Kalman temporal differences
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3055813)
