Adaptive importance sampling for value function approximation in off-policy reinforcement learning

DOI10.1016/J.NEUNET.2009.01.002MaRDI QIDQ1784527zbMATH OpenOpenAlexDBLPWikidataFDO

Authors Hirotaka Hachiya, Takayuki Akiyama, Masashi Sugiayma, Jan Peters

Publication date 27 September 2018

Published in Neural Networks (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1016/j.neunet.2009.01.002

zbMATH Keywords

adaptive importance samplingefficient sample reuseimportance-weighted cross-validationoff-policy reinforcement learningpolicy iterationvalue function approximation

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Recommendations

• Importance sampling techniques for policy optimization
• Efficient sample reuse in policy gradients with parameter-based exploration
• Off-policy temporal difference learning with distribution adaptation in fast mixing chains
• Efficient exploration through active learning for value function approximation in reinforcement learning
• An emphatic approach to the problem of off-policy temporal-difference learning

Cites work

• 10.1162/1532443041827907
• Covariate shift adaptation by importance weighted cross validation
• scientific article; zbMATH DE number 1321699 (Why is no real title available?)
• scientific article; zbMATH DE number 854710 (Why is no real title available?)
• Improving predictive inference under covariate shift by weighting the log-likelihood function
• Linear Statistical Inference and its Applications
• The elements of statistical learning. Data mining, inference, and prediction
• Trading Variance Reduction with Unbiasedness: The Regularized Subspace Information Criterion for Robust Model Selection in Kernel Regression

Cited in

(13)

• Dimensionality reduction for density ratio estimation in high-dimensional spaces
• Efficient exploration through active learning for value function approximation in reinforcement learning
• Least-squares two-sample test
• Importance sampling in reinforcement learning with an estimated behavior policy
• Multivariate error modeling and uncertainty quantification using importance (re-)weighting for Monte Carlo simulations in particle transport
• Reward-weighted regression with sample reuse for direct policy search in reinforcement learning
• Importance sampling techniques for policy optimization
• Direct density-ratio estimation with dimensionality reduction via least-squares hetero-distributional subspace search
• Policy Gradient Importance Sampling for Bayesian Inference
• Daisee: Adaptive importance sampling by balancing exploration and exploitation
• Learning under nonstationarity: covariate shift and class-balance change
• Model-based policy gradients with parameter-based exploration by least-squares conditional density estimation
• Semi-supervised speaker identification under covariate shift

This page was built for publication: Adaptive importance sampling for value function approximation in off-policy reinforcement learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1784527)