Learning algorithms for Markov decision processes with average cost
The authors propose and give for the first time a rigorous convergence analysis of two \(Q\)-learning algorithms for the optimal average cost control problem. One is a stochastic approximation analogue of the original relative value iteration (RVI) algorithm, another is a stochastic approximation analogue of a value iteration algorithm based on the stochastic shortest path (SSP) formulation of the problem [cf. \textit{D. Bertsekas} and \textit{J. N. Tsitsiklis}, Neuro-dynamic programming, Athena Scientific, Belmont, MA (1996; Zbl 0924.68163); \textit{V. S. Borkar} and \textit{S. P. Meyn}, SIAM J. Control Optimization 38, 447-469 (2000; Zbl 0990.62071)]. Both synchronous and asynchronous implementations are considered and analyzed here via studying the a.s. boundedness of \(\{Q^n\}\) and the stability behavior of the associated ODE for the \(Q\)-factor. Meaningfully, it is pointed out that due to a large state space, the use of acceleration techniques is worth exploring, while in reinforcement learning, the convergence rate is usually slow. Moreover, the case of an infinite state space is still an open issue. These may be important directions for this area in the future.
- Reinforcement learning based algorithms for average cost Markov decision processes
- Empirical Q-value iteration
- Reinforcement learning for long-run average cost.
- Actor-Critic--Type Learning Algorithms for Markov Decision Processes
- Q-learning and policy iteration algorithms for stochastic shortest path problems
- Natural actor-critic algorithms
- Reinforcement learning for long-run average cost.
- Approachability in Stackelberg stochastic games with vector costs
- Solutions of the average cost optimality equation for Markov decision processes with weakly continuous kernel: the fixed-point approach revisited
- Q-learning for Markov decision processes with a satisfiability criterion
- A reinforcement learning algorithm based on policy iteration for average reward: Empirical results with yield management and convergence analysis
- Average cost temporal-difference learning
- \(L^\ast\)-based learning of Markov decision processes (extended version)
- Fundamental design principles for reinforcement learning algorithms
- Batch policy learning in average reward Markov decision processes
- Whittle index based Q-learning for restless bandits with average reward
- A sojourn-based approach to semi-Markov reinforcement learning
- Deep reinforcement learning for wireless sensor scheduling in cyber-physical systems
- Fitted Q-iteration by functional networks for control problems
- Learning dynamic prices in electronic retail markets with customer segmentation
- Dynamic pricing models for electronic business
- Reinforcement learning based algorithms for average cost Markov decision processes
- Relative value iteration algorithm with soft state aggregation
- Empirical dynamic programming
- Q-learning and enhanced policy iteration in discounted dynamic programming
- Multiscale Q-learning with linear function approximation
- Opportunistic Transmission over Randomly Varying Channels
- Look-ahead control of conveyor-serviced production station by using potential-based online policy iteration
- Learning algorithms for Markov decision processes
- Q-learning and policy iteration algorithms for stochastic shortest path problems
- Analyzing anonymity attacks through noisy channels
- Risk-Sensitive Reinforcement Learning via Policy Gradient Search
- Variance-penalized Markov decision processes: dynamic programming and reinforcement learning techniques
- Optimal Distributed Uplink Channel Allocation: A Constrained MDP Formulation
- A perturbation approach to approximate value iteration for average cost Markov decision processes with Borel spaces and bounded costs.
- Approximation of average cost Markov decision processes using empirical distributions and concentration inequalities
- Empirical Q-value iteration
- A framework for transforming specifications in reinforcement learning
- Optimal sensor scheduling for remote state estimation with limited bandwidth: a deep reinforcement learning approach
- Stochastic Fixed-Point Iterations for Nonexpansive Maps: Convergence and Error Bounds
- Stochastic approximation and reinforcement learning: the interface and a little beyond
- Decision-making problem for two-player Markov game: perspective of feedback control
- Quantifying the likelihood of learning collusive strategy equilibria
- Stochastic approximation in non-Markovian environments
- The ODE method for stochastic approximation and reinforcement learning with Markovian noise
- Scalable Whittle index policy for real-time storage allocation in railway container yard
- Asynchronous stochastic approximation with applications to average-reward reinforcement learning
This page was built for publication: Learning algorithms for Markov decision processes with average cost
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2753225)
