Markov-Renewal Programming. I: Formulation, Finite Return Models

DOI10.1287/opre.11.6.938zbMath0126.15905OpenAlexW2139301602MaRDI QIDQ5332971

William S. Jewell

Publication date: 1963

Published in: Operations Research (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1287/opre.11.6.938

zbMATH Keywords

operations research

Related Items (68)

Variational characterizations in Markov decision processes ⋮ Computation techniques for large scale undiscounted markov decision processes ⋮ On the existence of relative values for undiscounted Markovian decision processes with a scalar gain rate ⋮ Discounted Stochastic Ratio Games ⋮ On undiscounted semi-Markov decision processes with absorbing states ⋮ Block-scaling of value-iteration for discounted Markov renewal programming ⋮ Replacement process decomposition for discounted Markov renewal programming ⋮ Solving Markovian decision processes by successive elimination of variables ⋮ A Brouwer fixed-point mapping approach to communicating Markov decision processes ⋮ On the solvability of Bellman's functional equations for Markov renewal programming ⋮ Sorbe una aplicacion de los procesos de renovacion markovianos a los juegos estocasticos no terminativos ⋮ Sensitivitätsanalysen in entscheidungsmodellen ⋮ Programacion secuencial en concurrencia con factor descuento continuo ⋮ Generalized Markovian decision processes ⋮ Computing semi-stationary optimal policies for multichain semi-Markov decision processes ⋮ Semi‐Markov processes and mobility† ⋮ A value-iteration scheme for undiscounted multichain Markov renewal programs ⋮ Nonstationary Markov decision problems with converging parameters ⋮ Application of Markov renewal theory and <scp>semi‐Markov</scp> decision processes in maintenance modeling and optimization of multi‐unit systems ⋮ Critique of probabilistic models: Application of the Semi‐Markov model to migration ⋮ Markov renewal decision processes with finite horizon ⋮ A natural extension of the MacQueen extrapolation ⋮ Unnamed Item ⋮ Unnamed Item ⋮ Optimal dynamic routing in flexible manufacturing systems with limited buffers ⋮ Obituary: William S. Jewell (1932--2003) ⋮ Denumerable semi-Markov decision chains with small interest rates ⋮ Some models of queueing control with switchover ⋮ Unnamed Item ⋮ A Policy Improvement Algorithm for Solving a Mixture Class of Perfect Information and AR-AT Semi-Markov Games ⋮ Solution of the unconditional extremum problem for a linear-fractional integral functional on a set of probability measures ⋮ A decision exclusion algorithm for a class of Markovian Decision Processes ⋮ Optimal control of semi-Markov processes with a backward stochastic differential equations approach ⋮ On-line scheduling of a robotic manufacturing cell with stochastic sequence-dependent processing rates ⋮ Research and optimization of semi-Markov queueing models ⋮ Semi-Markov decision processes with limiting ratio average rewards ⋮ On the existence of relative values for undiscounted multichain Markov decision processes ⋮ Markov ratio decision processes ⋮ On a control of a Markov chain under conditions with respect to the absolute stationary probabilities and cost ⋮ A Fixed Point Approach to Undiscounted Markov Renewal Programs ⋮ The bellman equation for vector-valued semi-markovian dyanmic programiing ⋮ Semi-Markov processes and their applications ⋮ Solving stochastic dynamic programming problems by linear programming — An annotated bibliography ⋮ Bounds on the fixed point of a monotone contraction operator ⋮ Exact distributions for reward functions on semi-Markov and Markov additive processes ⋮ Dynamic programming of expectation and variance ⋮ A second-order approximation for the variance of a renewal reward process ⋮ Continuous time control of the arrival process in an M/G/1 queue ⋮ Foolproof convergence in multichain policy iteration ⋮ Controlled semi-Markov processes with constraints on control strategies and construction of optimal strategies in reliability and safety models ⋮ Contraction mappings underlying undiscounted Markov decision problems ⋮ Solutions of semi-Markov control models with recursive discount rates and approximation by $\epsilon-$optimal policies ⋮ Markov Branching Decision Chains with Interest-Rate-Dependent Rewards ⋮ Improved iterative computation of the expected discounted return in Markov and semi-Markov chains ⋮ The Structure of the Functional of Accumulation Defined on a Trajectory of Semi-Markov Process with a Finite Set of States ⋮ The optimization of K-effect models by linear and dynamic programming ⋮ Fluctuations of a renewal-reward process ⋮ Finite state continuous time Markov decision processes with an infinite planning horizon ⋮ Multiple perspective dynamic decision making ⋮ A further anticycling rule in multichain policy iteration for undiscounted Markov renewal programs ⋮ Optimal control of stationary Markov processes ⋮ Semi-Markov decision processes with vector pay-offs ⋮ DYNAMIC ANALYSIS OF A MULTIVARIATE REWARD PROCESS DEFINED ON THE UMCP WITH APPLICATION TO OPTIMAL PREVENTIVE MAINTENANCE POLICY PROBLEMS IN MANUFACTURING ⋮ Computation of optimal policies in discounted semi-Markov decision chains ⋮ Generalized polynomial approximations in Markovian decision processes ⋮ The variational calculus and approximation in policy space for Markovian decision processes ⋮ Zur Extrapolation in Markoffschen Entscheidungsmodellen mit Diskontierung ⋮ Research of Optimum Strategy for Semi-Markov Queueing Models at Control of CBSMAP-Flow. Algorithmization

This page was built for publication: Markov-Renewal Programming. I: Formulation, Finite Return Models