Dynamic Optimization and Learning for Renewal Systems

DOI10.1109/TAC.2012.2204831MaRDI QIDQ5353069zbMATH OpenFDO

Authors Michael J. Neely

Publication date 8 September 2017

Published in IEEE Transactions on Automatic Control (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1011.5942

Mathematics Subject Classification ID

Dynamic programming (90C39) Markov and semi-Markov decision processes (90C40)

Abstract: We consider the problem of optimizing time averages in systems with independent and identically distributed behavior over renewal frames. This includes scheduling and task processing to maximize utility in stochastic networks with variable length scheduling modes. Every frame, a new policy is implemented that affects the frame size and that creates a vector of attributes. An algorithm is developed for choosing policies on each frame in order to maximize a concave function of the time average attribute vector, subject to additional time average constraints. The algorithm is based on Lyapunov optimization concepts and involves minimizing a ``drift-plus-penalty ratio over each frame. The algorithm can learn efficient behavior without a-priori statistical knowledge by sampling from the past. Our framework is applicable to a large class of problems, including Markov decision problems.

Cited in

(3)

• Opportunistic spectrum access in unslotted primary systems
• scientific article; zbMATH DE number 7626794 (Why is no real title available?)
• Asynchronous optimization over weakly coupled renewal systems

This page was built for publication: Dynamic Optimization and Learning for Renewal Systems

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5353069)