General time consistent discounting

DOI10.1016/J.TCS.2013.09.022MaRDI QIDQ391749zbMATH OpenOpenAlexDBLPWikidataFDO

Authors Tor Lattimore, Marcus Hutter

Publication date 13 January 2014

Published in Theoretical Computer Science (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1107.5528

zbMATH Keywords

game theorygeneral discountingrational agentssequential decision theorytime-consistency

Mathematics Subject Classification ID

Agent technology and artificial intelligence (68T42) Applications of game theory (91A80)

Abstract: A possibly immortal agent tries to maximise its summed discounted rewards over time, where discounting is used to avoid infinite utilities and encourage the agent to value current rewards more than future ones. Some commonly used discount functions lead to time-inconsistent behavior where the agent changes its plan over time. These inconsistencies can lead to very poor behavior. We generalise the usual discounted utility model to one where the discount function changes with the age of the agent. We then give a simple characterisation of time-(in)consistent discount functions and show the existence of a rational policy for an agent that knows its discount function is time-inconsistent.

Recommendations

• Time consistent discounting
• Heterogeneous discounting in economic problems
• Risk-induced discounting
• Trichotomic discounted utility
• Optimal discounting

Cites work

• A course in game theory.
• Asymptotically efficient adaptive allocation rules
• Consistent Plans
• General Discounting Versus Average Reward
• scientific article; zbMATH DE number 4078557 (Why is no real title available?)
• scientific article; zbMATH DE number 3638998 (Why is no real title available?)
• scientific article; zbMATH DE number 1321699 (Why is no real title available?)
• On the Existence of a Consistent Course of Action when Tastes are Changing
• Self-Optimizing and Pareto-Optimal Policies in General Environments based on Bayes-Mixtures
• Stationary Ordinal Utility and Impatience
• Subgame-perfect equilibria of finite- and infinite-horizon games
• Universal artificial intelligence. Sequential decisions based on algorithmic probability.

Cited in

(9)

• On the computability of Solomonoff induction and AIXI
• Information, inattention, perception, and discounting
• Time consistent discounting
• Extreme state aggregation beyond Markov decision processes
• Optimal discounting
• Learning time dependent choice
• New discounting functions
• Reward tampering problems and solutions in reinforcement learning: a causal influence diagram perspective
• A subgame perfect equilibrium reinforcement learning approach to time-inconsistent problems

This page was built for publication: General time consistent discounting

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q391749)