Trajectory modeling via random utility inverse reinforcement learning

DOI10.1016/J.INS.2024.120128MaRDI QIDQ6198767zbMATH OpenOpenAlexFDO

Authors Anselmo R. Pitombeira-Neto, Helano P. Santos, Ticiana L. Coelho da Silva, José Antônio F. de Macêdo

Publication date 20 March 2024

Published in Information Sciences (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2105.12092

zbMATH Keywords

external sensor datainverse reinforcement learningrandom utility theorytrajectory modeling

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Transportation, logistics and supply chain management (90B06) Traffic problems in operations research (90B20)

Abstract: We consider the problem of modeling trajectories of drivers in a road network from the perspective of inverse reinforcement learning. Cars are detected by sensors placed on sparsely distributed points on the street network of a city. As rational agents, drivers are trying to maximize some reward function unknown to an external observer. We apply the concept of random utility from econometrics to model the unknown reward function as a function of observed and unobserved features. In contrast to current inverse reinforcement learning approaches, we do not assume that agents act according to a stochastic policy; rather, we assume that agents act according to a deterministic optimal policy and show that randomness in data arises because the exact rewards are not fully observed by an external observer. We introduce the concept of extended state to cope with unobserved features and develop a Markov decision process formulation of drivers decisions. We present theoretical results which guarantee the existence of solutions and show that maximum entropy inverse reinforcement learning is a particular case of our approach. Finally, we illustrate Bayesian inference on model parameters through a case study with real trajectory data from a large city in Brazil.

Recommendations

• Inverse reinforcement learning from summary data
• Inverse reinforcement learning in partially observable environments
• Model-based inverse reinforcement learning for deterministic systems
• Bayesian learning of noisy Markov decision processes
• Data-driven inverse optimization with imperfect information

Cites work

• A spatial-temporal-semantic neural network algorithm for location prediction on moving objects
• A survey of inverse reinforcement learning: challenges, methods and progress
• Discrete Choice Methods with Simulation
• scientific article; zbMATH DE number 3965301 (Why is no real title available?)
• scientific article; zbMATH DE number 3665682 (Why is no real title available?)
• scientific article; zbMATH DE number 700091 (Why is no real title available?)
• scientific article; zbMATH DE number 2243374 (Why is no real title available?)
• Monte Carlo sampling methods using Markov chains and their applications
• Optimization models
• Reinforcement learning. An introduction
• Transportation systems analysis. Models and applications.

Cited in

(1)

• Rationally inattentive inverse reinforcement learning explains YouTube commenting behavior

This page was built for publication: Trajectory modeling via random utility inverse reinforcement learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6198767)