A multiagent reinforcement learning framework for off-policy evaluation in two-sided markets
From MaRDI portal
Publication:6138596
DOI10.1214/22-AOAS1700arXiv2202.10574MaRDI QIDQ6138596FDOQ6138596
Hongtu Zhu, Shikai Luo, Chengchun Shi, Ge Song, Runzhe Wan, Rui Song
Publication date: 16 January 2024
Published in: The Annals of Applied Statistics (Search for Journal in Brave)
Abstract: The two-sided markets such as ride-sharing companies often involve a group of subjects who are making sequential decisions across time and/or location. With the rapid development of smart phones and internet of things, they have substantially transformed the transportation landscape of human beings. In this paper we consider large-scale fleet management in ride-sharing companies that involve multiple units in different areas receiving sequences of products (or treatments) over time. Major technical challenges, such as policy evaluation, arise in those studies because (i) spatial and temporal proximities induce interference between locations and times; and (ii) the large number of locations results in the curse of dimensionality. To address both challenges simultaneously, we introduce a multi-agent reinforcement learning (MARL) framework for carrying policy evaluation in these studies. We propose novel estimators for mean outcomes under different products that are consistent despite the high-dimensionality of state-action space. The proposed estimator works favorably in simulation experiments. We further illustrate our method using a real dataset obtained from a two-sided marketplace company to evaluate the effects of applying different subsidizing policies. A Python implementation of our proposed method is available at https://github.com/RunzheStat/CausalMARL.
Full work available at URL: https://arxiv.org/abs/2202.10574
Cites Work
- Title not available (Why is that?)
- Personalized Policy Learning Using Longitudinal Mobile Health Data
- Evaluating marker-guided treatment selection strategies
- Title not available (Why is that?)
- Estimation and Inference of Heterogeneous Treatment Effects using Random Forests
- Causal Inference for Statistics, Social, and Biomedical Sciences
- Toward Causal Inference With Interference
- Program Evaluation and Causal Inference With High-Dimensional Data
- Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score
- Using decision lists to construct interpretable and parsimonious treatment regimes
- Statistical inference for the mean outcome under a possibly non-unique optimal treatment strategy
- Estimating Individualized Treatment Rules Using Outcome Weighted Learning
- Time Series Experiments and Causal Estimands: Exact Randomization Tests and Trading
- Performance guarantees for individualized treatment rules
- Basic properties of strong mixing conditions. A survey and some open questions
- High-dimensional \(A\)-learning for optimal dynamic treatment regimes
- Penalized Q-learning for dynamic treatment regimens
- Optimal Dynamic Treatment Regimes
- A Robust Method for Estimating Optimal Treatment Regimes
- Inference for non-regular parameters in optimal dynamic treatment regimes
- Optimal Structural Nested Models for Optimal Sequential Decisions
- Robust estimation of optimal dynamic treatment regimes for sequential treatment decisions
- Doubly robust policy evaluation and optimization
- New Statistical Learning Methods for Estimating Optimal Dynamic Treatment Regimes
- Title not available (Why is that?)
- Interpretable Dynamic Treatment Regimes
- Learning Optimal Distributionally Robust Individualized Treatment Rules
- Exactp-Values for Network Interference
- Multi-agent reinforcement learning: a selective overview of theories and algorithms
- Title not available (Why is that?)
- Regularized policy iteration with nonparametric function spaces
- Bayesian method for causal inference in spatially-correlated multivariate time series
- Estimating Dynamic Treatment Regimes in Mobile Health Using V-Learning
- Batch policy learning in average reward Markov decision processes
- Off-Policy Estimation of Long-Term Average Outcomes With Applications to Mobile Health
- Resampling‐based confidence intervals for model‐free robust inference on optimal treatment regimes
- Quantile-Optimal Treatment Regimes
- Title not available (Why is that?)
- The stratified micro-randomized trial design: sample size considerations for testing nested causal effects of time-varying treatments
- Maximin Projection Learning for Optimal Treatment Decision with Heterogeneous Individualized Treatment Effects
- Efficiently Breaking the Curse of Horizon in Off-Policy Evaluation with Double Reinforcement Learning
- Greedy outcome weighted tree learning of optimal personalized treatment rules
This page was built for publication: A multiagent reinforcement learning framework for off-policy evaluation in two-sided markets
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6138596)
