Constructing effective personalized policies using counterfactual inference from biased data sets with many features

DOI10.1007/S10994-018-5768-3zbMATH Open1493.68290arXiv1612.08082OpenAlexW2963806098MaRDI QIDQ2425241FDOQ2425241

Authors: Onur Atan, William R. Zame, Qiaojun Feng, Mihaela van der Schaar 

Publication date: 26 June 2019

Published in: Machine Learning (Search for Journal in Brave)

Abstract: This paper proposes a novel approach for constructing effective personalized policies when the observed data lacks counter-factual information, is biased and possesses many features. The approach is applicable in a wide variety of settings from healthcare to advertising to education to finance. These settings have in common that the decision maker can observe, for each previous instance, an array of features of the instance, the action taken in that instance, and the reward realized -- but not the rewards of actions that were not taken: the counterfactual information. Learning in such settings is made even more difficult because the observed data is typically biased by the existing policy (that generated the data) and because the array of features that might affect the reward in a particular instance -- and hence should be taken into account in deciding on an action in each particular instance -- is often vast. The approach presented here estimates propensity scores for the observed data, infers counterfactuals, identifies a (relatively small) number of features that are (most) relevant for each possible action and instance, and prescribes a policy to be followed. Comparison of the proposed algorithm against the state-of-art algorithm on actual datasets demonstrates that the proposed algorithm achieves a significant improvement in performance.

Full work available at URL: https://arxiv.org/abs/1612.08082

Recommendations

• More efficient policy learning via optimal retargeting
• Counterfactual reasoning and learning systems: the example of computational advertising
• Policy learning with observational data
• Efficient augmentation and relaxation learning for individualized treatment rules using observational data
• A reinforcement learning approach to personalized learning recommendation systems

zbMATH Keywords

constructing personalized policiesidentifying relevant featuresinferring counterfactuals

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Estimation in multivariate analysis (62H12) Computational aspects of data analysis and big data (68T09)

Cites Work

• Theoretical and empirical analysis of ReliefF and RReliefF
• Exploration-exploitation tradeoff using variance estimates in multi-armed bandits
• Recursive partitioning for heterogeneous causal effects
• Estimation and Inference of Heterogeneous Treatment Effects using Random Forests
• A Simple Method for Estimating Interactions Between a Treatment and a Large Number of Covariates
• Pattern classification.
• The central role of the propensity score in observational studies for causal effects
• Title not available (Why is that?)
• Doubly robust policy evaluation and optimization
• Title not available (Why is that?)
• Simple statistical gradient-following algorithms for connectionist reinforcement learning
• Title not available (Why is that?)
• 10.1162/153244303322753751
• Title not available (Why is that?)
• Feature selection for unsupervised learning
• Use of the Logistic Model in Retrospective Studies
• Title not available (Why is that?)

Uses Software

• ReliefF

This page was built for publication: Constructing effective personalized policies using counterfactual inference from biased data sets with many features

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2425241)