Experimental Evaluation of Individualized Treatment Rules

DOI10.1080/01621459.2021.1923511zbMATH Open1514.62239arXiv1905.05389MaRDI QIDQ112156FDOQ112156

Authors: Kosuke Imai, Michael Lingzhi Li, Kosuke Imai, Michael Lingzhi Li 

Publication date: 14 May 2019

Published in: Journal of the American Statistical Association (Search for Journal in Brave)

Abstract: The increasing availability of individual-level data has led to numerous applications of individualized (or personalized) treatment rules (ITRs). Policy makers often wish to empirically evaluate ITRs and compare their relative performance before implementing them in a target population. We propose a new evaluation metric, the population average prescriptive effect (PAPE). The PAPE compares the performance of ITR with that of non-individualized treatment rule, which randomly treats the same proportion of units. Averaging the PAPE over a range of budget constraints yields our second evaluation metric, the area under the prescriptive effect curve (AUPEC). The AUPEC represents an overall performance measure for evaluation, like the area under the receiver and operating characteristic curve (AUROC) does for classification, and is a generalization of the QINI coefficient utilized in uplift modeling. We use Neyman's repeated sampling framework to estimate the PAPE and AUPEC and derive their exact finite-sample variances based on random sampling of units and random assignment of treatment. We extend our methodology to a common setting, in which the same experimental data is used to both estimate and evaluate ITRs. In this case, our variance calculation incorporates the additional uncertainty due to random splits of data used for cross-validation. The proposed evaluation metrics can be estimated without requiring modeling assumptions, asymptotic approximation, or resampling methods. As a result, it is applicable to any ITR including those based on complex machine learning algorithms. The open-source software package is available for implementing the proposed methodology.

Full work available at URL: https://arxiv.org/abs/1905.05389

Recommendations

• Performance guarantees for individualized treatment rules
• Estimation and evaluation of linear individualized treatment rules to guarantee performance
• Tree-based methods for individualized treatment regimes
• Estimating individualized treatment rules using outcome weighted learning
• Optimal Individualized Decision Rules Using Instrumental Variable Methods

zbMATH Keywords

causal inferencemachine learningheterogeneous treatment effectsprecision medicineuplift modeling

Mathematics Subject Classification ID

Applications of statistics to biology and medical sciences; meta analysis (62P10) Learning and adaptive systems in artificial intelligence (68T05) Causal inference from observational studies (62D20)

Cites Work

• Estimating treatment effect heterogeneity in randomized program evaluation
• BART: Bayesian additive regression trees
• Generalized random forests
• Title not available (Why is that?)
• Recursive partitioning for heterogeneous causal effects
• Estimation and Inference of Heterogeneous Treatment Effects using Random Forests
• Causal inference for statistics, social, and biomedical sciences. An introduction
• Bayesian regression tree models for causal inference: regularization, confounding, and heterogeneous effects (with discussion)
• Statistical inference for the mean outcome under a possibly non-unique optimal treatment strategy
• Who should be treated? Empirical welfare maximization methods for treatment choice
• Estimating individualized treatment rules using outcome weighted learning
• Title not available (Why is that?)
• Statistical treatment rules for heterogeneous populations.
• Performance guarantees for individualized treatment rules

Cited In (10)

• Selecting and ranking individualized treatment rules with unmeasured confounding
• Evaluating marker-guided treatment selection strategies
• Do machine learning methods lead to similar individualized treatment rules? A comparison study on real data
• Subgroup analysis using Bernoulli‐gated hierarchical mixtures of experts models
• Title not available (Why is that?)
• Estimation and evaluation of linear individualized treatment rules to guarantee performance
• Estimating individualized treatment rules for ordinal treatments
• Modern approaches for evaluating treatment effect heterogeneity from clinical trials and observational data
• evalITR
• Performance guarantees for individualized treatment rules

This page was built for publication: Experimental Evaluation of Individualized Treatment Rules

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q112156)