Split-door criterion: identification of causal effects through auxiliary outcomes
From MaRDI portal
Abstract: We present a method for estimating causal effects in time series data when fine-grained information about the outcome of interest is available. Specifically, we examine what we call the split-door setting, where the outcome variable can be split into two parts: one that is potentially affected by the cause being studied and another that is independent of it, with both parts sharing the same (unobserved) confounders. We show that under these conditions, the problem of identification reduces to that of testing for independence among observed variables, and present a method that uses this approach to automatically find subsets of the data that are causally identified. We demonstrate the method by estimating the causal impact of Amazon's recommender system on traffic to product pages, finding thousands of examples within the dataset that satisfy the split-door criterion. Unlike past studies based on natural experiments that were limited to a single product category, our method applies to a large and representative sample of products viewed on the site. In line with previous work, we find that the widely-used click-through rate (CTR) metric overestimates the causal impact of recommender systems; depending on the product category, we estimate that 50-80% of the traffic attributed to recommender systems would have happened even without any recommendations. We conclude with guidelines for using the split-door criterion as well as a discussion of other contexts where the method can be applied.
Recommendations
- Inferring causal impact using Bayesian structural time-series models
- On identifying causal effects
- Time series experiments and causal estimands: exact randomization tests and trading
- IDENTIFIABILITY CRITERIA FOR CAUSAL EFFECTS OF JOINT INTERVENTIONS
- Time-series estimation of the effects of natural experiments
Cites work
- A Direct Approach to False Discovery Rates
- A review of modern multiple hypothesis testing, with particular attention to the false discovery proportion
- A survey of exact inference for contingency tables. With comments and a rejoinder by the author
- Adaptive and Dynamic Adaptive Procedures for False Discovery Rate Control and Estimation
- Causal inference for statistics, social, and biomedical sciences. An introduction
- Causality. Models, reasoning, and inference
- Causation, prediction, and search
- Design of observational studies
- Estimation of Entropy and Mutual Information
- Identification of Causal Effects Using Instrumental Variables
- Matched Sampling for Causal Effects
- Matching methods for causal inference: a review and a look forward
- Measuring and testing dependence by correlation of distances
- Multiple-Testing Strategy for Analyzing cDNA Array Data on Gene Expression
- Recommender Systems Handbook
- Split-door criterion: identification of causal effects through auxiliary outcomes
- Using Secondary Outcomes to Sharpen Inference in Randomized Experiments With Noncompliance
Cited in
(2)
This page was built for publication: Split-door criterion: identification of causal effects through auxiliary outcomes
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1728679)
