A panel quantile approach to attrition bias in big data: evidence from a randomized experiment

DOI10.1016/J.JECONOM.2018.12.006zbMATH Open1452.62917arXiv1808.03364OpenAlexW2886960092WikidataQ128725743 ScholiaQ128725743MaRDI QIDQ2000849FDOQ2000849

Carlos Lamarche, Matthew Harding

Publication date: 1 July 2019

Published in: Journal of Econometrics (Search for Journal in Brave)

Abstract: This paper introduces a quantile regression estimator for panel data models with individual heterogeneity and attrition. The method is motivated by the fact that attrition bias is often encountered in Big Data applications. For example, many users sign-up for the latest program but few remain active users several months later, making the evaluation of such interventions inherently very challenging. Building on earlier work by Hausman and Wise (1979), we provide a simple identification strategy that leads to a two-step estimation procedure. In the first step, the coefficients of interest in the selection equation are consistently estimated using parametric or nonparametric methods. In the second step, standard panel quantile methods are employed on a subset of weighted observations. The estimator is computationally easy to implement in Big Data applications with a large number of subjects. We investigate the conditions under which the parameter estimator is asymptotically Gaussian and we carry out a series of Monte Carlo simulations to investigate the finite sample properties of the estimator. Lastly, using a simulation exercise, we apply the method to the evaluation of a recent Time-of-Day electricity pricing experiment inspired by the work of Aigner and Hausman (1980).

Full work available at URL: https://arxiv.org/abs/1808.03364

zbMATH Keywords

quantile regressionbig dataattritionindividual effectstime-of-day pricing

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Nonparametric regression and quantile regression (62G08) Applications of statistics to economics (62P20) Statistical aspects of big data and data science (62R07)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Regression rank scores and regression quantiles
• Censored Regression Quantiles
• Locally Weighted Censored Quantile Regression
• Inference and missing data
• Limiting distributions for \(L_1\) regression estimators under general conditions
• Asymptotics for Lasso-type estimators.
• \(\ell_1\)-penalized quantile regression in high-dimensional sparse models
• Nonparametric identification in panels using quantiles
• Average and quantile effects in nonseparable panel models
• Estimation of Censored Quantile Regression for Panel Data With Fixed Effects
• Analysis of Semiparametric Regression Models for Repeated Outcomes in the Presence of Missing Data
• A simple approach to quantile regression for panel data
• Asymptotics for panel quantile regression models with individual effects
• Instrumental Variables Estimates of the Effect of Subsidized Training on the Quantiles of Trainee Earnings
• Set identification via quantile restrictions in short panels
• Inverse probability weighted estimation for general missing data problems
• Quantile regression for longitudinal data
• Estimating and testing a quantile regression model with interactive effects
• Quantile Regression Methods for Longitudinal Data with Drop-outs: Application to CD4 Cell Counts of Patients Infected with the Human Immunodeficiency Virus
• Robust penalized quantile regression estimation for panel data
• Analysis of Panel Data
• Inference for imputation estimators
• Simple estimators for nonparametric panel data models with sample attrition
• Handling attrition in longitudinal studies: the case for refreshment samples
• Inference in panel data models under attrition caused by unobservables
• Combining Panel Data Sets with Attrition and Refreshment Samples
• Estimation of a Panel Data Sample Selection Model
• Three-Step Censored Quantile Regression and Extramarital Affairs
• Multiple imputation in quantile regression
• Analysis of Multivariate Longitudinal Outcomes With Nonignorable Dropouts and Missing Covariates
• Quantile regression with covariates missing at random
• Efficient Quantile Regression Analysis With Missing Observations
• An informative subset-based estimator for censored quantile regression
• Efficient estimation of quantiles in missing data models

Cited In (4)

• Annals issue in honor of Jerry A. Hausman. Editors' introduction
• Wild bootstrap inference for penalized quantile regression for longitudinal data
• Machine learning panel data regressions with heavy-tailed dependent data: theory and application
• Analysis of global and local optima of regularized quantile regression in high dimensions: a subgradient approach

Uses Software

• quantreg

Recommendations

• Simple estimators for nonparametric panel data models with sample attrition 👍 👎
• Inference in panel data models under attrition caused by unobservables 👍 👎
• Combining Panel Data Sets with Attrition and Refreshment Samples 👍 👎
• Bayesian latent pattern mixture models for handling attrition in panel studies with refreshment samples 👍 👎
• Experimental Design Issues in Big Data: The Question of Bias 👍 👎
• Dealing with the biased effects issue when handling huge datasets: the case of INVALSI data 👍 👎

This page was built for publication: A panel quantile approach to attrition bias in big data: evidence from a randomized experiment

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2000849)