Personalized Policy Learning Using Longitudinal Mobile Health Data
From MaRDI portal
Abstract: We address the personalized policy learning problem using longitudinal mobile health application usage data. Personalized policy represents a paradigm shift from developing a single policy that may prescribe personalized decisions by tailoring. Specifically, we aim to develop the best policy, one per user, based on estimating random effects under generalized linear mixed model. With many random effects, we consider new estimation method and penalized objective to circumvent high-dimension integrals for marginal likelihood approximation. We establish consistency and optimality of our method with endogenous app usage. We apply our method to develop personalized push ("prompt") schedules in 294 app users, with a goal to maximize the prompt response rate given past app usage and other contextual factors. We found the best push schedule given the same covariates varied among the users, thus calling for personalized policies. Using the estimated personalized policies would have achieved a mean prompt response rate of 23% in these users at 16 weeks or later: this is a remarkable improvement on the observed rate (11%), while the literature suggests 3%-15% user engagement at 3 months after download. The proposed method compares favorably to existing estimation methods including using the R function "glmer" in a simulation study.
Recommendations
- Off-policy estimation of long-term average outcomes with applications to mobile health
- Estimating dynamic treatment regimes in mobile health using V-learning
- IntelligentPooling: practical Thompson sampling for mHealth
- Matched Learning for Optimizing Individualized Treatment Strategies Using Electronic Health Records
- A Bayesian time-varying effect model for behavioral mHealth data
- Learning when-to-treat policies
- scientific article; zbMATH DE number 1849144
- Estimating time-varying causal excursion effects in mobile health with binary outcomes
- Policy learning with observational data
Cites work
- A robust method for estimating optimal treatment regimes
- A survey of truncated-Newton methods
- Constructing dynamic treatment regimes over indefinite time horizons
- Estimating dynamic treatment regimes in mobile health using V-learning
- Estimating individualized treatment rules using outcome weighted learning
- Interactive model building for Q-learning
- Linear mixed models with endogenous covariates: modeling sequential treatment effects with application to a mobile health study
- Model Selection and Estimation in Regression with Grouped Variables
- New statistical learning methods for estimating optimal dynamic treatment regimes
- Optimal Dynamic Treatment Regimes
- Penalized Q-learning for dynamic treatment regimens
- Performance guarantees for individualized treatment rules
- Personalize treatment for longitudinal data using unspecified random-effects model
- Sequential multiple assignment randomized trial (SMART) with adaptive randomization for quality improvement in depression treatment program
- The Conjugate Gradient Method and Trust Regions in Large Scale Optimization
Cited in
(10)- Testing stationarity and change point detection in reinforcement learning
- A multiagent reinforcement learning framework for off-policy evaluation in two-sided markets
- Dynamic Causal Effects Evaluation in A/B Testing with a Reinforcement Learning Framework
- Off-policy estimation of long-term average outcomes with applications to mobile health
- Statistically Efficient Advantage Learning for Offline Reinforcement Learning in Infinite Horizons
- Iterated multisource exchangeability models for individualized inference with an application to mobile sensor data
- Projected state-action balancing weights for offline reinforcement learning
- glmer
- Value Enhancement of Reinforcement Learning via Efficient and Robust Trust Region Optimization
- A large-scale constrained joint modeling approach for predicting user activity, engagement, and churn with application to freemium mobile games
This page was built for publication: Personalized Policy Learning Using Longitudinal Mobile Health Data
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5857154)
