Orthogonal statistical learning

DOI10.1214/23-AOS2258arXiv1901.09036MaRDI QIDQ6136574FDOQ6136574

Authors: Dylan J. Foster, Vasilis Syrgkanis 

Publication date: 31 August 2023

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: We provide non-asymptotic excess risk guarantees for statistical learning in a setting where the population risk with respect to which we evaluate the target parameter depends on an unknown nuisance parameter that must be estimated from data. We analyze a two-stage sample splitting meta-algorithm that takes as input arbitrary estimation algorithms for the target parameter and nuisance parameter. We show that if the population risk satisfies a condition called Neyman orthogonality, the impact of the nuisance estimation error on the excess risk bound achieved by the meta-algorithm is of second order. Our theorem is agnostic to the particular algorithms used for the target and nuisance and only makes an assumption on their individual performance. This enables the use of a plethora of existing results from machine learning to give new guarantees for learning with a nuisance component. Moreover, by focusing on excess risk rather than parameter estimation, we can provide rates under weaker assumptions than in previous works and accommodate settings in which the target parameter belongs to a complex nonparametric class. We provide conditions on the metric entropy of the nuisance and target classes such that oracle rates of the same order as if we knew the nuisance parameter are achieved.

Full work available at URL: https://arxiv.org/abs/1901.09036

zbMATH Keywords

statistical learningdouble machine learningtreatment effectspolicy learninglocal Rademacher complexityNeyman orthogonality

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Minimax procedures in statistical decision theory (62C20) Causal inference from observational studies (62D20)

Cites Work

• Generalized random forests
• Introduction to empirical processes and semiparametric inference
• Adjusting for Nonignorable Drop-Out Using Semiparametric Nonresponse Models
• Title not available (Why is that?)
• Double/debiased machine learning for treatment and structural parameters
• Unified methods for censored longitudinal data and causality
• Semiparametric theory and missing data.
• Estimation of possibly misspecified semiparametric conditional moment restriction models with different conditioning variables
• Higher order influence functions and minimax estimation of nonlinear functionals
• The central role of the propensity score in observational studies for causal effects
• Root-N-Consistent Semiparametric Regression
• Title not available (Why is that?)
• Estimation of Regression Coefficients When Some Regressors Are Not Always Observed
• The Asymptotic Variance of Semiparametric Estimators
• Asymptotic Statistics
• Program evaluation and causal inference with high-dimensional data
• High-dimensional statistics. A non-asymptotic viewpoint
• Concentration inequalities. A nonasymptotic theory of independence
• Efficient Estimation of Models with Conditional Moment Restrictions Containing Unknown Functions
• Contributions to a general asymptotic statistical theory. With the assistance of W. Wefelmeyer
• Optimal sup-norm rates and uniform inference on nonlinear functionals of nonparametric IV regression
• The semiparametric efficiency bound for models of sequential moment restrictions containing unknown functions
• Instrumental Variable Estimation of Nonparametric Models
• On adaptive estimation
• Estimating individualized treatment rules using outcome weighted learning
• Improved rates and asymptotic normality for nonparametric neural network estimators
• Super Learner
• Performance guarantees for individualized treatment rules
• Local Rademacher complexities
• Title not available (Why is that?)
• Some results on generalized difference estimation and generalized regression estimation for finite populations
• Semiparametric Efficiency in Multivariate Regression Models with Missing Data
• Learning without concentration
• Advanced Lectures on Machine Learning
• Semi-Nonparametric IV Estimation of Shape-Invariant Engel Curves
• Nonparametric methods for inference in the presence of instrumental variables
• Information-theoretic determination of minimax rates of convergence
• The cross-validated adaptive epsilon-net estimator
• Title not available (Why is that?)
• Estimation of nonparametric conditional moment models with possibly nonsmooth generalized residuals
• Statistical causal inferences and their applications in public health research
• Efficient estimation of semiparametric conditional moment models with possibly nonsmooth residuals
• Regularization in kernel learning
• Empirical entropy, minimax regret and minimax risk
• Nonparametric regression with missing outcomes using weighted kernel estimating equations
• Discrepancy, chaining and subgaussian processes
• Title not available (Why is that?)
• Consistent estimation of the influence function of locally asymptotically linear estimators
• On the Efficiency of a Class of Non-Parametric Estimates
• Deep neural networks for estimation and inference
• Sieve Wald and QLR inferences on semi/nonparametric conditional moment models
• Policy learning with observational data
• Non-parametric Methods for Doubly Robust Estimation of Continuous Treatment Effects
• A Doubly Robust Censoring Unbiased Transformation
• Robust estimation of causal effects via a high-dimensional covariate balancing propensity score
• Title not available (Why is that?)
• Debiased machine learning of conditional average treatment effects and other causal functions
• Augmented minimax linear estimation
• Quasi-oracle estimation of heterogeneous treatment effects
• Robust Causal Inference with Continuous Instruments Using the Local Instrumental Variable Curve
• Orthogonal statistical learning
• Local Linear Forests
• Offline Multi-Action Policy Learning: Generalization and Optimization

Cited In (8)

• Orthogonal statistical learning
• Nonparametric Causal Effects Based on Longitudinal Modified Treatment Policies
• Orthogonalized Fisher discriminant
• A nonparametric doubly robust test for a continuous treatment effect
• Efficient and multiply robust risk estimation under general forms of dataset shift
• One-step estimation of differentiable Hilbert-valued parameters
• Towards optimal doubly robust estimation of heterogeneous causal effects
• Regularised orthogonal machine learning for nonlinear semiparametric models

This page was built for publication: Orthogonal statistical learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6136574)