Orthogonal statistical learning

DOI10.1214/23-AOS2258MaRDI QIDQ6136574FDOQ6136574

Authors Dylan J. Foster, Vasilis Syrgkanis 

Publication date 31 August 2023

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1901.09036

zbMATH Keywords

statistical learningdouble machine learningtreatment effectspolicy learninglocal Rademacher complexityNeyman orthogonality

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Minimax procedures in statistical decision theory (62C20) Causal inference from observational studies (62D20)

Abstract: We provide non-asymptotic excess risk guarantees for statistical learning in a setting where the population risk with respect to which we evaluate the target parameter depends on an unknown nuisance parameter that must be estimated from data. We analyze a two-stage sample splitting meta-algorithm that takes as input arbitrary estimation algorithms for the target parameter and nuisance parameter. We show that if the population risk satisfies a condition called Neyman orthogonality, the impact of the nuisance estimation error on the excess risk bound achieved by the meta-algorithm is of second order. Our theorem is agnostic to the particular algorithms used for the target and nuisance and only makes an assumption on their individual performance. This enables the use of a plethora of existing results from machine learning to give new guarantees for learning with a nuisance component. Moreover, by focusing on excess risk rather than parameter estimation, we can provide rates under weaker assumptions than in previous works and accommodate settings in which the target parameter belongs to a complex nonparametric class. We provide conditions on the metric entropy of the nuisance and target classes such that oracle rates of the same order as if we knew the nuisance parameter are achieved.

Recommendations

• Risk bounds for statistical learning
• A simple and general debiased machine learning theorem with finite-sample guarantees
• Nonexact oracle inequalities, \(r\)-learnability, and fast rates
• Fast learning rates in statistical inference through aggregation
• Inverse statistical learning

Cites work

• scientific article; zbMATH DE number 3169866 (Why is no real title available?)
• scientific article; zbMATH DE number 3738700 (Why is no real title available?)
• scientific article; zbMATH DE number 490141 (Why is no real title available?)
• scientific article; zbMATH DE number 1552503 (Why is no real title available?)
• scientific article; zbMATH DE number 823069 (Why is no real title available?)
• scientific article; zbMATH DE number 6542809 (Why is no real title available?)
• A Doubly Robust Censoring Unbiased Transformation
• Adjusting for Nonignorable Drop-Out Using Semiparametric Nonresponse Models
• Advanced Lectures on Machine Learning
• Asymptotic Statistics
• Augmented minimax linear estimation
• Concentration inequalities. A nonasymptotic theory of independence
• Consistent estimation of the influence function of locally asymptotically linear estimators
• Contributions to a general asymptotic statistical theory. With the assistance of W. Wefelmeyer
• Debiased machine learning of conditional average treatment effects and other causal functions
• Deep neural networks for estimation and inference
• Discrepancy, chaining and subgaussian processes
• Double/debiased machine learning for treatment and structural parameters
• Efficient Estimation of Models with Conditional Moment Restrictions Containing Unknown Functions
• Efficient estimation of semiparametric conditional moment models with possibly nonsmooth residuals
• Empirical entropy, minimax regret and minimax risk
• Estimating individualized treatment rules using outcome weighted learning
• Estimation of Regression Coefficients When Some Regressors Are Not Always Observed
• Estimation of nonparametric conditional moment models with possibly nonsmooth generalized residuals
• Estimation of possibly misspecified semiparametric conditional moment restriction models with different conditioning variables
• Generalized random forests
• High-dimensional statistics. A non-asymptotic viewpoint
• Higher order influence functions and minimax estimation of nonlinear functionals
• Improved rates and asymptotic normality for nonparametric neural network estimators
• Information-theoretic determination of minimax rates of convergence
• Instrumental Variable Estimation of Nonparametric Models
• Introduction to empirical processes and semiparametric inference
• Learning without concentration
• Local Linear Forests
• Local Rademacher complexities
• Non-parametric Methods for Doubly Robust Estimation of Continuous Treatment Effects
• Nonparametric methods for inference in the presence of instrumental variables
• Nonparametric regression with missing outcomes using weighted kernel estimating equations
• Offline Multi-Action Policy Learning: Generalization and Optimization
• On adaptive estimation
• On the Efficiency of a Class of Non-Parametric Estimates
• Optimal sup-norm rates and uniform inference on nonlinear functionals of nonparametric IV regression
• Orthogonal statistical learning
• Performance guarantees for individualized treatment rules
• Policy learning with observational data
• Program evaluation and causal inference with high-dimensional data
• Quasi-oracle estimation of heterogeneous treatment effects
• Regularization in kernel learning
• Robust Causal Inference with Continuous Instruments Using the Local Instrumental Variable Curve
• Robust estimation of causal effects via a high-dimensional covariate balancing propensity score
• Root-N-Consistent Semiparametric Regression
• Semi-Nonparametric IV Estimation of Shape-Invariant Engel Curves
• Semiparametric Efficiency in Multivariate Regression Models with Missing Data
• Semiparametric theory and missing data.
• Sieve Wald and QLR inferences on semi/nonparametric conditional moment models
• Some results on generalized difference estimation and generalized regression estimation for finite populations
• Statistical causal inferences and their applications in public health research
• Super Learner
• The Asymptotic Variance of Semiparametric Estimators
• The central role of the propensity score in observational studies for causal effects
• The cross-validated adaptive epsilon-net estimator
• The semiparametric efficiency bound for models of sequential moment restrictions containing unknown functions
• Unified methods for censored longitudinal data and causality

Cited in

(8)

• Efficient and multiply robust risk estimation under general forms of dataset shift
• One-step estimation of differentiable Hilbert-valued parameters
• Towards optimal doubly robust estimation of heterogeneous causal effects
• Regularised orthogonal machine learning for nonlinear semiparametric models
• Orthogonal statistical learning
• Orthogonalized Fisher discriminant
• Nonparametric Causal Effects Based on Longitudinal Modified Treatment Policies
• A nonparametric doubly robust test for a continuous treatment effect

This page was built for publication: Orthogonal statistical learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6136574)