Universal sieve-based strategies for efficient estimation using machine learning tools

DOI10.3150/20-BEJ1309MaRDI QIDQ1983607zbMATH OpenFDO

Authors Hongxiang Qiu, Alex Luedtke, Marco Carone

Publication date 10 September 2021

Published in Bernoulli (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2003.01856

zbMATH Keywords

asymptotic efficiencynonparametric inferencesieve estimation

Mathematics Subject Classification ID

Applications of sieve methods (11N36) Nonparametric estimation (62G05) Ridge regression; shrinkage estimators (Lasso) (62J07) Learning and adaptive systems in artificial intelligence (68T05)

Abstract: Suppose that we wish to estimate a finite-dimensional summary of one or more function-valued features of an underlying data-generating mechanism under a nonparametric model. One approach to estimation is by plugging in flexible estimates of these features. Unfortunately, in general, such estimators may not be asymptotically efficient, which often makes these estimators difficult to use as a basis for inference. Though there are several existing methods to construct asymptotically efficient plug-in estimators, each such method either can only be derived using knowledge of efficiency theory or is only valid under stringent smoothness assumptions. Among existing methods, sieve estimators stand out as particularly convenient because efficiency theory is not required in their construction, their tuning parameters can be selected data adaptively, and they are universal in the sense that the same fits lead to efficient plug-in estimators for a rich class of estimands. Inspired by these desirable properties, we propose two novel universal approaches for estimating function-valued features that can be analyzed using sieve estimation theory. Compared to traditional sieve estimators, these approaches are valid under more general conditions on the smoothness of the function-valued features by utilizing flexible estimates that can be obtained, for example, using machine learning.

Review text

Regression models attempt to explain the behavior of a variable of interest (\(Y\)) from explanatory variables (\(X\)). After a random sample, one can write the regression equation as \(y_i = g(x_i) + e_i\), with \(E(e_i|x_i) = 0\), \(i = 1, \ldots, n\). The regression equation is nonparametric when \(g(x)\) cannot be summarized by a finite set of parameters. In this context, sieve regression estimators can be used to estimate \(g(x)\). Examples of sieve regression are polynomials and splines. Unfortunately, some properties, like asymptotic efficiency, cannot be achieved using sieve regression estimators. In their work, Qiu, Luedtke and Carone proposed two approaches: estimating the unknown function with Highly Adaptive Lasso and using data-adaptive series based on an initial maximum likelihood fit, which can overcome some difficulties related to sieve theory without performance loss against existing methods.

Recommendations

• On methods of sieves and penalization
• Convergence rate of sieve estimates
• Toward Computerized Efficient Estimation in Infinite-Dimensional Models
• Nonparametric two-step sieve M estimation and inference
• Efficient Estimation of Models with Conditional Moment Restrictions Containing Unknown Functions

Cites work

• A wavelet tour of signal processing. The sparse way.
• Asymptotic inference of causal effects with observational studies trimmed by the estimated propensity scores
• Contributions to a general asymptotic statistical theory. With the assistance of W. Wefelmeyer
• Convergence rates and asymptotic normality for series estimators
• Double/debiased machine learning for treatment and structural parameters
• Greedy function approximation: A gradient boosting machine.
• scientific article; zbMATH DE number 1057566 (Why is no real title available?)
• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• Inefficient estimators of the bivariate survival function for three models
• Investigating Smooth Multiple Regression by the Method of Average Derivatives
• Multidimensional variation for quasi-Monte Carlo
• Nearly unbiased variable selection under minimax concave penalty
• Nonparametric estimators which can be ``plugged-in.
• Nonparametric variable importance assessment using machine learning techniques
• On methods of sieves and penalization
• Stochastic gradient boosting.
• Targeted learning in data science. Causal inference for complex longitudinal studies
• The bootstrap and Edgeworth expansion
• Twicing Kernels and a Small Bias Property of Semiparametric Estimators
• Unified methods for censored longitudinal data and causality
• Weak convergence and empirical processes. With applications to statistics

Cited in

(2)

• Exploring the impact of post-training rounding in regression models.
• Sequential estimation of generalized linear model coefficients

This page was built for publication: Universal sieve-based strategies for efficient estimation using machine learning tools

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1983607)