Statistical inference in sparse high-dimensional additive models

DOI10.1214/20-AOS2011MaRDI QIDQ820814zbMATH OpenOpenAlexFDO

Authors Enno Mammen, Martin Wahl, Karl B. Gregory

Publication date 28 September 2021

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1603.07632

zbMATH Keywords

additive modelsbias correctionLassonear-orthogonalitynonparametric curve estimation

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Asymptotic properties of nonparametric inference (62G20) Ridge regression; shrinkage estimators (Lasso) (62J07)

Abstract: In this paper we discuss the estimation of a nonparametric component $f_1$ of a nonparametric additive model $Y=f_1(X_1)+\mathrm{...}+f_q(X_q)+epsilon$. We allow the number $q$ of additive components to grow to infinity and we make sparsity assumptions about the number of nonzero additive components. We compare this estimation problem with that of estimating $f_1$ in the oracle model $Z=f_1(X_1)+epsilon$, for which the additive components $f_2,dots,f_q$ are known. We construct a two-step presmoothing-and-resmoothing estimator of $f_1$ and state finite-sample bounds for the difference between our estimator and some smoothing estimators $hat{f}_1^{ext(oracle)}$ in the oracle model. In an asymptotic setting these bounds can be used to show asymptotic equivalence of our estimator and the oracle estimators; the paper thus shows that, asymptotically, under strong enough sparsity conditions, knowledge of $f_2,dots,f_q$ has no effect on estimation accuracy. Our first step is to estimate $f_1$ with an undersmoothed estimator based on near-orthogonal projections with a group Lasso bias correction. We then construct pseudo responses $hatY$ by evaluating a debiased modification of our undersmoothed estimator of $f_1$ at the design points. In the second step the smoothing method of the oracle estimator $hat{f}_1^{ext(oracle)}$ is applied to a nonparametric regression problem with responses $hatY$ and covariates $X_1$. Our mathematical exposition centers primarily on establishing properties of the presmoothing estimator. We present simulation results demonstrating close-to-oracle performance of our estimator in practical applications.

Recommendations

• scientific article; zbMATH DE number 2232736
• Sparse additive models
• High-dimensional Bayesian inference in nonparametric additive models
• Algorithms for learning sparse additive models with interactions in high dimensions
• A sparse additive model for high-dimensional interactions with an exposure variable
• High-dimensional additive modeling
• Nonparametric independence screening in sparse ultra-high-dimensional additive models
• Sparse model identification and learning for ultra-high-dimensional additive partially linear models
• High-Dimensional Sparse Additive Hazards Regression
• Marginal empirical likelihood independence screening in sparse ultra-high dimensional additive models

Cites work

• \(\chi^{2}\)-confidence sets in high-dimensional regression
• A general theory of hypothesis tests and confidence regions for sparse high dimensional models
• Additive regression and other nonparametric models
• Asymptotic Confidence Regions for High-Dimensional Structured Sparsity
• Component selection and smoothing in multivariate nonparametric regression
• Concentration inequalities and model selection. Ecole d'Eté de Probabilités de Saint-Flour XXXIII -- 2003.
• Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
• Confidence intervals for low dimensional parameters in high dimensional linear models
• Double/debiased machine learning for treatment and structural parameters
• Fast learning rate of multiple kernel learning: trade-off between sparsity and smoothness
• High-dimensional additive modeling
• scientific article; zbMATH DE number 1064642 (Why is no real title available?)
• Inference on treatment effects after selection among high-dimensional controls
• Inverses of Band Matrices and Local Convergence of Spline Projections
• Least squares after model selection in high-dimensional sparse models
• Local asymptotics for regression splines and confidence regions
• Minimax-optimal rates for sparse additive models over kernel classes via convex programming
• Model selection for regression on a random design
• Nonparametric independence screening in sparse ultra-high-dimensional additive models
• Nonparametric regression with the scale depending on auxiliary variable
• On asymptotically optimal confidence regions and tests for high-dimensional models
• On deep learning as a remedy for the curse of dimensionality in nonparametric regression
• On pointwise adaptive curve estimation based on inhomogeneous data
• Optimal estimation in additive regression models
• Optimal pointwise adaptive methods in nonparametric estimation
• Optimal spatial adaptation to inhomogeneous smoothness: An approach based on kernel estimates with variable bandwidth selectors
• Parsimonious additive models
• Rate optimal estimation with the integration method in the presence of many covariates
• Rate-optimal estimation for a general class of nonparametric regression models with unknown link functions
• Selection of variables and dimension reduction in high-dimensional non-parametric regression
• Simultaneous analysis of Lasso and Dantzig selector
• Sparse additive models
• Sparse estimation in functional linear regression
• Sparse varying coefficient models for longitudinal data
• The benefit of group sparsity in group inference with de-biased scaled group Lasso
• Uniformly valid post-regularization confidence regions for many functional parameters in z-estimation framework
• Variable selection in nonparametric additive models

Cited in

(20)

• High-dimensional additive modeling
• New inference concepts for analysing complex data
• PAC-Bayesian estimation and prediction in sparse additive models
• Doubly penalized estimation in additive regression with high-dimensional data
• Statistical inference in compound functional models
• EDITORIAL: RECENT ADVANCES IN SPARSE STATISTICAL MODELING
• Oracle-efficient nonparametric estimation of an additive model with an unknown link function
• Sparse additive models
• High-Dimensional Sparse Additive Hazards Regression
• Penalized estimation of high-dimensional models under a generalized sparsity condition
• Minimax optimal rates of estimation in high dimensional additive models
• Algorithms for learning sparse additive models with interactions in high dimensions
• Sparse additive regression on a regular lattice
• Sharp lower bound for regression with measurement errors and its implication for ill-posedness of functional regression
• Efficient functional Lasso kernel smoothing for high-dimensional additive regression
• Sparse additive support vector machines in bounded variation space
• Smooth backfitting for additive hazard rates
• Estimation and uniform inference in sparse high-dimensional additive models
• Functional Lasso kernel smoothing for additive regression with interaction effects
• Estimation in additive models with highly or non-highly correlated covariates

This page was built for publication: Statistical inference in sparse high-dimensional additive models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q820814)