Doubly penalized estimation in additive regression with high-dimensional data

DOI10.1214/18-AOS1757zbMATH Open1436.62358arXiv1704.07229OpenAlexW2966000184MaRDI QIDQ2328052FDOQ2328052

Authors: Yanyan Li 

Publication date: 9 October 2019

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: Additive regression provides an extension of linear regression by modeling the signal of a response as a sum of functions of covariates of relatively low complexity. We study penalized estimation in high-dimensional nonparametric additive regression where functional semi-norms are used to induce smoothness of component functions and the empirical $L_2$ norm is used to induce sparsity. The functional semi-norms can be of Sobolev or bounded variation types and are allowed to be different amongst individual component functions. We establish new oracle inequalities for the predictive performance of such methods under three simple technical conditions: a sub-gaussian condition on the noise, a compatibility condition on the design and the functional classes under consideration, and an entropy condition on the functional classes. For random designs, the sample compatibility condition can be replaced by its population version under an additional condition to ensure suitable convergence of empirical norms. In homogeneous settings where the complexities of the component functions are of the same order, our results provide a spectrum of explicit convergence rates, from the so-called slow rate without requiring the compatibility condition to the fast rate under the hard sparsity or certain $L_q$ sparsity to allow many small components in the true regression function. These results significantly broadens and sharpens existing ones in the literature.

Full work available at URL: https://arxiv.org/abs/1704.07229

Recommendations

• Statistical inference in sparse high-dimensional additive models
• High-dimensional additive modeling
• Minimax optimal rates of estimation in high dimensional additive models
• Sparse Additive Models
• Sparsity oracle inequalities for the Lasso

zbMATH Keywords

total variationhigh-dimensional datareproducing kernel Hilbert spacepenalized estimationadditive modeltrend filteringSobolev spacemetric entropyANOVA modelbounded variation space

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Linear regression; mixed models (62J05) Asymptotic distribution theory in statistics (62E20) Parametric tolerance and confidence regions (62F25) Analysis of variance and covariance (ANOVA) (62J10) Generalized linear models (logistic models) (62J12) Robustness and adaptive procedures (parametric inference) (62F35) Hilbert spaces with reproducing kernels (= (proper) functional Hilbert spaces, including de Branges-Rovnyak and other structured spaces) (46E22)

Cites Work

• Smoothing spline ANOVA models
• High-dimensional additive modeling
• Weak convergence and empirical processes. With applications to statistics
• Title not available (Why is that?)
• On the conditions used to prove oracle results for the Lasso
• Simultaneous analysis of Lasso and Dantzig selector
• Adaptive piecewise polynomial estimation via trend filtering
• Additive regression and other nonparametric models
• Locally adaptive regression splines
• Persistene in high-dimensional linear predictor-selection and the virtue of overparametrization
• Sparsity oracle inequalities for the Lasso
• Title not available (Why is that?)
• Optimal global rates of convergence for nonparametric regression
• Minimax-optimal nonparametric regression in high dimensions
• Title not available (Why is that?)
• Minimax-optimal rates for sparse additive models over kernel classes via convex programming
• Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion
• Sparse Additive Models
• Minimax optimal rates of estimation in high dimensional additive models
• Sparsity in multiple kernel learning
• Component selection and smoothing in multivariate nonparametric regression
• Variable selection in nonparametric additive models
• Additive models with trend filtering
• $\ell_1$ Trend Filtering
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Nonparametric regression under qualitative smoothness assumptions
• Fast learning rate of multiple kernel learning: trade-off between sparsity and smoothness
• The partial linear model in high dimensions
• Statistical inference in compound functional models
• Doubly penalized estimation in additive regression with high-dimensional data

Cited In (17)

• Linear and nonlinear signal detection and estimation in high-dimensional nonparametric regression under weak sparsity
• Block-wise primal-dual algorithms for large-scale doubly penalized ANOVA modeling
• Large Scale Prediction with Decision Trees
• Sparse additive regression on a regular lattice
• Grouped variable selection with discrete optimization: computational and statistical perspectives
• Sharp lower bound for regression with measurement errors and its implication for ill-posedness of functional regression
• Extreme eigenvalues of nonlinear correlation matrices with applications to additive models
• PAC-Bayesian estimation and prediction in sparse additive models
• Doubly penalized estimation in additive regression with high-dimensional data
• Asymptotically faster estimation of high-dimensional additive models using subspace learning
• High-dimensional additive modeling
• Improved Estimation of High-dimensional Additive Models Using Subspace Learning
• Minimax-optimal rates for sparse additive models over kernel classes via convex programming
• Sparse additive support vector machines in bounded variation space
• Hierarchical Total Variations and Doubly Penalized ANOVA Modeling for Multivariate Nonparametric Regression
• Penalized least squares estimation in the additive model with different smoothness for the components
• Minimax optimal rates of estimation in high dimensional additive models

Uses Software

• gss

This page was built for publication: Doubly penalized estimation in additive regression with high-dimensional data

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2328052)