Doubly penalized estimation in additive regression with high-dimensional data
From MaRDI portal
(Redirected from Publication:2328052)
total variationhigh-dimensional datareproducing kernel Hilbert spacepenalized estimationadditive modeltrend filteringSobolev spacemetric entropyANOVA modelbounded variation space
Nonparametric regression and quantile regression (62G08) Linear regression; mixed models (62J05) Asymptotic distribution theory in statistics (62E20) Parametric tolerance and confidence regions (62F25) Analysis of variance and covariance (ANOVA) (62J10) Generalized linear models (logistic models) (62J12) Robustness and adaptive procedures (parametric inference) (62F35) Hilbert spaces with reproducing kernels (= (proper) functional Hilbert spaces, including de Branges-Rovnyak and other structured spaces) (46E22)
Abstract: Additive regression provides an extension of linear regression by modeling the signal of a response as a sum of functions of covariates of relatively low complexity. We study penalized estimation in high-dimensional nonparametric additive regression where functional semi-norms are used to induce smoothness of component functions and the empirical norm is used to induce sparsity. The functional semi-norms can be of Sobolev or bounded variation types and are allowed to be different amongst individual component functions. We establish new oracle inequalities for the predictive performance of such methods under three simple technical conditions: a sub-gaussian condition on the noise, a compatibility condition on the design and the functional classes under consideration, and an entropy condition on the functional classes. For random designs, the sample compatibility condition can be replaced by its population version under an additional condition to ensure suitable convergence of empirical norms. In homogeneous settings where the complexities of the component functions are of the same order, our results provide a spectrum of explicit convergence rates, from the so-called slow rate without requiring the compatibility condition to the fast rate under the hard sparsity or certain sparsity to allow many small components in the true regression function. These results significantly broadens and sharpens existing ones in the literature.
Recommendations
Cites work
- scientific article; zbMATH DE number 5654889 (Why is no real title available?)
- scientific article; zbMATH DE number 49190 (Why is no real title available?)
- scientific article; zbMATH DE number 47282 (Why is no real title available?)
- scientific article; zbMATH DE number 477682 (Why is no real title available?)
- scientific article; zbMATH DE number 713342 (Why is no real title available?)
- scientific article; zbMATH DE number 3261770 (Why is no real title available?)
- $\ell_1$ Trend Filtering
- Adaptive piecewise polynomial estimation via trend filtering
- Additive models with trend filtering
- Additive regression and other nonparametric models
- Component selection and smoothing in multivariate nonparametric regression
- Doubly penalized estimation in additive regression with high-dimensional data
- Fast learning rate of multiple kernel learning: trade-off between sparsity and smoothness
- High-dimensional additive modeling
- Locally adaptive regression splines
- Minimax optimal rates of estimation in high dimensional additive models
- Minimax-optimal nonparametric regression in high dimensions
- Minimax-optimal rates for sparse additive models over kernel classes via convex programming
- Nonparametric regression under qualitative smoothness assumptions
- Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion
- On the conditions used to prove oracle results for the Lasso
- Optimal global rates of convergence for nonparametric regression
- Persistene in high-dimensional linear predictor-selection and the virtue of overparametrization
- Simultaneous analysis of Lasso and Dantzig selector
- Smoothing spline ANOVA models
- Sparse additive models
- Sparsity in multiple kernel learning
- Sparsity oracle inequalities for the Lasso
- Statistical inference in compound functional models
- The partial linear model in high dimensions
- Variable selection in nonparametric additive models
- Weak convergence and empirical processes. With applications to statistics
Cited in
(17)- Linear and nonlinear signal detection and estimation in high-dimensional nonparametric regression under weak sparsity
- Block-wise primal-dual algorithms for large-scale doubly penalized ANOVA modeling
- Large Scale Prediction with Decision Trees
- Sparse additive regression on a regular lattice
- Grouped variable selection with discrete optimization: computational and statistical perspectives
- Sharp lower bound for regression with measurement errors and its implication for ill-posedness of functional regression
- Extreme eigenvalues of nonlinear correlation matrices with applications to additive models
- PAC-Bayesian estimation and prediction in sparse additive models
- Doubly penalized estimation in additive regression with high-dimensional data
- Asymptotically faster estimation of high-dimensional additive models using subspace learning
- High-dimensional additive modeling
- Improved Estimation of High-dimensional Additive Models Using Subspace Learning
- Minimax-optimal rates for sparse additive models over kernel classes via convex programming
- Sparse additive support vector machines in bounded variation space
- Hierarchical Total Variations and Doubly Penalized ANOVA Modeling for Multivariate Nonparametric Regression
- Minimax optimal rates of estimation in high dimensional additive models
- Penalized least squares estimation in the additive model with different smoothness for the components
This page was built for publication: Doubly penalized estimation in additive regression with high-dimensional data
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2328052)
