Regression in Tensor Product Spaces by the Method of Sieves

DOI10.48550/ARXIV.2206.02994arXiv2206.02994OpenAlexW4389458143MaRDI QIDQ104871FDOQ104871

Authors: Tianyu Zhang, Noah Simon, Tianyu Zhang, Noah Simon 

Publication date: 7 June 2022

Published in: Electronic Journal of Statistics (Search for Journal in Brave)

Abstract: Estimation of a conditional mean (linking a set of features to an outcome of interest) is a fundamental statistical task. While there is an appeal to flexible nonparametric procedures, effective estimation in many classical nonparametric function spaces (e.g., multivariate Sobolev spaces) can be prohibitively difficult -- both statistically and computationally -- especially when the number of features is large. In this paper, we present (penalized) sieve estimators for regression in nonparametric tensor product spaces: These spaces are more amenable to multivariate regression, and allow us to, in-part, avoid the curse of dimensionality. Our estimators can be easily applied to multivariate nonparametric problems and have appealing statistical and computational properties. Moreover, they can effectively leverage additional structures such as feature sparsity. In this manuscript, we give theoretical guarantees, indicating that the predictive performance of our estimators scale favorably in dimension. In addition, we also present numerical examples to compare the finite-sample performance of the proposed estimators with several popular machine learning methods.

Full work available at URL: https://arxiv.org/abs/2206.02994

zbMATH Keywords

multivariate regressionorthonormal basissparse nonparametric models

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Nonparametric regression and quantile regression (62G08)

Cites Work

• Weak convergence and empirical processes. With applications to statistics
• Title not available (Why is that?)
• Multivariate adaptive regression splines
• Smoothing spline ANOVA for exponential families, with application to the Wisconsin epidemiological study of diabetic retinopathy. (The 1994 Neyman Memorial Lecture)
• Consistency of cross validation for comparing regression procedures
• Estimation and testing under sparsity. École d'Été de Probabilités de Saint-Flour XLV -- 2015
• Support Vector Machines
• Title not available (Why is that?)
• High-Dimensional Statistics
• High-Dimensional Probability
• Title not available (Why is that?)
• All of Nonparametric Statistics
• Introduction to nonparametric estimation
• Tensor products of Sobolev-Besov spaces and applications to approximation from the hyperbolic cross
• Number of lattice points in the hyperbolic cross
• Nonparametric curve estimation. Methods, theory, and applications
• Optimal global rates of convergence for nonparametric regression
• Tensor product space ANOVA models.
• Smoothing spline ANOVA models for large data sets with Bernoulli observations and the randomized GACV.
• A local maximal inequality under uniform entropy
• Hyperbolic cross approximation. Lecture notes given at the courses on constructive approximation and harmonic analysis, Barcelona, Spain, May 30 -- June 3, 2016
• A sieve stochastic gradient descent estimator for online nonparametric regression in Sobolev ellipsoids
• New reproducing kernel functions in the reproducing kernel Sobolev spaces
• Convergence rates of least squares regression estimators with heavy-tailed errors
• Minimax-optimal nonparametric regression in high dimensions
• On the mathematical foundations of learning
• Recent developments in the theory of function spaces with dominating mixed smoothness.
• Kernel-based Approximation Methods using MATLAB
• Spline interpolation on sparse grids
• From high oscillation to rapid approximation. IV: Accelerating convergence
• Curve fitting by polynomial-trigonometric regression
• Title not available (Why is that?)
• Title not available (Why is that?)
• Mercer's theorem on general domains: on the interaction between measures, kernels, and RKHSs
• Multivariate Approximation
• Isotropic and dominating mixed Besov spaces: A comparison
• Title not available (Why is that?)
• Smoothing Spline ANOVA for Multivariate Bernoulli Observations With Application to Ophthalmology Data
• Title not available (Why is that?)
• On the entropy numbers of the mixed smoothness function classes
• Minimax Rates of Estimation for High-Dimensional Linear Regression Over $\ell_q$-Balls
• Sparse grids
• Probability in Banach spaces. Isoperimetry and processes
• Minimax-optimal rates for sparse additive models over kernel classes via convex programming
• Function spaces with dominating mixed smoothness
• Title not available (Why is that?)
• An Online Projection Estimator for Nonparametric Regression in Reproducing Kernel Hilbert Spaces
• Optimal estimation in additive regression models
• Approximation of mixed order Sobolev functions on the \(d\)-torus: asymptotics, preasymptotics, and \(d\)-dependence

Cited In (1)

• Sieve

This page was built for publication: Regression in Tensor Product Spaces by the Method of Sieves

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q104871)