Estimation of a regression function on a manifold by fully connected deep neural networks

DOI10.1016/J.JSPI.2022.05.008MaRDI QIDQ2676904zbMATH OpenOpenAlexWikidataFDO

Authors Michael Kohler, Sophie Langer, U. Reif

Publication date 28 September 2022

Published in Journal of Statistical Planning and Inference (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2107.09532

zbMATH Keywords

curse of dimensionalitydeep neural networksnonparametric regressionrate of convergence

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Nonparametric regression and quantile regression (62G08) Asymptotic properties of nonparametric inference (62G20) Artificial neural networks and deep learning (68T07)

Abstract: Estimation of a regression function from independent and identically distributed data is considered. The $L_2$ error with integration with respect to the distribution of the predictor variable is used as the error criterion. The rate of convergence of least squares estimates based on fully connected spaces of deep neural networks with ReLU activation function is analyzed for smooth regression functions. It is shown that in case that the distribution of the predictor variable is concentrated on a manifold, these estimates achieve a rate of convergence which depends on the dimension of the manifold and not on the number of components of the predictor variable.

Recommendations

• Nonparametric regression on low-dimensional manifolds using deep ReLU networks: function approximation and statistical recovery
• On the rate of convergence of fully connected deep neural network regression estimates
• Estimation of a Function of Low Local Dimensionality by Deep Neural Networks
• Deep nonparametric regression on approximate manifolds: nonasymptotic error bounds with polynomial prefactors
• Robust deep neural network estimation for multi-dimensional functional data
• Regression on manifolds using data-dependent regularization with applications in computer vision
• Multiscale regression on unknown manifolds
• Analysis of the rate of convergence of fully connected deep neural network regression estimates with smooth activation function
• Regression on manifolds: estimation of the exterior derivative
• Regression on parametric manifolds: Estimation of spatial fields, functional outputs, and parameters from noisy data

Cites work

• A distribution-free theory of nonparametric regression
• Adaptive approximation and generalization of deep neural network with intrinsic dimensionality
• Ambient approximation on hypersurfaces
• Analysis of the rate of convergence of fully connected deep neural network regression estimates with smooth activation function
• Approximating smooth functions by deep neural networks with sigmoid activation function
• Approximation and estimation bounds for artificial neural networks
• Approximation by superpositions of a sigmoidal function
• Bayesian manifold regression
• Deep learning
• Deep Network Approximation for Smooth Functions
• Deep neural networks for estimation and inference
• Deep vs. shallow networks: an approximation theory perspective
• Diffusion maps
• Distribution-free consistency results in nonparametric discrimination and regression function estimation
• Eignets for function approximation on manifolds
• Error bounds for approximations with deep ReLU networks
• Estimation of a Regression Function by Maxima of Minima of Linear Functions
• From graph to manifold Laplacian: the convergence rate
• scientific article; zbMATH DE number 7064043 (Why is no real title available?)
• Laplacian Eigenmaps for Dimensionality Reduction and Data Representation
• Nonparametric Regression Based on Hierarchical Interaction Models
• Nonparametric regression using deep neural networks with ReLU activation function
• On deep learning as a remedy for the curse of dimensionality in nonparametric regression
• On the rate of convergence of fully connected deep neural network regression estimates
• Optimal global rates of convergence for noiseless regression estimation problems with adaptively chosen design
• Optimal global rates of convergence for nonparametric regression
• Polyharmonic and related kernels on manifolds: interpolation and approximation
• Scattered Data Fitting on Surfaces Using Projected Powell-Sabin Splines
• Theoretical issues in deep networks
• Universal approximation bounds for superpositions of a sigmoidal function
• Universal Approximation Depth and Errors of Narrow Belief Networks with Discrete Units

Cited in

(13)

• Estimation error analysis of deep learning on the regression problem on the variable exponent Besov space
• On the rate of convergence of fully connected deep neural network regression estimates
• A direct approach for function approximation on data defined manifolds
• Deep learning Gauss-Manin connections
• Nonparametric regression using deep neural networks with ReLU activation function
• Analysis of the rate of convergence of fully connected deep neural network regression estimates with smooth activation function
• On deep learning as a remedy for the curse of dimensionality in nonparametric regression
• Side effects of learning from low-dimensional data embedded in a Euclidean space
• Deep neural networks for estimation and inference
• Analysis of the rate of convergence of least squares neural network regression estimates in case of measurement errors
• Manifold Oblique Random Forests: Towards Closing the Gap on Convolutional Deep Networks
• Deep nonparametric regression on approximate manifolds: nonasymptotic error bounds with polynomial prefactors
• Empirical risk minimization in the interpolating regime with application to neural network learning

Describes a project that uses

Uses Software

• AlexNet
• ImageNet

This page was built for publication: Estimation of a regression function on a manifold by fully connected deep neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2676904)