Nonparametric regression using deep neural networks with ReLU activation function

DOI10.1214/19-AOS1875zbMath1459.62059arXiv1708.06633OpenAlexW3102511045MaRDI QIDQ2215715

Publication date: 14 December 2020

Published in: The Annals of Statistics (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1708.06633

zbMATH Keywords

wavelets additive models nonparametric regression multilayer neural networks rectified linear unit (ReLU)minimax estimation risk rectified linear activation function

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Nontrigonometric harmonic analysis involving wavelets and other special systems (42C40) Artificial neural networks and deep learning (68T07) Minimax procedures in statistical decision theory (62C20) Neural nets and related approaches to inference from stochastic processes (62M45)

Related Items (88)

Stationary Density Estimation of Itô Diffusions Using Deep Learning ⋮ Neural network approximation ⋮ Mathematical foundations of machine learning. Abstracts from the workshop held March 21--27, 2021 (hybrid meeting) ⋮ Convergence rates of deep ReLU networks for multiclass classification ⋮ Particle dual averaging: optimization of mean field neural network with global convergence rate analysis* ⋮ Deep learning for the partially linear Cox model ⋮ Discussion to: \textit{Bayesian graphical models for modern biological applications} by Y. Ni, V. Baladandayuthapani, M. Vannucci and F.C. Stingo ⋮ On the SQH method for solving optimal control problems with non-smooth state cost functionals or constraints ⋮ On the minimax optimality and superiority of deep neural network learning over sparse parameter spaces ⋮ Approximation bounds for norm constrained neural networks with applications to regression and GANs ⋮ Conditional predictive inference for stable algorithms ⋮ Estimation of a regression function on a manifold by fully connected deep neural networks ⋮ Smoothing and adaptation of shifted Pólya tree ensembles ⋮ Neural network approximation: three hidden layers are enough ⋮ A deep network construction that adapts to intrinsic dimensionality beyond the domain ⋮ Theory of deep convolutional neural networks. III: Approximating radial functions ⋮ Inference on heterogeneous treatment effects in high‐dimensional dynamic panels under weak dependence ⋮ Comprehensive study of variational Bayes classification for dense deep neural networks ⋮ The Kolmogorov-Arnold representation theorem revisited ⋮ Fast convergence rates of deep neural networks for classification ⋮ Statistical guarantees for regularized neural networks ⋮ Asymptotic properties of neural network sieve estimators ⋮ Estimation of projection pursuit regression via alternating linearization ⋮ Debiased machine learning of set-identified linear models ⋮ Consistent Sparse Deep Learning: Theory and Computation ⋮ Space-time error estimates for deep neural network approximations for differential equations ⋮ Data augmentation for Bayesian deep learning ⋮ Drift estimation for a multi-dimensional diffusion process using deep neural networks ⋮ Unnamed Item ⋮ Deep Neural Networks with ReLU-Sine-Exponential Activations Break Curse of Dimensionality in Approximation on Hölder Class ⋮ Convergence rates for shallow neural networks learned by gradient descent ⋮ Deep neural network classifier for multidimensional functional data ⋮ A multivariate Riesz basis of ReLU neural networks ⋮ Nonexact oracle inequalities, \(r\)-learnability, and fast rates ⋮ Individualized risk assessment of preoperative opioid use by interpretable neural network regression ⋮ Improved Analysis of PINNs: Alleviate the CoD for Compositional Solutions ⋮ Off-Policy Confidence Interval Estimation with Confounded Markov Decision Process ⋮ Optimal deep neural networks by maximization of the approximation power ⋮ Deep learning theory of distribution regression with CNNs ⋮ Deep nonparametric regression on approximate manifolds: nonasymptotic error bounds with polynomial prefactors ⋮ Optimal convergence rates of deep neural networks in a classification setting ⋮ Learning ability of interpolating deep convolutional neural networks ⋮ Deep learning for inverse problems with unknown operator ⋮ Adaptive variational Bayes: optimality, computation and applications ⋮ Correction to: ``Nonparametric regression using deep neural networks with ReLU activation function ⋮ Causal inference of general treatment effects using neural networks with a diverging number of confounders ⋮ Asset pricing with neural networks: significance tests ⋮ Assumption-lean falsification tests of rate double-robustness of double-machine-learning estimators ⋮ Analysis of the rate of convergence of two regression estimates defined by neural features which are easy to implement ⋮ Intrinsic and extrinsic deep learning on manifolds ⋮ Multiclass classification for multidimensional functional data through deep neural networks ⋮ Estimating a regression function in exponential families by model selection ⋮ Error assessment of an adaptive finite elements -- neural networks method for an elliptic parametric PDE ⋮ On the rate of convergence of a deep recurrent neural network estimate in a regression problem with dependent data ⋮ Unnamed Item ⋮ Unnamed Item ⋮ Unnamed Item ⋮ Analysis of the rate of convergence of fully connected deep neural network regression estimates with smooth activation function ⋮ Approximating smooth functions by deep neural networks with sigmoid activation function ⋮ Integral representations of shallow neural network with Rectified Power Unit activation function ⋮ Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations ⋮ Optimal nonparametric inference via deep neural network ⋮ Over-parametrized deep neural networks minimizing the empirical risk do not generalize well ⋮ Learning sparse deep neural networks with a spike-and-slab prior ⋮ Deep Network Approximation Characterized by Number of Neurons ⋮ A selective overview of deep learning ⋮ Estimation of an improved surrogate model in uncertainty quantification by neural networks ⋮ Estimation error analysis of deep learning on the regression problem on the variable exponent Besov space ⋮ On the rate of convergence of fully connected deep neural network regression estimates ⋮ Fast generalization error bound of deep learning without scale invariance of activation functions ⋮ Approximation rates for neural networks with encodable weights in smoothness spaces ⋮ On deep learning as a remedy for the curse of dimensionality in nonparametric regression ⋮ Function approximation by deep neural networks with parameters \(\{0, \pm \frac{1}{2}, \pm 1,2\}\) ⋮ Nonparametric regression with modified ReLU networks ⋮ Unnamed Item ⋮ On the rate of convergence of image classifiers based on convolutional neural networks ⋮ Measurement error models: from nonparametric methods to deep neural networks ⋮ High-performance statistical computing in the computing environments of the 2020s ⋮ Unnamed Item ⋮ Computer Model Calibration with Time Series Data Using Deep Learning and Quantile Regression ⋮ Robust deep neural network estimation for multi-dimensional functional data ⋮ A new deep neural network algorithm for multiple stopping with applications in options pricing ⋮ Error bounds for ReLU networks with depth and width parameters ⋮ Analysis of convolutional neural network image classifiers in a hierarchical max-pooling model with additional local pooling ⋮ Multilinear Compressive Sensing and an Application to Convolutional Linear Networks ⋮ Approximating functions with multi-features by deep convolutional neural networks ⋮ DNN expression rate analysis of high-dimensional PDEs: application to option pricing ⋮ Approximation spaces of deep neural networks

Uses Software

Cites Work

This page was built for publication: Nonparametric regression using deep neural networks with ReLU activation function