Deep vs. shallow networks: An approximation theory perspective
From MaRDI portal
Publication:2835988
DOI10.1142/S0219530516400042zbMath1355.68233arXiv1608.03287OpenAlexW2513671774WikidataQ125020678 ScholiaQ125020678MaRDI QIDQ2835988
Tomaso Poggio, Hrushikesh N. Mhaskar
Publication date: 30 November 2016
Published in: Analysis and Applications (Search for Journal in Brave)
Full work available at URL: https://arxiv.org/abs/1608.03287
Learning and adaptive systems in artificial intelligence (68T05) Rate of convergence, degree of approximation (41A25) Approximation by arbitrary nonlinear expressions; widths and entropy (41A46)
Related Items (79)
Low-rank approximation of continuous functions in Sobolev spaces with dominating mixed smoothness ⋮ A Proof that Artificial Neural Networks Overcome the Curse of Dimensionality in the Numerical Approximation of Black–Scholes Partial Differential Equations ⋮ On sharpness of an error bound for deep ReLU network approximation ⋮ High-Dimensional Learning Under Approximate Sparsity with Applications to Nonsmooth Estimation and Regularized Neural Networks ⋮ Feedforward Neural Networks and Compositional Functions with Applications to Dynamical Systems ⋮ Deep distributed convolutional neural networks: Universality ⋮ Theoretical issues in deep networks ⋮ Full error analysis for the training of deep neural networks ⋮ Why does deep and cheap learning work so well? ⋮ Voronovskaja type theorems and high-order convergence neural network operators with sigmoidal functions ⋮ On the approximation by single hidden layer feedforward neural networks with fixed weights ⋮ Saturation classes for MAX-product neural network operators activated by sigmoidal functions ⋮ An analysis of training and generalization errors in shallow and deep networks ⋮ Universal approximation with quadratic deep networks ⋮ Estimation of a regression function on a manifold by fully connected deep neural networks ⋮ Stable parameterization of continuous and piecewise-linear functions ⋮ A deep network construction that adapts to intrinsic dimensionality beyond the domain ⋮ Theory of deep convolutional neural networks. III: Approximating radial functions ⋮ De Rham compatible deep neural network FEM ⋮ Approximating smooth and sparse functions by deep neural networks: optimal approximation rates and saturation ⋮ Rates of approximation by ReLU shallow neural networks ⋮ Three ways to solve partial differential equations with neural networks — A review ⋮ Kähler geometry of framed quiver moduli and machine learning ⋮ The Kolmogorov-Arnold representation theorem revisited ⋮ On decision regions of narrow deep neural networks ⋮ Approximation Analysis of Convolutional Neural Networks ⋮ Time discretization in the solution of parabolic PDEs with ANNs ⋮ Function approximation by deep networks ⋮ Overall error analysis for the training of deep neural networks via stochastic gradient descent with random initialisation ⋮ Learning sparse and smooth functions by deep sigmoid nets ⋮ Dunkl analouge of Szász Schurer Beta bivariate operators ⋮ Error convergence and engineering-guided hyperparameter search of PINNs: towards optimized I-FENN performance ⋮ Data augmentation for Bayesian deep learning ⋮ On the number of regions of piecewise linear neural networks ⋮ Lower bounds for artificial neural network approximations: a proof that shallow neural networks fail to overcome the curse of dimensionality ⋮ Approximation of compositional functions with ReLU neural networks ⋮ Local approximation of operators ⋮ Mini-workshop: Analysis of data-driven optimal control. Abstracts from the mini-workshop held May 9--15, 2021 (hybrid meeting) ⋮ Topology optimization based on deep representation learning (DRL) for compliance and stress-constrained design ⋮ Deep learning in high dimension: Neural network expression rates for generalized polynomial chaos expansions in UQ ⋮ Nonparametric regression using deep neural networks with ReLU activation function ⋮ Rectified deep neural networks overcome the curse of dimensionality for nonsmooth value functions in zero-sum games of nonlinear stiff systems ⋮ Convergence rate of DeepONets for learning operators arising from advection-diffusion equations ⋮ Convergence of the deep BSDE method for coupled FBSDEs ⋮ Applied harmonic analysis and data processing. Abstracts from the workshop held March 25--31, 2018 ⋮ Function approximation with zonal function networks with activation functions analogous to the rectified linear unit functions ⋮ Estimates for the neural network operators of the max-product type with continuous and \(p\)-integrable functions ⋮ Unnamed Item ⋮ Butterfly-Net: Optimal Function Representation Based on Convolutional Neural Networks ⋮ Asymptotic expansion for neural network operators of the Kantorovich type and high order of approximation ⋮ Quantitative estimates involving K-functionals for neural network-type operators ⋮ Deep autoencoder based energy method for the bending, vibration, and buckling analysis of Kirchhoff plates with transfer learning ⋮ Statistical insights into deep neural network learning in subspace classification ⋮ Investigating deep energy method applications in thermoelasticity ⋮ Compositional sparsity of learnable functions ⋮ Relevant sampling in a reproducing kernel subspace of Orlicz space ⋮ Variational temporal convolutional networks for I-FENN thermoelasticity ⋮ Approximation rates for deep calibration of (rough) stochastic volatility models ⋮ A linear relation between input and first layer in neural networks ⋮ Learning and approximating piecewise smooth functions by deep sigmoid neural networks ⋮ Choice of interior penalty coefficient for interior penalty discontinuous Galerkin method for Biot's system by employing machine learning ⋮ Universality of deep convolutional neural networks ⋮ On the rate of convergence of fully connected deep neural network regression estimates ⋮ A direct approach for function approximation on data defined manifolds ⋮ An analytic layer-wise deep learning framework with applications to robotics ⋮ On deep learning as a remedy for the curse of dimensionality in nonparametric regression ⋮ Application of deep learning neural network to identify collision load conditions based on permanent plastic deformation of shell structures ⋮ Deep neural networks for rotation-invariance approximation and learning ⋮ Robust randomized optimization with k nearest neighbors ⋮ Generalization Error of Minimum Weighted Norm and Kernel Interpolation ⋮ Optimal adaptive control of partially uncertain linear continuous-time systems with state delay ⋮ Super-resolution meets machine learning: approximation of measures ⋮ Stable recovery of entangled weights: towards robust identification of deep neural networks from minimal samples ⋮ A measure theoretical approach to the mean-field maximum principle for training NeurODEs ⋮ Error bounds for ReLU networks with depth and width parameters ⋮ Optimal Approximation with Sparsely Connected Deep Neural Networks ⋮ Approximating functions with multi-features by deep convolutional neural networks ⋮ Nonlinear approximation and (deep) ReLU networks ⋮ Approximation spaces of deep neural networks
Cites Work
- Neocognition: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position
- When is approximation by Gaussian networks necessarily a linear process?
- Limitations of the approximation capabilities of neural networks with one hidden layer
- Weighted quadrature formulas and approximation by zonal function networks on the sphere
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
This page was built for publication: Deep vs. shallow networks: An approximation theory perspective
