Deep Network Approximation for Smooth Functions

Shijun Zhang, Zuowei Shen, Haizhao Yang, Jian-feng Lu

Publication date: 7 October 2021

Published in: SIAM Journal on Mathematical Analysis (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2001.03040

zbMATH Keywords

polynomial approximation curse of dimensionality smooth function function composition deep ReLU network

Mathematics Subject Classification ID

Multidimensional problems (41A63) Rate of convergence, degree of approximation (41A25) Artificial intelligence (68Txx)

Related Items (57)

Stationary Density Estimation of Itô Diffusions Using Deep Learning ⋮ Neural network approximation ⋮ Discretization of parameter identification in PDEs using neural networks ⋮ Uniform approximation rates and metric entropy of shallow neural networks ⋮ ReLU deep neural networks from the hierarchical basis perspective ⋮ Quantitative Approximation Results for Complex-Valued Neural Networks ⋮ Deep Ritz Method for the Spectral Fractional Laplacian Equation Using the Caffarelli--Silvestre Extension ⋮ The Discovery of Dynamics via Linear Multistep Methods and Deep Learning: Error Estimation ⋮ DeepParticle: learning invariant measure by a deep neural network minimizing Wasserstein distance on data generated from an interacting particle method ⋮ Deep Neural Network Surrogates for Nonsmooth Quantities of Interest in Shape Uncertainty Quantification ⋮ Approximation bounds for norm constrained neural networks with applications to regression and GANs ⋮ Estimation of a regression function on a manifold by fully connected deep neural networks ⋮ Simultaneous neural network approximation for smooth functions ⋮ Deep Neural Networks for Solving Large Linear Systems Arising from High-Dimensional Problems ⋮ On the capacity of deep generative networks for approximating distributions ⋮ Deep learning methods for partial differential equations and related parameter identification problems ⋮ Learning-informed parameter identification in nonlinear time-dependent PDEs ⋮ Symplectic learning for Hamiltonian neural networks ⋮ Universal regular conditional distributions via probabilistic transformers ⋮ Exponential ReLU neural network approximation rates for point and edge singularities ⋮ A Hybrid Method for Three-Dimensional Semi-Linear Elliptic Equations ⋮ Active learning based sampling for high-dimensional nonlinear partial differential equations ⋮ A mathematical perspective of machine learning ⋮ SignReLU neural network and its approximation ability ⋮ Deep Neural Networks with ReLU-Sine-Exponential Activations Break Curse of Dimensionality in Approximation on Hölder Class ⋮ A Variational Neural Network Approach for Glacier Modelling with Nonlinear Rheology ⋮ Collocation approximation by deep neural ReLU networks for parametric and stochastic PDEs with lognormal inputs ⋮ Improved Analysis of PINNs: Alleviate the CoD for Compositional Solutions ⋮ Connections between Operator-Splitting Methods and Deep Neural Networks with Applications in Image Segmentation ⋮ Deep nonparametric regression on approximate manifolds: nonasymptotic error bounds with polynomial prefactors ⋮ Deep learning via dynamical systems: an approximation perspective ⋮ A Scalable Deep Learning Approach for Solving High-Dimensional Dynamic Optimal Transport ⋮ The universal approximation theorem for complex-valued neural networks ⋮ Analysis of the rate of convergence of two regression estimates defined by neural features which are easy to implement ⋮ Intrinsic and extrinsic deep learning on manifolds ⋮ On the rate of convergence of a deep recurrent neural network estimate in a regression problem with dependent data ⋮ Greedy training algorithms for neural networks and applications to PDEs ⋮ Approximation with one-bit polynomials in Bernstein form ⋮ Lipschitz widths ⋮ Approximation in shift-invariant spaces with deep ReLU neural networks ⋮ Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations ⋮ Error analysis based on inverse modified differential equations for discovery of dynamics using linear multistep methods and deep learning ⋮ How do noise tails impact on deep ReLU networks? ⋮ Factor Augmented Sparse Throughput Deep ReLU Neural Networks for High Dimensional Regression ⋮ Weighted variation spaces and approximation by shallow ReLU networks ⋮ Solving PDEs on unknown manifolds with machine learning ⋮ Gauss Newton method for solving variational problems of PDEs with neural network discretizaitons ⋮ Low dimensional approximation and generalization of multivariate functions on smooth manifolds using deep ReLU neural networks ⋮ Calibrating multi-dimensional complex ODE from noisy data via deep neural networks ⋮ Nonasymptotic bounds for adversarial excess risk under misspecified models ⋮ Computing ground states of Bose-Einstein condensation by normalized deep neural network ⋮ Approximation results for gradient flow trained shallow neural networks in \(1d\) ⋮ Generalization error guaranteed auto-encoder-based nonlinear model reduction for operator learning ⋮ Mini-workshop: Mathematics of entropic AI in the natural sciences. Abstracts from the mini-workshop held April 7--12, 2024 ⋮ On the rate of convergence of fully connected deep neural network regression estimates ⋮ Spline representation and redundancies of one-dimensional ReLU neural network models ⋮ Nonlinear approximation and (deep) ReLU networks

Uses Software

Cites Work

This page was built for publication: Deep Network Approximation for Smooth Functions