Higher-order quasi-Monte Carlo training of deep neural networks
From MaRDI portal
Publication:5015302
Abstract: We present a novel algorithmic approach and an error analysis leveraging Quasi-Monte Carlo points for training deep neural network (DNN) surrogates of Data-to-Observable (DtO) maps in engineering design. Our analysis reveals higher-order consistent, deterministic choices of training points in the input data space for deep and shallow Neural Networks with holomorphic activation functions such as tanh. These novel training points are proved to facilitate higher-order decay (in terms of the number of training samples) of the underlying generalization error, with consistency error bounds that are free from the curse of dimensionality in the input data space, provided that DNN weights in hidden layers satisfy certain summability conditions. We present numerical experiments for DtO maps from elliptic and parabolic PDEs with uncertain inputs that confirm the theoretical analysis.
Recommendations
- Quasi-Monte Carlo sampling for solving partial differential equations by deep neural networks
- Deep learning in high dimension: neural network expression rates for generalized polynomial chaos expansions in UQ
- Deep UQ: learning deep neural network surrogate models for high dimensional uncertainty quantification
- Deep neural network surrogates for nonsmooth quantities of interest in shape uncertainty quantification
- Constructive deep ReLU neural network approximation
Cites work
- scientific article; zbMATH DE number 1215244 (Why is no real title available?)
- scientific article; zbMATH DE number 1405266 (Why is no real title available?)
- A multi-level procedure for enhancing accuracy of machine learning algorithms
- A quasi-Monte Carlo data compression algorithm for machine learning
- Advanced Lectures on Machine Learning
- Breaking the curse of dimensionality in sparse polynomial approximation of parametric PDEs
- Certified reduced basis methods for parametrized partial differential equations
- Computational higher order quasi-Monte Carlo integration
- Convergence rates of best \(N\)-term Galerkin approximations for a class of elliptic SPDEs
- Convergence rates of high dimensional Smolyak quadrature
- Deep ReLU networks and high-order finite element methods
- Deep learning
- Deep learning observables in computational fluid dynamics
- Deep neural network expression of posterior expectations in Bayesian PDE inversion
- Domain uncertainty quantification in computational electromagnetics
- Enhancing Accuracy of Deep Learning Algorithms by Training with Low-Discrepancy Sequences
- Exponential ReLU DNN expression of holomorphic maps in high dimension
- Foundations of machine learning
- High-dimensional integration: The quasi-Monte Carlo way
- Higher order QMC Petrov-Galerkin discretization for affine parametric operator equations with random field inputs
- Higher order quasi-Monte Carlo integration for Bayesian PDE inversion
- Higher order quasi-Monte Carlo integration for holomorphic, parametric operator equations
- Iterative surrogate model optimization (ISMO): an active learning algorithm for PDE constrained optimization with deep neural networks
- Low-discrepancy point sets obtained by digital constructions over finite fields
- Machine learning for fast and reliable solution of time-dependent differential equations
- Multilevel higher-order quasi-Monte Carlo Bayesian estimation
- On the distribution of points in a cube and the approximate evaluation of integrals
- Optimal approximation with sparsely connected deep neural networks
- Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
- Reconstructing a neural net from its output
- Reduced basis methods for partial differential equations. An introduction
- Richardson extrapolation of polynomial lattice rules
- Shape holomorphy of the Calderón projector for the Laplacian in \(\mathbb{R}^2\)
- Shape holomorphy of the stationary Navier-Stokes equations
- Solving high-dimensional partial differential equations using deep learning
- Solving the Kolmogorov PDE by means of deep learning
- Sparse adaptive approximation of high dimensional parametric initial value problems
Cited in
(10)- Convergence Analysis of a Quasi-Monte CarloBased Deep Learning Algorithm for Solving Partial Differential Equations
- Deep learning based on randomized quasi-Monte Carlo method for solving linear Kolmogorov partial differential equation
- An overview on deep learning-based approximation methods for partial differential equations
- Wavenumber-Explicit Parametric Holomorphy of Helmholtz Solutions in the Context of Uncertainty Quantification
- Numerical analysis of physics-informed neural networks and related models in physics-informed machine learning
- The deep arbitrary polynomial chaos neural network or how deep artificial neural networks could benefit from data-driven homogeneous chaos theory
- Computing ground states of Bose-Einstein condensation by normalized deep neural network
- Adaptive quadratures for nonlinear approximation of low-dimensional PDEs using smooth neural networks
- Quasi-Monte Carlo sampling for solving partial differential equations by deep neural networks
- Greedy training algorithms for neural networks and applications to PDEs
This page was built for publication: Higher-order quasi-Monte Carlo training of deep neural networks
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5015302)
