Variational Monte Carlo -- bridging concepts of machine learning and high-dimensional partial differential equations

DOI10.1007/S10444-019-09723-8MaRDI QIDQ2305540zbMATH OpenOpenAlexFDO

Authors Martin Eigel, Philipp Trunschke, S. Wolf, Reinhold Schneider

Publication date 11 March 2020

Published in Advances in Computational Mathematics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1810.01348

zbMATH Keywords

machine learningpartial differential equationsstatistical learningtree tensor networksuncertainty quantification

Mathematics Subject Classification ID

Multilinear algebra, tensor calculus (15A69) PDEs with randomness, stochastic partial differential equations (35R60) General nonlinear regression (62J02) Monte Carlo methods (65C05) Algorithms for approximation of functions (65D15) Numerical solution of discretized equations for boundary value problems involving PDEs (65N22) Finite element, Rayleigh-Ritz and Galerkin methods for boundary value problems involving PDEs (65N30) Learning and adaptive systems in artificial intelligence (68T05)

Abstract: A statistical learning approach for parametric PDEs related to Uncertainty Quantification is derived. The method is based on the minimization of an empirical risk on a selected model class and it is shown to be applicable to a broad range of problems. A general unified convergence analysis is derived, which takes into account the approximation and the statistical errors. By this, a combination of theoretical results from numerical analysis and statistics is obtained. Numerical experiments illustrate the performance of the method with the model class of hierarchical tensors.

Recommendations

• Algorithms for solving high dimensional PDEs: from nonlinear Monte Carlo to machine learning
• Solving parametric PDE problems with artificial neural networks
• The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems
• Solving Kolmogorov PDEs without the curse of dimensionality via deep learning and asymptotic expansion with Malliavin calculus
• Numerical solution of the parametric diffusion equation by deep neural networks

Cites work

• N-term Wiener chaos approximation rates for elliptic PDEs with lognormal Gaussian random inputs
• A convergent adaptive stochastic Galerkin finite element method with quasi-optimal spatial meshes
• A new scheme for the tensor representation
• A Proof that Artificial Neural Networks Overcome the Curse of Dimensionality in the Numerical Approximation of Black–Scholes Partial Differential Equations
• A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations
• A Stochastic Collocation Method for Elliptic Partial Differential Equations with Random Input Data
• A theory of the learnable
• Adaptive stochastic Galerkin FEM
• Adaptive stochastic Galerkin FEM for lognormal coefficients in hierarchical tensor representations
• Adaptive stochastic Galerkin FEM with hierarchical tensor representations
• An adaptive multilevel Monte Carlo method with stochastic bounds for quantities of interest with uncertain data
• An Anisotropic Sparse Grid Stochastic Collocation Method for Partial Differential Equations with Random Input Data
• An introduction to computational stochastic PDEs
• An introduction to statistical learning. With applications in R
• Analysis of the generalization error: empirical risk minimization over deep artificial neural networks overcomes the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations
• Analytic regularity and polynomial approximation of parametric and stochastic elliptic PDE's
• Approximation in learning theory
• Approximation of high-dimensional parametric PDEs
• Convergence rates of best \(N\)-term Galerkin approximations for a class of elliptic SPDEs
• Decision theoretic generalizations of the PAC model for neural net and other learning applications
• Deep learning
• DeepXDE: a deep learning library for solving differential equations
• Energy norm A posteriori error estimation for parametric operator equations
• Error Estimates for Multivariate Regression on Discretized Function Spaces
• Finite Elements
• Galerkin methods for linear and nonlinear elliptic stochastic partial differential equations
• scientific article; zbMATH DE number 3877692 (Why is no real title available?)
• scientific article; zbMATH DE number 49187 (Why is no real title available?)
• scientific article; zbMATH DE number 1391397 (Why is no real title available?)
• Inferring solutions of differential equations using noisy multi-fidelity data
• Learning Theory
• Lectures on the finite element method. Notes by S. Kesavan, Akhil Ranjan, M. Vanninathan
• Local equilibration error estimators for guaranteed error control in adaptive stochastic higher-order Galerkin finite element methods
• Low rank tensor recovery via iterative hard thresholding
• Machine learning of linear differential equations using Gaussian processes
• Necessary and Sufficient Conditions for the Uniform Convergence of Means to their Expectations
• Non-intrusive tensor reconstruction for high-dimensional random PDEs
• Numerical analysis for integral and related operator equations.
• On the stability and accuracy of least squares approximations
• Online learning and online convex optimization
• Optimal approximation with sparsely connected deep neural networks
• Optimal weighted least-squares methods
• Parametric PDEs: sparse or low-rank approximations?
• Pattern recognition and machine learning.
• Proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients
• Sequential sampling for optimal weighted least squares approximations in hierarchical spaces
• Size-independent sample complexity of neural networks
• Solution of stochastic partial differential equations using Galerkin finite element techniques
• Solving high-dimensional partial differential equations using deep learning
• Solving the Kolmogorov PDE by means of deep learning
• Sparse tensor discretizations of high-dimensional parametric and stochastic PDEs
• Spectral Methods for Uncertainty Quantification
• Stochastic differential equations. An introduction with applications.
• Stochastic processes and applications. Diffusion processes, the Fokker-Planck and Langevin equations
• Tensor networks and hierarchical tensors for the solution of high-dimensional partial differential equations
• Tensor spaces and numerical tensor calculus
• Tensor-train decomposition
• The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems
• The elements of statistical learning. Data mining, inference, and prediction
• Theory of uniform convergence of frequencies of events to their probabilities and problems of search for an optimal solution from empirical data
• To be or not to be intrusive? The solution of parametric and stochastic equations -- proper generalized decomposition
• Vegas revisited: Adaptive Monte Carlo integration beyond factorization

Cited in

(19)

• Numerical solution of the parametric diffusion equation by deep neural networks
• A theoretical analysis of deep neural networks and parametric PDEs
• Low-rank tensor reconstruction of concentrated densities with application to Bayesian inversion
• Variational problems in machine learning and their solution with finite elements
• A multi-level procedure for enhancing accuracy of machine learning algorithms
• Convergence bounds for empirical nonlinear least-squares
• scientific article; zbMATH DE number 7307488 (Why is no real title available?)
• Approximative policy iteration for exit time feedback control problems driven by stochastic differential equations using tensor train format
• Adaptive Nonintrusive Reconstruction of Solutions to High-Dimensional Parametric PDEs
• Data-Driven Tensor Train Gradient Cross Approximation for Hamilton–Jacobi–Bellman Equations
• Probabilistic partition of unity networks for high‐dimensional regression problems
• Pricing High-Dimensional Bermudan Options with Hierarchical Tensor Formats
• Randomized residual-based error estimators for the proper generalized decomposition approximation of parametrized problems
• Convergence of variational Monte Carlo simulation and scale-invariant pre-training
• Mini-workshop: Nonlinear approximation of high-dimensional functions in scientific computing. Abstracts from the mini-workshop held October 15--20, 2023
• Approximating the stationary Bellman equation by hierarchical tensor products
• Sample complexity bounds for the local convergence of least squares approximation
• Weighted sparsity and sparse tensor networks for least squares approximation
• Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space

Describes a project that uses

Uses Software

• FEniCS
• ALEA
• PRMLT
• DeepXDE
• xerus

This page was built for publication: Variational Monte Carlo -- bridging concepts of machine learning and high-dimensional partial differential equations

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2305540)