A multi-level procedure for enhancing accuracy of machine learning algorithms
From MaRDI portal
Bayesian inference (62F15) Artificial neural networks and deep learning (68T07) Bayesian problems; characterization of Bayes procedures (62C10) Finite difference methods for initial value and initial-boundary value problems involving PDEs (65M06) Finite volume methods for initial value and initial-boundary value problems involving PDEs (65M08) Probabilistic methods, particle methods, etc. for initial value and initial-boundary value problems involving PDEs (65M75)
Abstract: We propose a multi-level method to increase the accuracy of machine learning algorithms for approximating observables in scientific computing, particularly those that arise in systems modeled by differential equations. The algorithm relies on judiciously combining a large number of computationally cheap training data on coarse resolutions with a few expensive training samples on fine grid resolutions. Theoretical arguments for lowering the generalization error, based on reducing the variance of the underlying maps, are provided and numerical evidence, indicating significant gains over underlying single-level machine learning algorithms, are presented. Moreover, we also apply the multi-level algorithm in the context of forward uncertainty quantification and observe a considerable speed-up over competing algorithms.
Recommendations
- A machine learning framework for data driven acceleration of computations of differential equations
- Multilevel Fine-Tuning: Closing Generalization Gaps in Approximation of Solution Maps under a Limited Budget for Training Data
- Machine learning and computational mathematics
- Multi-level stochastic approximation algorithms
- Variational Monte Carlo -- bridging concepts of machine learning and high-dimensional partial differential equations
Cites work
- scientific article; zbMATH DE number 1215244 (Why is no real title available?)
- scientific article; zbMATH DE number 2000348 (Why is no real title available?)
- A machine learning framework for data driven acceleration of computations of differential equations
- An artificial neural network as a troubled-cell indicator
- Approximation by superpositions of a sigmoidal function
- Deep learning
- Deep learning observables in computational fluid dynamics
- Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
- Entropy stable scheme on two-dimensional unstructured grids for Euler equations
- Error bounds for approximations with deep ReLU networks
- Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations
- Hidden physics models: machine learning of nonlinear partial differential equations
- Multi-level Monte Carlo finite volume methods for nonlinear systems of conservation laws in multi-dimensions
- Multilayer feedforward networks are universal approximators
- Multilevel Monte Carlo Path Simulation
- On the mathematical foundations of learning
- Solving high-dimensional partial differential equations using deep learning
- Sparse tensor multi-level Monte Carlo finite volume methods for hyperbolic conservation laws with random initial data
- Statistical solutions of hyperbolic systems of conservation laws: Numerical approximation
- Survey of multifidelity methods in uncertainty propagation, inference, and optimization
- The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems
- Understanding machine learning. From theory to algorithms
Cited in
(13)- Connections between deep learning and partial differential equations
- Multilevel Fine-Tuning: Closing Generalization Gaps in Approximation of Solution Maps under a Limited Budget for Training Data
- Generalization error in the deep Ritz method with smooth activation functions
- Error analysis for deep neural network approximations of parametric hyperbolic conservation laws
- Enhancing Accuracy of Deep Learning Algorithms by Training with Low-Discrepancy Sequences
- An operator learning perspective on parameter-to-observable maps
- Multi-index ensemble Kalman filtering
- Multi-stage Neural Networks: Function Approximator of Machine Precision
- GFN: a graph feedforward network for resolution-invariant reduced operator learning in multifidelity applications
- Higher-order quasi-Monte Carlo training of deep neural networks
- A physics-informed multi-fidelity approach for the estimation of differential equations parameters in low-data or large-noise regimes
- Multifidelity Monte Carlo method for rapid uncertainty quantification in \(\mathrm{CO_2}\) storage applications
- Bayesian deep learning with multilevel trace-class neural networks
This page was built for publication: A multi-level procedure for enhancing accuracy of machine learning algorithms
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5014840)
