Variational inference in neural functional prior using normalizing flows: application to differential equation and operator learning problems
From MaRDI portal
(Redirected from Publication:6132292)
Abstract: Physics-informed deep learning have recently emerged as an effective tool for leveraging both observational data and available physical laws. Physics-informed neural networks (PINNs) and deep operator networks (DeepONets) are two such models. The former encodes the physical laws via the automatic differentiation, while the latter learns the hidden physics from data. Generally, the noisy and limited observational data as well as the overparameterization in neural networks (NNs) result in uncertainty in predictions from deep learning models. In [1], a Bayesian framework based on the {{Generative Adversarial Networks}} (GAN) has been proposed as a unified model to quantify uncertainties in predictions of PINNs as well as DeepONets. Specifically, the proposed approach in [1] has two stages: (1) prior learning, and (2) posterior estimation. At the first stage, the GANs are employed to learn a functional prior either from a prescribed function distribution, e.g., Gaussian process, or from historical data and available physics. At the second stage, the Hamiltonian Monte Carlo (HMC) method is utilized to estimate the posterior in the latent space of GANs. However, the vanilla HMC does not support the mini-batch training, which limits its applications in problems with big data. In the present work, we propose to use the normalizing flow (NF) models in the context of variational inference, which naturally enables the minibatch training, as the alternative to HMC for posterior estimation in the latent space of GANs. A series of numerical experiments, including a nonlinear differential equation problem and a 100-dimensional Darcy problem, are conducted to demonstrate that NF with full-/mini-batch training are able to achieve similar accuracy as the ``gold rule HMC.
Recommendations
- Normalizing flows for probabilistic modeling and inference
- Bayesian learning via neural Schrödinger-Föllmer flows
- Variational inference with NoFAS: normalizing flow with adaptive surrogate for computationally expensive models
- Variational Inference for Stochastic Differential Equations
- Variational training of neural network approximations of solution maps for physical models
- Variational Bayes Solution of Linear Neural Networks and Its Generalization Performance
- Variational inference for nonlinear inverse problems via neural net kernels: comparison to Bayesian neural networks, application to topology optimization
- Deep variational inference
- Normalizing Flows Aided Variational Inference: A Useful Alternative to MCMC?
- Stochastic normalizing flows for inverse problems: a Markov chains viewpoint
Cites work
- A composite neural network that learns from multi-fidelity data: application to function approximation and inverse PDE problems
- A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data
- A fast multi-fidelity method with uncertainty quantification for complex data correlations: application to vortex-induced vibrations of marine risers
- B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data
- Bayesian physics informed neural networks for real-world nonlinear dynamical systems
- DGM: a deep learning algorithm for solving partial differential equations
- Learning functional priors and posteriors from data and physics
- Multi-fidelity Bayesian neural networks: algorithms and applications
- Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
- Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems
- Solving high-dimensional partial differential equations using deep learning
- The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems
- The no-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo
- Uncertainty quantification in scientific machine learning: methods, metrics, and comparisons
Cited in
(4)- Preface: machine-learning approaches for computational mechanics
- Multi-head physics-informed neural networks for learning functional priors and uncertainty quantification
- Quasipolynomial Normalisation in Deep Inference via Atomic Flows and Threshold Formulae
- From local interactions to global operators: scalable Gaussian process operator for physical systems
This page was built for publication: Variational inference in neural functional prior using normalizing flows: application to differential equation and operator learning problems
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6132292)
