Variational inference in neural functional prior using normalizing flows: application to differential equation and operator learning problems

DOI10.1007/S10483-023-2997-7zbMATH Open1518.76046arXiv2302.10448OpenAlexW4382894442MaRDI QIDQ6132292FDOQ6132292

Author name not available (Why is that?)

Publication date: 14 July 2023

Published in: AMM. Applied Mathematics and Mechanics. (English Edition) (Search for Journal in Brave)

Abstract: Physics-informed deep learning have recently emerged as an effective tool for leveraging both observational data and available physical laws. Physics-informed neural networks (PINNs) and deep operator networks (DeepONets) are two such models. The former encodes the physical laws via the automatic differentiation, while the latter learns the hidden physics from data. Generally, the noisy and limited observational data as well as the overparameterization in neural networks (NNs) result in uncertainty in predictions from deep learning models. In [1], a Bayesian framework based on the {{Generative Adversarial Networks}} (GAN) has been proposed as a unified model to quantify uncertainties in predictions of PINNs as well as DeepONets. Specifically, the proposed approach in [1] has two stages: (1) prior learning, and (2) posterior estimation. At the first stage, the GANs are employed to learn a functional prior either from a prescribed function distribution, e.g., Gaussian process, or from historical data and available physics. At the second stage, the Hamiltonian Monte Carlo (HMC) method is utilized to estimate the posterior in the latent space of GANs. However, the vanilla HMC does not support the mini-batch training, which limits its applications in problems with big data. In the present work, we propose to use the normalizing flow (NF) models in the context of variational inference, which naturally enables the minibatch training, as the alternative to HMC for posterior estimation in the latent space of GANs. A series of numerical experiments, including a nonlinear differential equation problem and a 100-dimensional Darcy problem, are conducted to demonstrate that NF with full-/mini-batch training are able to achieve similar accuracy as the ``gold rule HMC.

Full work available at URL: https://arxiv.org/abs/2302.10448

zbMATH Keywords

uncertainty quantificationgenerative adversarial networkHamiltonian Monte Carlo method100-dimensional Darcy problem

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Flows in porous media; filtration; seepage (76S05) Basic methods in fluid mechanics (76M99)

Cites Work

• The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo
• DGM: a deep learning algorithm for solving partial differential equations
• Solving high-dimensional partial differential equations using deep learning
• The Deep Ritz Method: a deep learning-based numerical algorithm for solving variational problems
• Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
• Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems
• Uncertainty quantification in scientific machine learning: methods, metrics, and comparisons
• A composite neural network that learns from multi-fidelity data: application to function approximation and inverse PDE problems
• B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data
• Multi-fidelity Bayesian neural networks: algorithms and applications
• A fast multi-fidelity method with uncertainty quantification for complex data correlations: application to vortex-induced vibrations of marine risers
• Bayesian physics informed neural networks for real-world nonlinear dynamical systems
• A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data
• Learning functional priors and posteriors from data and physics

Cited In (2)

• Quasipolynomial Normalisation in Deep Inference via Atomic Flows and Threshold Formulae
• Preface: machine-learning approaches for computational mechanics

   Recommendations

• Title not available (Why is that?) 👍 👎
• Bayesian learning via neural Schrödinger-Föllmer flows 👍 👎
• Variational inference with NoFAS: normalizing flow with adaptive surrogate for computationally expensive models 👍 👎
• Variational Inference for Stochastic Differential Equations 👍 👎
• Variational training of neural network approximations of solution maps for physical models 👍 👎
• Variational Bayes Solution of Linear Neural Networks and Its Generalization Performance 👍 👎
• Variational inference for nonlinear inverse problems via neural net kernels: comparison to Bayesian neural networks, application to topology optimization 👍 👎
• Deep Variational Inference 👍 👎
• Normalizing Flows Aided Variational Inference: A Useful Alternative to MCMC? 👍 👎
• Stochastic Normalizing Flows for Inverse Problems: A Markov Chains Viewpoint 👍 👎

This page was built for publication: Variational inference in neural functional prior using normalizing flows: application to differential equation and operator learning problems

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6132292)