Quality measures for the evaluation of machine learning architectures on the quantification of epistemic and aleatoric uncertainties in complex dynamical systems
From MaRDI portal
(Redirected from Publication:6153910)
Abstract: Machine learning methods for the construction of data-driven reduced order model models are used in an increasing variety of engineering domains, especially as a supplement to expensive computational fluid dynamics for design problems. An important check on the reliability of surrogate models is Uncertainty Quantification (UQ), a self assessed estimate of the model error. Accurate UQ allows for cost savings by reducing both the required size of training data sets and the required safety factors, while poor UQ prevents users from confidently relying on model predictions. We examine several machine learning techniques, including both Gaussian processes and a family UQ-augmented neural networks: Ensemble neural networks (ENN), Bayesian neural networks (BNN), Dropout neural networks (D-NN), and Gaussian neural networks (G-NN). We evaluate UQ accuracy (distinct from model accuracy) using two metrics: the distribution of normalized residuals on validation data, and the distribution of estimated uncertainties. We apply these metrics to two model data sets, representative of complex dynamical systems: an ocean engineering problem in which a ship traverses irregular wave episodes, and a dispersive wave turbulence system with extreme events, the Majda-McLaughlin-Tabak model. We present conclusions concerning model architecture and hyperparameter tuning.
Recommendations
- Deep UQ: learning deep neural network surrogate models for high dimensional uncertainty quantification
- Uncertainty quantification in scientific machine learning: methods, metrics, and comparisons
- Assessments of epistemic uncertainty using Gaussian stochastic weight averaging for fluid-flow regression
- Bayesian reduced-order deep learning surrogate model for dynamic systems described by partial differential equations
- Estimation of an improved surrogate model in uncertainty quantification by neural networks
Cites work
- 10.1162/15324430260185646
- A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data
- A one-dimensional model for dispersive wave turbulence
- Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods
- Bayesian neural networks: an introduction and survey
- Convergence of sparse variational inference in Gaussian processes regression
- Dispersive wave turbulence in one dimension
- scientific article; zbMATH DE number 6378127 (Why is no real title available?)
- scientific article; zbMATH DE number 7626805 (Why is no real title available?)
- Karhunen-Loève representation of stochastic ocean waves
- Model reduction and neural networks for parametric PDEs
- Note on a modification to the nonlinear Schrödinger equation for application to deep water waves
- Output-Weighted Optimal Sampling for Bayesian Experimental Design and Uncertainty Quantification
- Probabilistic machine learning. An introduction
- Quantification and prediction of extreme events in a one-dimensional nonlinear dispersive wave model
- Sequential Bayesian experimental design for estimation of extreme-event probability in stochastic input-to-response systems
- Sequential sampling strategy for extreme event statistics in nonlinear dynamical systems
- The disintegration of wave trains on deep water Part 1. Theory
- Uncertainty quantification in scientific machine learning: methods, metrics, and comparisons
- Understanding machine learning. From theory to algorithms
- Wave turbulence in one-dimensional models
- Weak turbulence and collapses in the Majda--McLaughlin--Tabak equation: Fluxes in wavenumber and in amplitude space
Cited in
(3)- Assessments of epistemic uncertainty using Gaussian stochastic weight averaging for fluid-flow regression
- Uncertainty quantification in autoencoders predictions: applications in aerodynamics
- Full-field experiment-aided virtual modelling framework for inverse-based stochastic prediction of structures with elastoplasticity
This page was built for publication: Quality measures for the evaluation of machine learning architectures on the quantification of epistemic and aleatoric uncertainties in complex dynamical systems
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6153910)
