Ensemble Kalman inversion: a derivative-free technique for machine learning tasks
From MaRDI portal
Publication:5197869
Abstract: The standard probabilistic perspective on machine learning gives rise to empirical risk-minimization tasks that are frequently solved by stochastic gradient descent (SGD) and variants thereof. We present a formulation of these tasks as classical inverse or filtering problems and, furthermore, we propose an efficient, gradient-free algorithm for finding a solution to these problems using ensemble Kalman inversion (EKI). Applications of our approach include offline and online supervised learning with deep neural networks, as well as graph-based semi-supervised learning. The essence of the EKI procedure is an ensemble based approximate gradient descent in which derivatives are replaced by differences from within the ensemble. We suggest several modifications to the basic method, derived from empirically successful heuristics developed in the context of SGD. Numerical results demonstrate wide applicability and robustness of the proposed algorithm.
Recommendations
- Ensemble Kalman filter for neural network-based one-shot inversion
- Convergence acceleration of ensemble Kalman inversion in nonlinear settings
- Well posedness and convergence analysis of the ensemble Kalman inversion
- Gradient flow structure and convergence analysis of the ensemble Kalman inversion for nonlinear forward models
- Adaptive Tikhonov strategies for stochastic ensemble Kalman inversion
Cites work
- scientific article; zbMATH DE number 3850830 (Why is no real title available?)
- scientific article; zbMATH DE number 3945130 (Why is no real title available?)
- scientific article; zbMATH DE number 1269194 (Why is no real title available?)
- scientific article; zbMATH DE number 823069 (Why is no real title available?)
- scientific article; zbMATH DE number 1408945 (Why is no real title available?)
- A Bayesian level set method for geometric inverse problems
- A Stochastic Approximation Method
- Adaptive subgradient methods for online learning and stochastic optimization
- AlexNet
- Analysis of the ensemble Kalman filter for inverse problems
- Analysis of the ensemble and polynomial chaos Kalman filters in Bayesian inverse problems
- Computational Methods for Inverse Problems
- Data Assimilation
- Data. A collection of problems from many fields for the student and research worker
- Deep learning
- Diffuse Interface Models on Graphs for Classification of High Dimensional Data
- Ensemble Kalman methods for inverse problems
- Entropy-SGD: biasing gradient descent into wide valleys
- FitNets
- Gaussian processes for machine learning.
- Learning from examples as an inverse problem
- Machine learning. A probabilistic perspective
- Minimizing finite sums with the stochastic average gradient
- Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies?
- Nonparametric stochastic approximation with large step-sizes
- Reducing the Dimensionality of Data with Neural Networks
- Ridge Regression: Biased Estimation for Nonorthogonal Problems
- Stochastic Estimation of the Maximum of a Regression Function
- Uncertainty quantification in graph-based classification of high dimensional data
Cited in
(59)- Nesterov acceleration for ensemble Kalman inversion and variants
- Adaptive Ensemble Kalman Inversion with Statistical Linearization
- Efficient Bayesian physics informed neural networks for inverse problems via ensemble Kalman inversion
- Quantification and reduction of uncertainties in a wind turbine numerical model based on a global sensitivity analysis and a recursive Bayesian inference approach
- Recent trends on nonlinear filtering for inverse problems
- Continuous limits of residual neural networks in case of large input data
- Convergence acceleration of ensemble Kalman inversion in nonlinear settings
- Ensemble Kalman methods: a mean-field perspective
- EnKSGD: a class of preconditioned black box optimization and inversion algorithms
- Analysis of stochastic gradient descent in continuous time
- Scientific machine learning for closure models in multiscale problems: a review
- Deep limits of residual neural networks
- Ensemble Kalman filter for neural network-based one-shot inversion
- Dropout ensemble Kalman inversion for high dimensional inverse problems
- Efficient estimation of hydraulic conductivity heterogeneity with non-redundant measurement information
- Parallel ensemble Kalman method with total variation regularization for large-scale field inversion
- Hyperparameter optimization for randomized algorithms: a case study on random features
- Consensus‐based sampling
- Subsampling in ensemble Kalman inversion
- Uncertainty quantification for DeepONet with ensemble Kalman inversion
- Interacting Langevin diffusions: gradient structure and ensemble Kalman sampler
- Affine invariant ensemble transform methods to improve predictive uncertainty in neural networks
- Iterated Kalman methodology for inverse problems
- A localized consensus-based sampling algorithm
- Gradient flow structure and convergence analysis of the ensemble Kalman inversion for nonlinear forward models
- Continuous Time Limit of the Stochastic Ensemble Kalman Inversion: Strong Convergence Analysis
- Hierarchical ensemble Kalman methods with sparsity-promoting generalized gamma hyperpriors
- Adaptive Tikhonov strategies for stochastic ensemble Kalman inversion
- Adaptive regularisation for ensemble Kalman inversion
- Mean-field and kinetic descriptions of neural differential equations
- Bayesian experimental design for model discrepancy calibration: an auto-differentiable ensemble Kalman inversion approach
- Affine invariant interacting Langevin dynamics for Bayesian inference
- Iterative ensemble Kalman methods: a unified perspective with some new variants
- Localized ensemble Kalman inversion
- On convergence rates of adaptive ensemble Kalman inversion for linear ill-posed problems
- Neural dynamical operator: continuous spatial-temporal model with gradient-based and derivative-free optimization methods
- Identifying a fractional order and a time-dependent coefficient in a time-fractional diffusion wave equation
- Sequential Kalman tuning of the t-preconditioned Crank-Nicolson algorithm: efficient, adaptive and gradient-free inference for Bayesian inverse problems
- Tikhonov regularization within ensemble Kalman inversion
- Fokker-Planck particle systems for Bayesian inference: computational approaches
- l_p regularization for ensemble Kalman inversion
- Covariance-modulated optimal transport and gradient flows
- A stabilization of a continuous limit of the ensemble Kalman inversion
- Online learning of both state and dynamics using ensemble Kalman filters
- The fundamental subspaces of ensemble Kalman inversion
- Harmonic analysis on graphs via Bratteli diagrams and path-space measures
- Calibrate, emulate, sample
- Large-eddy simulation-based shape optimization for mitigating turbulent wakes of a bluff body using the regularized ensemble Kalman method
- Derivative-Free Bayesian Inversion Using Multiscale Dynamics
- Scalable Bayesian physics-informed Kolmogorov-Arnold networks
- Solving continuum and rarefied flows using differentiable programming
- Active learning of model discrepancy with Bayesian experimental design
- Ensemble Kalman method for learning turbulence models from indirect observation data
- Ensemble inference methods for models with noisy and expensive likelihoods
- Derivative-free optimization on Riemannian manifolds using simplex gradient approximations
- Affine-invariant ensemble transform methods for logistic regression
- Ensemble Gradient for Learning Turbulence Models from Indirect Observations
- A framework for learning symbolic turbulence models from indirect observation data via neural networks and feature importance analysis
- Discrete gradients for computational Bayesian inference
Describes a project that uses
Uses Software
This page was built for publication: Ensemble Kalman inversion: a derivative-free technique for machine learning tasks
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5197869)
