Manifold learning for accelerating coarse-grained optimization
From MaRDI portal
Publication:2194441
derivative-free optimizationdiffusion mapsgeometric harmonicsLangevin equationmanifold learningsimulated annealing
Approximation methods and numerical treatment of dynamical systems (37M99) Dynamical systems in optimization and economics (37N40) Learning and adaptive systems in artificial intelligence (68T05) General topics in computing methodologies (68U01) Derivative-free methods and methods using generalized derivatives (90C56)
Abstract: Algorithms proposed for solving high-dimensional optimization problems with no derivative information frequently encounter the "curse of dimensionality," becoming ineffective as the dimension of the parameter space grows. One feature of a subclass of such problems that are effectively low-dimensional is that only a few parameters (or combinations thereof) are important for the optimization and must be explored in detail. Knowing these parameters/ combinations in advance would greatly simplify the problem and its solution. We propose the data-driven construction of an effective (coarse-grained, "trend") optimizer, based on data obtained from ensembles of brief simulation bursts with an "inner" optimization algorithm, that has the potential to accelerate the exploration of the parameter space. The trajectories of this "effective optimizer" quickly become attracted onto a slow manifold parameterized by the few relevant parameter combinations. We obtain the parameterization of this low-dimensional, effective optimization manifold on the fly using data mining/manifold learning techniques on the results of simulation (inner optimizer iteration) burst ensembles and exploit it locally to "jump" forward along this manifold. As a result, we can bias the exploration of the parameter space towards the few, important directions and, through this "wrapper algorithm," speed up the convergence of traditional optimization algorithms.
Recommendations
- Manifold learning for parameter reduction
- Manifold mapping: a two-level optimization technique
- A stochastic subspace approach to gradient-free optimization in high dimensions
- Global optimization using random embeddings
- A consensus-based global optimization method for high dimensional machine learning problems
Cites work
- `` Direct Search Solution of Numerical and Statistical Problems
- A new framework for extracting coarse-grained models from time series with multiscale structure
- A note on diffusion limits of chaotic skew-product flows
- A Simplex Method for Function Minimization
- A taxonomy of global optimization methods based on response surfaces
- Analysis of the ensemble Kalman filter for inverse problems
- Closed-form likelihood expansions for multivariate diffusions
- Computing on virtual slow manifolds of fast stochastic systems
- Derivative-free optimization: a review of algorithms and comparison of software implementations
- Diffusion maps
- Diffusions for Global Optimization
- Equation of state calculations by fast computing machines
- Equation-free, coarse-grained multiscale computation: enabling microscopic simulators to perform system-level analysis
- Geometric diffusions as a tool for harmonic analysis and structure definition of data: diffusion maps
- Geometric harmonics: a novel tool for multiscale out-of-sample extension of empirical functions
- Global optimization by multilevel coordinate search
- Heterogeneous multiscale methods: a review
- scientific article; zbMATH DE number 3497315 (Why is no real title available?)
- scientific article; zbMATH DE number 1274356 (Why is no real title available?)
- scientific article; zbMATH DE number 1857675 (Why is no real title available?)
- Improving dynamical properties of metropolized discretizations of overdamped Langevin dynamics
- Introduction to Derivative-Free Optimization
- Large Sample Properties of Generalized Method of Moments Estimators
- Lipschitzian optimization without the Lipschitz constant
- Maximum Likelihood Estimation of Discretely Sampled Diffusions: A Closed-form Approximation Approach
- Non-linear independent component analysis with diffusion maps
- Numerical techniques for multi-scale dynamical systems with stochastic effects
- Optimization by simulated annealing
- Parameter estimation for multiscale diffusions
- Parsimonious representation of nonlinear dynamical systems through manifold learning: a chemotaxis case study
- Particle Filters for Multiscale Diffusions
- Principal component analysis.
- Projective Methods for Stiff Differential Equations: Problems with Gaps in Their Eigenvalue Spectrum
- Semiparametric drift and diffusion estimation for multiscale diffusions
- Stochastic differential equations. An introduction with applications.
- Stochastic processes and applications. Diffusion processes, the Fokker-Planck and Langevin equations
- Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions
- Trust Region Methods
Cited in
(7)- Equation-free patch scheme for efficient computational homogenisation via self-adjoint coupling
- Manifold learning for parameter reduction
- Manifold learning for efficient gravitational search algorithm
- Manifold learning for coherent design interpolation based on geometrical and topological descriptors
- scientific article; zbMATH DE number 1167661 (Why is no real title available?)
- Multi‐fidelity data fusion through parameter space reduction with applications to automotive engineering
- Scientific machine learning
This page was built for publication: Manifold learning for accelerating coarse-grained optimization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2194441)
