Bayesian optimization in a billion dimensions via random embeddings
From MaRDI portal
Abstract: Bayesian optimization techniques have been successfully applied to robotics, planning, sensor placement, recommendation, advertising, intelligent user interfaces and automatic algorithm configuration. Despite these successes, the approach is restricted to problems of moderate dimension, and several workshops on Bayesian optimization have identified its scaling to high-dimensions as one of the holy grails of the field. In this paper, we introduce a novel random embedding idea to attack this problem. The resulting Random EMbedding Bayesian Optimization (REMBO) algorithm is very simple, has important invariance properties, and applies to domains with both categorical and continuous variables. We present a thorough theoretical analysis of REMBO. Empirical results confirm that REMBO can effectively solve problems with billions of dimensions, provided the intrinsic dimensionality is low. They also show that REMBO achieves state-of-the-art performance in optimizing the 47 discrete parameters of a popular mixed integer linear programming solver.
Recommendations
- On the choice of the low-dimensional domain for global optimization via random embeddings
- Fast Bayesian hyperparameter optimization on large datasets
- Constrained Bayesian optimization with noisy experiments
- Survey on Bayesian optimization methodology and applications
- Global optimization using random embeddings
Cited in
(49)- Inverse reinforcement learning from summary data
- Fast Bayesian hyperparameter optimization on large datasets
- Gaussian processes for history-matching: application to an unconventional gas reservoir
- Constrained Bayesian optimization with noisy experiments
- Design optimization under uncertainties of a mesoscale implant in biological tissues using a probabilistic learning algorithm
- Expected improvement for expensive optimization: a review
- A taxonomy of weight learning methods for statistical relational learning
- A one-bit, comparison-based gradient estimator
- High-dimensional Bayesian optimization using low-dimensional feature spaces
- Entropy-based closure for probabilistic learning on manifolds
- Efficient global optimization method via clustering/classification methods and exploration strategy
- High-dimensional Bayesian optimization with projections using quantile Gaussian processes
- On the choice of the low-dimensional domain for global optimization via random embeddings
- On two continuum armed bandit problems in high dimensions
- Bound-constrained global optimization of functions with low effective dimensionality using multiple random embeddings
- Active Learning for Enumerating Local Minima Based on Gaussian Process Derivatives
- An analysis of covariance parameters in Gaussian process-based optimization
- Gaussian process-based dimension reduction for goal-oriented sequential design
- Output-Weighted Optimal Sampling for Bayesian Experimental Design and Uncertainty Quantification
- A Supervised Learning Approach Involving Active Subspaces for an Efficient Genetic Algorithm in High-Dimensional Optimization Problems
- Query efficient posterior estimation in scientific experiments via Bayesian active learning
- Sequential Learning of Active Subspaces
- Zeroth-order regularized optimization (ZORO): approximately sparse gradients and adaptive sampling
- Hierarchical invention of theorem proving strategies
- Scalable subspace methods for derivative-free nonlinear least-squares optimization
- Optimization by moving ridge functions: derivative-free optimization for computationally intensive functions
- Gradient and uncertainty enhanced sequential sampling for global fit
- TREGO: a trust-region framework for efficient global optimization
- (Global) optimization: historical notes and recent developments
- Optimization on Manifolds via Graph Gaussian Processes
- Scalable Bayesian optimization with randomized prior networks
- A cooperative approach to efficient global optimization
- Global optimization using random embeddings
- Scalable Bayesian optimization with generalized product of experts
- Probabilistic nonconvex constrained optimization with fixed number of function evaluations
- Taking another step: a simple approach to high-dimensional Bayesian optimization
- An integrated method for simultaneous calibration and parameter selection in computer models
- Sensitivity Prewarping for Local Surrogate Modeling
- Sequential Optimization in Locally Important Dimensions
- Projected Gaussian Markov improvement algorithm for high-dimensional discrete optimization via simulation
- A model aggregation approach for high-dimensional large-scale optimization
- Review of large-scale simulation optimization
- Fully adaptive zeroth-order method for minimizing functions with compressible gradients
- Bayesian optimization
- Unified smoothing approach for best hyperparameter selection problem using a bilevel optimization strategy
- Additive Multi-Index Gaussian Process Modeling, with Application to Multi-Physics Surrogate Modeling of the Quark-Gluon Plasma
- Dice and slice simulation optimization for high-dimensional discrete problems
- Hyperband: a novel bandit-based approach to hyperparameter optimization
- Efficient Active Learning Strategies for Computer Experiments
This page was built for publication: Bayesian optimization in a billion dimensions via random embeddings
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2790144)
