Tutorial on Amortized Optimization
From MaRDI portal
Publication:6139544
Numerical optimization and variational techniques (65K10) Learning and adaptive systems in artificial intelligence (68T05) Research exposition (monographs, survey articles) pertaining to computer science (68-02) Artificial neural networks and deep learning (68T07) Research exposition (monographs, survey articles) pertaining to numerical analysis (65-02)
Abstract: Optimization is a ubiquitous modeling tool and is often deployed in settings which repeatedly solve similar instances of the same problem. Amortized optimization methods use learning to predict the solutions to problems in these settings, exploiting the shared structure between similar problem instances. These methods have been crucial in variational inference and reinforcement learning and are capable of solving optimization problems many orders of magnitudes times faster than traditional optimization methods that do not use amortization. This tutorial presents an introduction to the amortized optimization foundations behind these advancements and overviews their applications in variational inference, sparse coding, gradient-based meta-learning, control, reinforcement learning, convex optimization, optimal transport, and deep equilibrium networks. The source code for this tutorial is available at https://github.com/facebookresearch/amortized-optimization-tutorial.
Recommendations
Cites work
- scientific article; zbMATH DE number 6377992 (Why is no real title available?)
- scientific article; zbMATH DE number 5957269 (Why is no real title available?)
- scientific article; zbMATH DE number 3791104 (Why is no real title available?)
- scientific article; zbMATH DE number 1113195 (Why is no real title available?)
- scientific article; zbMATH DE number 1843025 (Why is no real title available?)
- scientific article; zbMATH DE number 2107836 (Why is no real title available?)
- scientific article; zbMATH DE number 7255066 (Why is no real title available?)
- scientific article; zbMATH DE number 7626805 (Why is no real title available?)
- scientific article; zbMATH DE number 5223994 (Why is no real title available?)
- scientific article; zbMATH DE number 5060482 (Why is no real title available?)
- A Class of Methods for Solving Nonlinear Simultaneous Equations
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
- A Flexible and Efficient Algorithmic Framework for Constrained Matrix and Tensor Factorization
- A brief introduction to manifold optimization
- A tutorial on the cross-entropy method
- Accelerated optimization on Riemannian manifolds via discrete constrained variational integrators
- An Introduction to Variational Autoencoders
- An introduction to variational methods for graphical models
- An iterative thresholding algorithm for linear inverse problems with a sparsity constraint
- Anderson acceleration for fixed-point iterations
- Atomic decomposition by basis pursuit
- CVXPY: a Python-embedded modeling language for convex optimization
- Conic optimization via operator splitting and homogeneous self-dual embedding
- Convex optimization: algorithms and complexity
- Data-driven nonsmooth optimization
- Distributed optimization and statistical learning via the alternating direction method of multipliers
- Elements of Information Theory
- End-to-end training of deep visuomotor policies
- Envelope Theorems for Arbitrary Choice Sets
- Globally Convergent Type-I Anderson Acceleration for Nonsmooth Fixed-Point Iterations
- Graphical models, exponential families, and variational inference
- Implicit Functions and Solution Mappings
- Intrinsic statistics on Riemannian manifolds: Basic tools for geometric measurements
- Iterative Procedures for Nonlinear Integral Equations
- JuMP: a modeling language for mathematical optimization
- Julia: a fresh approach to numerical computing
- Large scale model predictive control with neural networks and primal active sets
- Learning to predict the cosmological structure formation
- Lectures on convex optimization
- Machine learning for combinatorial optimization: a methodological tour d'horizon
- Machine learning. A probabilistic perspective
- Mathematics for Machine Learning
- Modern homotopy methods in optimization
- Monte Carlo gradient estimation in machine learning
- OSQP: an operator splitting solver for quadratic programs
- On learning and branching: a survey
- OnActor-Critic Algorithms
- Online Mixed-Integer Optimization in Milliseconds
- Optimal transport for applied mathematicians. Calculus of variations, PDEs, and modeling
- Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ 1 minimization
- Reinforcement learning. An introduction
- Sensitivity analysis of generalized equations
- Sensitivity and stability analysis for nonlinear programming
- Solution refinement at regular points of conic problems
- Solving and learning nonlinear PDEs with Gaussian processes
- The Theory of Max-Min, with Applications
- The principle of maximum entropy
- The voice of optimization
- \({\mathcal Q}\)-learning
This page was built for publication: Tutorial on Amortized Optimization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6139544)
