Optimization in machine learning: a distribution-space approach
This paper presents a novel perspective on optimization problems in machine learning by framing them as convex functional minimization over a function space, subject to non-convex constraints introduced by model parameterization. The authors propose a distribution-space approach that transforms the original non-convex optimization problem into a convex one by considering the space of probability distributions over the training parameters. This reformulation allows for the development of numerical algorithms based on mixture distributions, which can perform approximate optimization directly in the distribution space. The paper not only establishes the consistency of this approximation but also demonstrates the numerical efficacy of the proposed algorithm through simple examples, highlighting its potential as an alternative approach to large-scale optimization in machine learning.
- scientific article; zbMATH DE number 1786133
- Optimization problems for machine learning: a survey
- The interplay of optimization and machine learning research
- From inexact optimization to learning via gradient concentration
- Distributed optimization and statistical learning via the alternating direction method of multipliers
- A mean field view of the landscape of two-layer neural networks
- Approximation by superpositions of a sigmoidal function
- Approximation of probability distributions by convex mixtures of Gaussian measures
- Bagging predictors
- Breaking the curse of dimensionality with convex neural networks
- Density estimation through convex combinations of densities: Approximation and estimation bounds
- scientific article; zbMATH DE number 6378127 (Why is no real title available?)
- scientific article; zbMATH DE number 931614 (Why is no real title available?)
- scientific article; zbMATH DE number 7415108 (Why is no real title available?)
- Langevin equation for the density of a system of interacting Langevin processes
- Mean field analysis of neural networks: a law of large numbers
- Multilayer feedforward networks are universal approximators
- Non-convex optimization for machine learning
- Recursive Bayesian estimation using Gaussian sums
- Universal approximation bounds for superpositions of a sigmoidal function
- Universal series induced by approximate identities and some relevant applications
This page was built for publication: Optimization in machine learning: a distribution-space approach
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6575304)
