Bias of homotopic gradient descent for the hinge loss
From MaRDI portal
Publication:2045131
Abstract: Gradient descent is a simple and widely used optimization method for machine learning. For homogeneous linear classifiers applied to separable data, gradient descent has been shown to converge to the maximal margin (or equivalently, the minimal norm) solution for various smooth loss functions. The previous theory does not, however, apply to non-smooth functions such as the hinge loss which is widely used in practice. Here, we study the convergence of a homotopic variant of gradient descent applied to the hinge loss and provide explicit convergence rates to the max-margin solution for linearly separable data.
Recommendations
Cites work
- Convex optimization: algorithms and complexity
- Entropy-SGD: biasing gradient descent into wide valleys
- Estimation of dependences based on empirical data. Transl. from the Russian by Samuel Kotz
- scientific article; zbMATH DE number 5957245 (Why is no real title available?)
- scientific article; zbMATH DE number 3446442 (Why is no real title available?)
- Learning ReLU Networks on Linearly Separable Data: Algorithm, Optimality, and Generalization
- Optimization methods for large-scale machine learning
- Support-vector networks
- The implicit bias of gradient descent on separable data
- The nature of statistical learning theory.
- Towards a deeper geometric, analytic and algorithmic understanding of margins
- Training a Support Vector Machine in the Primal
Cited in
(5)
This page was built for publication: Bias of homotopic gradient descent for the hinge loss
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2045131)