Variational Gram functions: convex analysis and optimization
From MaRDI portal
Abstract: We propose a new class of convex penalty functions, called emph{variational Gram functions} (VGFs), that can promote pairwise relations, such as orthogonality, among a set of vectors in a vector space. These functions can serve as regularizers in convex optimization problems arising from hierarchical classification, multitask learning, and estimating vectors with disjoint supports, among other applications. We study convexity for VGFs, and give efficient characterizations for their convex conjugates, subdifferentials, and proximal operators. We discuss efficient optimization algorithms for regularized loss minimization problems where the loss admits a common, yet simple, variational representation and the regularizer is a VGF. These algorithms enjoy a simple kernel trick, an efficient line search, as well as computational advantages over first order methods based on the subdifferential or proximal maps. We also establish a general representer theorem for such learning problems. Lastly, numerical experiments on a hierarchical classification problem are presented to demonstrate the effectiveness of VGFs and the associated optimization algorithms.
Recommendations
- Variational properties of matrix functions via the generalized matrix-fractional function
- On proximal gradient method for the convex problems regularized with the group reproducing kernel norm
- Variational properties of value functions
- Convex geometry of the generalized matrix-fractional function
- Regularizers for structured sparsity
Cites work
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
- A note on the positive semidefinite minimum rank of a sign pattern matrix
- A trace inequality of John von Neumann
- Breakpoint searching algorithms for the continuous quadratic knapsack problem
- Conditions for Positive and Nonnegative Definiteness in Terms of Pseudoinverses
- Convex Analysis
- Convex analysis and monotone operator theory in Hilbert spaces
- Convex geometry of the generalized matrix-fractional function
- Determinantal point processes for machine learning
- Finding the largest low-rank clusters with Ky Fan 2-k-norm and _1-norm
- Gradient methods for minimizing composite functions
- Gradient-based algorithms with applications to signal-recovery problems
- Handbook of semidefinite programming. Theory, algorithms, and applications
- scientific article; zbMATH DE number 1804115 (Why is no real title available?)
- scientific article; zbMATH DE number 3534286 (Why is no real title available?)
- scientific article; zbMATH DE number 1089164 (Why is no real title available?)
- scientific article; zbMATH DE number 2107836 (Why is no real title available?)
- scientific article; zbMATH DE number 823379 (Why is no real title available?)
- scientific article; zbMATH DE number 6125590 (Why is no real title available?)
- scientific article; zbMATH DE number 3298486 (Why is no real title available?)
- scientific article; zbMATH DE number 3341597 (Why is no real title available?)
- Low rank multivariate regression
- Monotone Operators and the Proximal Point Algorithm
- New perspectives on k-support and cluster norms
- Optimization with sparsity-inducing penalties
- Prox-Method with Rate of Convergence O(1/t) for Variational Inequalities with Lipschitz Continuous Monotone Operators and Smooth Convex-Concave Saddle Point Problems
- Regularizers for structured sparsity
- Robust optimization
- Semidefinite Programming
- Solving variational inequalities with monotone operators on domains given by linear minimization oracles
- Some Systematic Supersaturated Designs
- Variational Analysis
Cited in
(3)
This page was built for publication: Variational Gram functions: convex analysis and optimization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4602347)
