A Comparative Study of Pairwise Learning Methods Based on Kernel Ridge Regression
From MaRDI portal
Abstract: Many machine learning problems can be formulated as predicting labels for a pair of objects. Problems of that kind are often referred to as pairwise learning, dyadic prediction or network inference problems. During the last decade kernel methods have played a dominant role in pairwise learning. They still obtain a state-of-the-art predictive performance, but a theoretical analysis of their behavior has been underexplored in the machine learning literature. In this work we review and unify existing kernel-based algorithms that are commonly used in different pairwise learning settings, ranging from matrix filtering to zero-shot learning. To this end, we focus on closed-form efficient instantiations of Kronecker kernel ridge regression. We show that independent task kernel ridge regression, two-step kernel ridge regression and a linear matrix filter arise naturally as a special case of Kronecker kernel ridge regression, implying that all these methods implicitly minimize a squared loss. In addition, we analyze universality, consistency and spectral filtering properties. Our theoretical results provide valuable insights in assessing the advantages and limitations of existing pairwise learning methods.
Recommendations
- On the robustness of regularized pairwise learning methods based on kernels
- Advances in Neural Networks – ISNN 2005
- Error analysis of kernel regularized pairwise learning with a strongly convex loss
- Generalized vec trick for fast learning of pairwise kernel models
- scientific article; zbMATH DE number 7295804
- Kernel ridge regression
- Kernel ridge vs. principal component regression: minimax bounds and the qualification of regularization operators
Cites work
- A new approach to collaborative filtering: operator estimation with spectral regularization
- An alternative two stage least squares (2SLS) estimator for latent variable equations
- Efficient regularized least-squares algorithms for conditional ranking on relational data
- Exact matrix completion via convex optimization
- Graph-Based Semi-Supervised Learning and Spectral Kernel Design
- scientific article; zbMATH DE number 1194487 (Why is no real title available?)
- scientific article; zbMATH DE number 45848 (Why is no real title available?)
- Kernels for vector-valued functions: a review
- Learning intransitive reciprocal relations with kernel methods
- Low-rank matrix completion using alternating minimization
- Machine Learning: ECML 2004
- Multi-output learning via spectral filtering
- On regularization algorithms in learning theory
- On spectral learning
- On the influence of the kernel on the consistency of support vector machines
- Pairwise support vector machines and their application to large scale problems
- Pattern recognition and machine learning.
- Randomized Algorithms for Matrices and Data
- Shifted Kronecker Product Systems
- Spectral Algorithms for Supervised Learning
- Spectral regularization algorithms for learning large incomplete matrices
- The elements of statistical learning. Data mining, inference, and prediction
- The ubiquitous Kronecker product
- Value regularization and Fenchel duality
Cited in
(4)
This page was built for publication: A Comparative Study of Pairwise Learning Methods Based on Kernel Ridge Regression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5157226)
