A Riemannian gossip approach to subspace learning on Grassmann manifold

DOI10.1007/S10994-018-05775-XzbMATH Open1493.68309arXiv1705.00467OpenAlexW2790379190WikidataQ115381966 ScholiaQ115381966MaRDI QIDQ2320541FDOQ2320541

Authors: Bamdev Mishra, Hiroyuki Kasai, Pratik Jawanpuria, Atul Saroop 

Publication date: 23 August 2019

Published in: Machine Learning (Search for Journal in Brave)

Abstract: In this paper, we focus on subspace learning problems on the Grassmann manifold. Interesting applications in this setting include low-rank matrix completion and low-dimensional multivariate regression, among others. Motivated by privacy concerns, we aim to solve such problems in a decentralized setting where multiple agents have access to (and solve) only a part of the whole optimization problem. The agents communicate with each other to arrive at a consensus, i.e., agree on a common quantity, via the gossip protocol. We propose a novel cost function for subspace learning on the Grassmann manifold, which is a weighted sum of several sub-problems (each solved by an agent) and the communication cost among the agents. The cost function has a finite sum structure. In the proposed modeling approach, different agents learn individual local subspace but they achieve asymptotic consensus on the global learned subspace. The approach is scalable and parallelizable. Numerical experiments show the efficacy of the proposed decentralized algorithms on various matrix completion and multivariate regression benchmarks.

Full work available at URL: https://arxiv.org/abs/1705.00467

Recommendations

• Riemannian geometry of Grassmann manifolds with a view on algorithmic computation
• Dictionary Learning on Grassmann Manifolds
• Local convergence of an algorithm for subspace identification from partial data
• Low-rank matrix completion by Riemannian optimization
• Decentralized and privacy-preserving low-rank matrix completion

zbMATH Keywords

multivariate regressionmatrix completionmanifold optimizationstochastic gradientsnon-linear gossip

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Estimation in multivariate analysis (62H12) Statistics on manifolds (62R30) Stochastic programming (90C15) Grassmannians, Schubert varieties, flag manifolds (14M15)

Cites Work

• Manopt, a Matlab toolbox for optimization on manifolds
• Low-rank matrix completion via preconditioned optimization on the Grassmann manifold
• Model Selection and Estimation in Regression with Grouped Variables
• Pattern recognition and machine learning.
• Kernels for vector-valued functions: a review
• A Singular Value Thresholding Algorithm for Matrix Completion
• The Geometry of Algorithms with Orthogonality Constraints
• Title not available (Why is that?)
• Exact matrix completion via convex optimization
• Parallel stochastic gradient algorithms for large-scale matrix completion
• Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm
• Consensus optimization on manifolds
• Matrix Completion From a Few Entries
• Title not available (Why is that?)
• Title not available (Why is that?)
• Convex multi-task feature learning
• Subspace Evolution and Transfer (SET) for Low-Rank Matrix Completion
• Structured low-rank approximation with missing data
• Title not available (Why is that?)
• A new approach to collaborative filtering: operator estimation with spectral regularization
• Stochastic Gradient Descent on Riemannian Manifolds
• Riemannian Consensus for Manifolds With Bounded Curvature
• Gossip algorithms
• A Geometric Approach to Low-Rank Matrix Completion
• Regression on fixed-rank positive semidefinite matrices: a Riemannian approach
• Flexible latent variable models for multi-task learning
• Fixed-rank matrix factorizations and Riemannian low-rank optimization
• Learning multiple tasks with kernel methods
• Dictionary Learning on Grassmann Manifolds
• A Bayesian/information theoretic model of learning to learn via multiple task sampling
• Riemannian Stochastic Variance Reduced Gradient Algorithm with Retraction and Vector Transport
• Decentralized and privacy-preserving low-rank matrix completion
• Learning using privileged information: SVM+ and weighted SVM

Cited In (4)

• Riemannian Natural Gradient Methods
• Variational Bayes on manifolds
• Recent Advances in Stochastic Riemannian Optimization
• Faster Riemannian Newton-type optimization by subsampling and cubic regularization

Uses Software

• UCI-ml
• Manopt
• Jellyfish
• RTRMC
• PRMLT

This page was built for publication: A Riemannian gossip approach to subspace learning on Grassmann manifold

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2320541)