Stochastic Scale Invariant Power Iteration for KL-divergence Nonnegative Matrix Factorization

arXiv2304.11268MaRDI QIDQ6433933FDOQ6433933

Authors: Cheolmin Kim, Youngseok Kim, Diego Klabjan 

Publication date: 21 April 2023

Abstract: We introduce a mini-batch stochastic variance-reduced algorithm to solve finite-sum scale invariant problems which cover several examples in machine learning and statistics such as principal component analysis (PCA) and estimation of mixture proportions. The algorithm is a stochastic generalization of scale invariant power iteration, specializing to power iteration when full-batch is used for the PCA problem. In convergence analysis, we show the expectation of the optimality gap decreases at a linear rate under some conditions on the step size, epoch length, batch size and initial iterate. Numerical experiments on the non-negative factorization problem with the Kullback-Leibler divergence using real and synthetic datasets demonstrate that the proposed stochastic approach not only converges faster than state-of-the-art deterministic algorithms but also produces excellent quality robust solutions.

This page was built for publication: Stochastic Scale Invariant Power Iteration for KL-divergence Nonnegative Matrix Factorization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6433933)