Computation of the maximum likelihood estimator in low-rank factor analysis

DOI10.1007/S10107-019-01370-7zbMATH Open1415.90074arXiv1801.05935OpenAlexW2964108664WikidataQ128287820 ScholiaQ128287820MaRDI QIDQ2425172FDOQ2425172

Authors: Koulik Khamaru, Rahul Mazumder 

Publication date: 26 June 2019

Published in: Mathematical Programming. Series A. Series B (Search for Journal in Brave)

Abstract: Factor analysis, a classical multivariate statistical technique is popularly used as a fundamental tool for dimensionality reduction in statistics, econometrics and data science. Estimation is often carried out via the Maximum Likelihood (ML) principle, which seeks to maximize the likelihood under the assumption that the positive definite covariance matrix can be decomposed as the sum of a low rank positive semidefinite matrix and a diagonal matrix with nonnegative entries. This leads to a challenging rank constrained nonconvex optimization problem. We reformulate the low rank ML Factor Analysis problem as a nonlinear nonsmooth semidefinite optimization problem, study various structural properties of this reformulation and propose fast and scalable algorithms based on difference of convex (DC) optimization. Our approach has computational guarantees, gracefully scales to large problems, is applicable to situations where the sample covariance matrix is rank deficient and adapts to variants of the ML problem with additional constraints on the problem parameters. Our numerical experiments demonstrate the significant usefulness of our approach over existing state-of-the-art approaches.

Full work available at URL: https://arxiv.org/abs/1801.05935

Recommendations

• Certifiably optimal low rank factor analysis
• Maximum likelihood factor analysis with rank-deficient sample covariance matrices
• An alternating minimization algorithm for factor analysis.
• On maximum likelihood estimation in factor analysis-an algebraic derivation
• Maximum likelihood for matrices with rank constraints

zbMATH Keywords

sparsitysemidefinite optimizationdifference of convex optimizationspectral functionsmaximum likelihood factor analysislow-rank constraint

Mathematics Subject Classification ID

Factor analysis and principal components; correspondence analysis (62H25) Convex programming (90C25) Nonlinear programming (90C30) Semidefinite programming (90C22) Computational methods for problems pertaining to operations research and mathematical programming (90-08)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Latent variable models and factor analysis. A unified approach
• Conic optimization via operator splitting and homogeneous self-dual embedding
• Variational Analysis
• The Concave-Convex Procedure
• Title not available (Why is that?)
• Pattern recognition and machine learning.
• Sparse inverse covariance estimation with the graphical lasso
• Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Statistical analysis of factor models of high dimension
• EM algorithms for ML factor analysis
• Some contributions to maximum likelihood factor analysis
• Title not available (Why is that?)
• Fast low-rank modifications of the thin singular value decomposition
• Diagonal and low-rank matrix decompositions, correlation matrices, and ellipsoid fitting
• Maximum likelihood factor analysis with rank-deficient sample covariance matrices
• Title not available (Why is that?)
• Derivatives of Spectral Functions
• Convex analysis and nonlinear optimization. Theory and examples.
• Convex Analysis on the Hermitian Matrices
• Statistical inference of minimum rank factor analysis
• All convex invariant functions of hermitian matrices
• Computing B-stationary points of nonsmooth DC programs
• On the pervasiveness of difference-convexity in optimization and statistics
• Difference-of-convex learning: directional stationarity, optimality, and sparsity
• Certifiably optimal low rank factor analysis

Cited In (5)

• Title not available (Why is that?)
• Maximum likelihood factor analysis with rank-deficient sample covariance matrices
• Certifiably optimal low rank factor analysis
• A Matrix-Free Likelihood Method for Exploratory Factor Analysis of High-Dimensional Gaussian Data
• Interpreting latent variables in factor models via convex optimization

Uses Software

• PROPACK
• glasso
• ElemStatLearn
• SCS
• PRMLT

This page was built for publication: Computation of the maximum likelihood estimator in low-rank factor analysis

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2425172)