Main effects and interactions in mixed and incomplete data frames
From MaRDI portal
(Redirected from Publication:146484)
Abstract: A mixed data frame (MDF) is a table collecting categorical, numerical and count observations. The use of MDF is widespread in statistics and the applications are numerous from abundance data in ecology to recommender systems. In many cases, an MDF exhibits simultaneously main effects, such as row, column or group effects and interactions, for which a low-rank model has often been suggested. Although the literature on low-rank approximations is very substantial, with few exceptions, existing methods do not allow to incorporate main effects and interactions while providing statistical guarantees. The present work fills this gap. We propose an estimation method which allows to recover simultaneously the main effects and the interactions. We show that our method is near optimal under conditions which are met in our targeted applications. We also propose an optimization algorithm which provably converges to an optimal solution. Numerical experiments reveal that our method, mimi, performs well when the main effects are sparse and the interaction matrix has low-rank. We also show that mimi compares favorably to existing methods, in particular when the main effects are significantly large compared to the interactions, and when the proportion of missing entries is large. The method is available as an R package on the Comprehensive R Archive Network.
Recommendations
Cites work
- scientific article; zbMATH DE number 6107964 (Why is no real title available?)
- scientific article; zbMATH DE number 1834445 (Why is no real title available?)
- A Max-Norm Constrained Minimization Approach to 1-Bit Matrix Completion
- A coordinate gradient descent method for nonsmooth separable minimization
- Adaptive multinomial matrix completion
- Flexible low-rank statistical modeling with missing data and side information
- Generalized low rank models
- Imputation of Mixed Data With Multilevel Singular Value Decomposition
- Literature survey on low rank approximation of matrices
- Matrix completion and low-rank SVD via fast alternating least squares
- Matrix completion with covariate information
- Modeling item-item similarities for personalized recommendations on Yahoo! front page
- Multiple factor analysis by example using R
- Poisson Matrix Recovery and Completion
- Rank-Sparsity Incoherence for Matrix Decomposition
- Recovery of Low-Rank Plus Compressed Sparse Matrices With Application to Unveiling Traffic Anomalies
- Robust Matrix Decomposition With Sparse Corruptions
- Robust matrix completion
- Robust principal component analysis?
- Simple structure in component analysis techniques for mixtures of qualitative and quantitative variables
- Spectral regularization algorithms for learning large incomplete matrices
- Statistics for high-dimensional data. Methods, theory and applications.
Cited in
(7)- Generalized Low-Rank Plus Sparse Tensor Estimation by Fast Riemannian Optimization
- Matrix completion under complex survey sampling
- An adaptation for iterative structured matrix completion
- Imputation and low-rank estimation with missing not at random data
- Safety signal detection with control of latent factors
- mimi
- Detecting arrays for main effects
This page was built for publication: Main effects and interactions in mixed and incomplete data frames
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q146484)
