Eigen-stratified models

DOI10.1007/S11081-020-09592-XMaRDI QIDQ2138309zbMATH OpenOpenAlexFDO

Authors Yanyan Li

Publication date 11 May 2022

Published in Optimization and Engineering (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2001.10389

zbMATH Keywords

optimizationdata sciencemachine learningconvex optimization

Mathematics Subject Classification ID

Convex programming (90C25)

Abstract: Stratified models depend in an arbitrary way on a selected categorical feature that takes $K$ values, and depend linearly on the other $n$ features. Laplacian regularization with respect to a graph on the feature values can greatly improve the performance of a stratified model, especially in the low-data regime. A significant issue with Laplacian-regularized stratified models is that the model is $K$ times the size of the base model, which can be quite large. We address this issue by formulating eigen-stratifed models, which are stratified models with an additional constraint that the model parameters are linear combinations of some modest number $m$ of bottom eigenvectors of the graph Laplacian, i.e., those associated with the $m$ smallest eigenvalues. With eigen-stratified models, we only need to store the $m$ bottom eigenvectors and the corresponding coefficients as the stratified model parameters. This leads to a reduction, sometimes large, of model size when $mleqn$ and $mllK$. In some cases, the additional regularization implicit in eigen-stratified models can improve out-of-sample performance over standard Laplacian regularized stratified models.

Recommendations

• Stratified Gaussian graphical models
• Integrative Analysis for High-Dimensional Stratified Models
• Models of random sparse eigenmatrices and Bayesian analysis of multivariate structure
• Stratified exponential families: Graphical models and model selection
• Fitting Laplacian regularized stratified Gaussian models
• A stochastic version of the Eigen model
• Eigenvalues and structure of compartmental models
• Estimating multivariate latent-structure models

Cites work

• scientific article; zbMATH DE number 43706 (Why is no real title available?)
• scientific article; zbMATH DE number 2107836 (Why is no real title available?)
• scientific article; zbMATH DE number 7370577 (Why is no real title available?)
• scientific article; zbMATH DE number 964896 (Why is no real title available?)
• scientific article; zbMATH DE number 3070747 (Why is no real title available?)
• Algorithms, graph theory, and linear equations in Laplacian matrices
• Anderson Accelerated Douglas--Rachford Splitting
• CVXPY: a Python-embedded modeling language for convex optimization
• Disciplined convex programming
• Discrete Cosine Transform
• Distributed optimization and statistical learning via the alternating direction method of multipliers
• Eigenvalues of the Laplacian of a graph^∗
• Fastest Mixing Markov Chain on a Graph
• Fastest mixing Markov chain on graphs with symmetries
• Graph Coloring Using Eigenvalue Decomposition
• Laplacian graph eigenvectors
• New methods to color the vertices of a graph
• Spectra of graphs
• The Fastest Mixing Markov Process on a Graph and a Connection to a Maximum Variance Unfolding Problem
• Vibration modes of large structures by an automatic matrix-reductionmethod
• Which wheel graphs are determined by their Laplacian spectra?

Describes a project that uses

Uses Software

• CVX
• CVXPY
• word2vec
• CVXR

This page was built for publication: Eigen-stratified models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2138309)