Eigen-stratified models

DOI10.1007/S11081-020-09592-XzbMATH Open1492.90131arXiv2001.10389OpenAlexW3120254229MaRDI QIDQ2138309FDOQ2138309

Authors: Yanyan Li 

Publication date: 11 May 2022

Published in: Optimization and Engineering (Search for Journal in Brave)

Abstract: Stratified models depend in an arbitrary way on a selected categorical feature that takes $K$ values, and depend linearly on the other $n$ features. Laplacian regularization with respect to a graph on the feature values can greatly improve the performance of a stratified model, especially in the low-data regime. A significant issue with Laplacian-regularized stratified models is that the model is $K$ times the size of the base model, which can be quite large. We address this issue by formulating eigen-stratifed models, which are stratified models with an additional constraint that the model parameters are linear combinations of some modest number $m$ of bottom eigenvectors of the graph Laplacian, i.e., those associated with the $m$ smallest eigenvalues. With eigen-stratified models, we only need to store the $m$ bottom eigenvectors and the corresponding coefficients as the stratified model parameters. This leads to a reduction, sometimes large, of model size when $mleqn$ and $mllK$. In some cases, the additional regularization implicit in eigen-stratified models can improve out-of-sample performance over standard Laplacian regularized stratified models.

Full work available at URL: https://arxiv.org/abs/2001.10389

Recommendations

• Stratified Gaussian graphical models
• Integrative Analysis for High-Dimensional Stratified Models
• Models of random sparse eigenmatrices and Bayesian analysis of multivariate structure
• Stratified exponential families: Graphical models and model selection
• Fitting Laplacian regularized stratified Gaussian models
• A stochastic version of the Eigen model
• Eigenvalues and structure of compartmental models
• Estimating multivariate latent-structure models

zbMATH Keywords

optimizationdata sciencemachine learningconvex optimization

Mathematics Subject Classification ID

Convex programming (90C25)

Cites Work

• Disciplined convex programming
• CVXPY: a Python-embedded modeling language for convex optimization
• Title not available (Why is that?)
• Distributed optimization and statistical learning via the alternating direction method of multipliers
• Title not available (Why is that?)
• Eigenvalues of the Laplacian of a graph^∗
• Spectra of graphs
• Title not available (Why is that?)
• Laplacian graph eigenvectors
• The Fastest Mixing Markov Process on a Graph and a Connection to a Maximum Variance Unfolding Problem
• New methods to color the vertices of a graph
• Which wheel graphs are determined by their Laplacian spectra?
• Discrete Cosine Transform
• Fastest mixing Markov chain on graphs with symmetries
• Fastest Mixing Markov Chain on a Graph
• Algorithms, graph theory, and linear equations in Laplacian matrices
• Graph Coloring Using Eigenvalue Decomposition
• Vibration modes of large structures by an automatic matrix-reductionmethod
• Title not available (Why is that?)
• Anderson Accelerated Douglas--Rachford Splitting
• Title not available (Why is that?)

Uses Software

• CVX
• CVXPY
• word2vec
• CVXR

This page was built for publication: Eigen-stratified models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2138309)