Probing the relationship between latent linear dynamical systems and low-rank recurrent neural network models

DOI10.1162/NECO_A_01522zbMATH Open1500.92011arXiv2110.09804OpenAlexW4288069434MaRDI QIDQ5044201FDOQ5044201

Authors: Adrian Valente, Srdjan Ostojic, Jonathan W. Pillow 

Publication date: 24 October 2022

Published in: Neural Computation (Search for Journal in Brave)

Abstract: A large body of work has suggested that neural populations exhibit low-dimensional dynamics during behavior. However, there are a variety of different approaches for modeling low-dimensional neural population activity. One approach involves latent linear dynamical system (LDS) models, in which population activity is described by a projection of low-dimensional latent variables with linear dynamics. A second approach involves low-rank recurrent neural networks (RNNs), in which population activity arises directly from a low-dimensional projection of past activity. Although these two modeling approaches have strong similarities, they arise in different contexts and tend to have different domains of application. Here we examine the precise relationship between latent LDS models and linear low-rank RNNs. When can one model class be converted to the other, and vice versa? We show that latent LDS models can only be converted to RNNs in specific limit cases, due to the non-Markovian property of latent LDS models. Conversely, we show that linear RNNs can be mapped onto LDS models, with latent dimensionality at most twice the rank of the RNN.

Full work available at URL: https://arxiv.org/abs/2110.09804

Recommendations

• Shaping dynamics with multiple populations in low-rank recurrent networks
• Opening the black box: low-dimensional dynamics in high-dimensional recurrent neural networks
• Metric entropy limits on recurrent neural network learning of linear dynamical systems
• Mapping low-dimensional dynamics to high-dimensional neural activity: a derivation of the ring model from the neural engineering framework
• Low-dimensional manifolds support multiplexed integrations in recurrent neural networks

Mathematics Subject Classification ID

Neural networks for/in biological studies, artificial life and related topics (92B20)

Cites Work

• Pattern recognition and machine learning.
• High-Dimensional Statistics
• Estimating a State-Space Model from Point Process Observations
• Modeling of continuous time dynamical systems with input by recurrent neural networks
• Shaping Dynamics With Multiple Populations in Low-Rank Recurrent Networks

This page was built for publication: Probing the relationship between latent linear dynamical systems and low-rank recurrent neural network models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5044201)