Parametric Inference in the Large Data Limit Using Maximally Informative Models

DOI10.1162/NECO_A_00568zbMATH Open1418.62362arXiv1212.3647OpenAlexW2018553805WikidataQ30746132 ScholiaQ30746132MaRDI QIDQ5378337FDOQ5378337

Gurinder S. Atwal, Justin B. Kinney

Publication date: 12 June 2019

Published in: Neural Computation (Search for Journal in Brave)

Abstract: Motivated by data-rich experiments in transcriptional regulation and sensory neuroscience, we consider the following general problem in statistical inference. When exposed to a high-dimensional signal S, a system of interest computes a representation R of that signal which is then observed through a noisy measurement M. From a large number of signals and measurements, we wish to infer the "filter" that maps S to R. However, the standard method for solving such problems, likelihood-based inference, requires perfect a priori knowledge of the "noise function" mapping R to M. In practice such noise functions are usually known only approximately, if at all, and using an incorrect noise function will typically bias the inferred filter. Here we show that, in the large data limit, this need for a pre-characterized noise function can be circumvented by searching for filters that instead maximize the mutual information I[M;R] between observed measurements and predicted representations. Moreover, if the correct filter lies within the space of filters being explored, maximizing mutual information becomes equivalent to simultaneously maximizing every dependence measure that satisfies the Data Processing Inequality. It is important to note that maximizing mutual information will typically leave a small number of directions in parameter space unconstrained. We term these directions "diffeomorphic modes" and present an equation that allows these modes to be derived systematically. The presence of diffeomorphic modes reflects a fundamental and nontrivial substructure within parameter space, one that is obscured by standard likelihood-based inference.

Full work available at URL: https://arxiv.org/abs/1212.3647

zbMATH Keywords

statistical inferenceparameter spacecorrect filterlarge data limit

Mathematics Subject Classification ID

Asymptotic properties of parametric estimators (62F12) Applications of statistics to biology and medical sciences; meta analysis (62P10) Inference from stochastic processes and prediction (62M20)

Cites Work

• Title not available (Why is that?)
• Equitability, mutual information, and the maximal information coefficient
• Title not available (Why is that?)
• Title not available (Why is that?)
• Information Theory and Statistics: A Tutorial
• Field Theories for Learning Probability Distributions
• Analyzing Neural Responses to Natural Signals: Maximally Informative Dimensions
• Learning Quadratic Receptive Fields from Neural Responses to Natural Stimuli

Cited In (2)

• Learning quantitative sequence-function relationships from massively parallel experiments
• Bayesian Feature Selection with Strongly Regularizing Priors Maps to the Ising Model

This page was built for publication: Parametric Inference in the Large Data Limit Using Maximally Informative Models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5378337)