The Poisson transform for unnormalised statistical models

DOI10.1007/S11222-015-9559-4MaRDI QIDQ5963779zbMATH OpenOpenAlexFDO

Authors Simon Barthelmé, Nicolas Chopin

Publication date 23 February 2016

Published in Statistics and Computing (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1406.2839

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Inference from spatial processes (62M30) Point processes (e.g., Poisson, Cox, Hawkes processes) (60G55)

Abstract: Contrary to standard statistical models, unnormalised statistical models only specify the likelihood function up to a constant. While such models are natural and popular, the lack of normalisation makes inference much more difficult. Here we show that inferring the parameters of a unnormalised model on a space $Omega$ can be mapped onto an equivalent problem of estimating the intensity of a Poisson point process on $Omega$. The unnormalised statistical model now specifies an intensity function that does not need to be normalised. Effectively, the normalisation constant may now be inferred as just another parameter, at no loss of information. The result can be extended to cover non-IID models, which includes for example unnormalised models for sequences of graphs (dynamical graphs), or for sequences of binary vectors. As a consequence, we prove that unnormalised parameteric inference in non-IID models can be turned into a semi-parametric estimation problem. Moreover, we show that the noise-contrastive divergence of Gutmann & Hyv"arinen (2012) can be understood as an approximation of the Poisson transform, and extended to non-IID settings. We use our results to fit spatial Markov chain models of eye movements, where the Poisson transform allows us to turn a highly non-standard model into vanilla semi-parametric logistic regression.

Recommendations

• scientific article; zbMATH DE number 6276120
• Statistical inference with unnormalized discrete models and localized homogeneous divergences
• Noise contrastive estimation: asymptotic properties, formal comparison with MC-MLE
• Some extensions of score matching
• Estimation of non-normalized statistical models by score matching

Cites work

• scientific article; zbMATH DE number 227027 (Why is no real title available?)
• scientific article; zbMATH DE number 6276120 (Why is no real title available?)
• An efficient Markov chain Monte Carlo method for distributions with intractable normalising constants
• An efficient learning procedure for deep Boltzmann machines
• Asymptotic Statistics
• Fast Stable Restricted Maximum Likelihood and Marginal Likelihood Estimation of Semiparametric Generalized Linear Models
• Generalized additive models. An introduction with R.
• Justifying and Generalizing Contrastive Divergence
• Logistic regression for spatial Gibbs point processes
• Maximum likelihood estimation for spatial models by Markov chain Monte Carlo stochastic approximation
• On Russian roulette estimates for Bayesian inference with doubly-intractable likelihoods
• Posterior sampling when the normalizing constant is unknown
• Spatial logistic regression and change-of-support in Poisson point processes
• The elements of statistical learning. Data mining, inference, and prediction
• Training Products of Experts by Minimizing Contrastive Divergence

Cited in

(3)

• On the classical choice of variance stabilizing transformations and an application for a Poisson variate
• Likelihood-free inference via classification
• Noise contrastive estimation: asymptotic properties, formal comparison with MC-MLE

Describes a project that uses

Uses Software

• gamair

This page was built for publication: The Poisson transform for unnormalised statistical models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5963779)