Monte Carlo Information-Geometric Structures

DOI10.1007/978-3-030-02520-5_5MaRDI QIDQ4967756zbMATH OpenOpenAlexFDO

Authors Gaëtan Hadjeres, Frank Nielsen

Publication date 10 July 2019

Published in Geometric Structures of Information (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1803.07225

zbMATH Keywords

Bregman generatorclusteringcomputational information geometrydually flat information geometryexponential familymixture familystatistical manifoldstochastic Monte Carlo integration

Mathematics Subject Classification ID

Probability distributions: general theory (60E05) Statistical aspects of information-theoretic topics (62B10) Monte Carlo methods (65C05) Measures of information, entropy (94A17)

Abstract: Exponential families and mixture families are parametric probability models that can be geometrically studied as smooth statistical manifolds with respect to any statistical divergence like the Kullback-Leibler (KL) divergence or the Hellinger divergence. When equipping a statistical manifold with the KL divergence, the induced manifold structure is dually flat, and the KL divergence between distributions amounts to an equivalent Bregman divergence on their corresponding parameters. In practice, the corresponding Bregman generators of mixture/exponential families require to perform definite integral calculus that can either be too time-consuming (for exponentially large discrete support case) or even do not admit closed-form formula (for continuous support case). In these cases, the dually flat construction remains theoretical and cannot be used by information-geometric algorithms. To bypass this problem, we consider performing stochastic Monte Carlo (MC) estimation of those integral-based mixture/exponential family Bregman generators. We show that, under natural assumptions, these MC generators are almost surely Bregman generators. We define a series of dually flat information geometries, termed Monte Carlo Information Geometries, that increasingly-finely approximate the untractable geometry. The advantage of this MCIG is that it allows a practical use of the Bregman algorithmic toolbox on a wide range of probability distribution families. We demonstrate our approach with a clustering task on a mixture family manifold.

Recommendations

• Information-geometric Markov chain Monte Carlo methods using diffusions
• Geometrical Monte Carlo method and its modifications
• Information geometry, simulation and complexity in Gaussian random fields
• Information geometry
• Information geometry
• scientific article; zbMATH DE number 1023413
• Information geometry
• Information Geometry
• Information geometric methods for complexity

Cites work

• \(k\)-means clustering with Hölder divergences
• A relationship between the second derivatives of a convex function and of its conjugate
• A sequential structure of statistical manifolds on deformed exponential family
• A Student's Guide to Vectors and Tensors
• Bregman Voronoi diagrams
• Clustering with Bregman divergences.
• Estimation and Moment Recursion Relations for Multimodal Distributions of the Exponential Family
• Geometric Modeling in Probability and Statistics
• Geometry of minimum contrast
• scientific article; zbMATH DE number 4088961 (Why is no real title available?)
• scientific article; zbMATH DE number 44578 (Why is no real title available?)
• scientific article; zbMATH DE number 107482 (Why is no real title available?)
• scientific article; zbMATH DE number 1055955 (Why is no real title available?)
• scientific article; zbMATH DE number 2107836 (Why is no real title available?)
• scientific article; zbMATH DE number 2117879 (Why is no real title available?)
• scientific article; zbMATH DE number 2120941 (Why is no real title available?)
• Hypothesis testing, information divergence and computational geometry
• Inference in Ising models
• Information geometry and its applications
• Informative barycentres in statistics
• On Bregman Voronoi diagrams
• On the smallest enclosing information disk
• Reference duality and representation duality in information geometry
• Relative loss bounds for on-line density estimation with the exponential family of distributions
• Second order efficiency of minimum contrast estimators in a curved exponential family
• Sided and Symmetrized Bregman Centroids
• Spontaneous clustering via minimum gamma-divergence
• Statistical Manifolds of Univariate Elliptic Distributions
• Symbolic computation with monotone operators
• The Burbea-Rao and Bhattacharyya Centroids
• The geometry of Hessian structures
• The geometry of proper scoring rules

Cited in

(2)

• The analytic dually flat space of the mixture family of two prescribed distinct Cauchy distributions
• Feature-preserving point cloud filtering via mixture family manifold

This page was built for publication: Monte Carlo Information-Geometric Structures

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4967756)