Generalized twin Gaussian processes using Sharma-Mittal divergence

DOI10.1007/S10994-015-5497-9MaRDI QIDQ747261zbMATH OpenOpenAlexFDO

Authors Mohamed Elhoseiny, Ahmed Elgammal

Publication date 23 October 2015

Published in Machine Learning (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1409.7480

zbMATH Keywords

image reconstructionpose estimationstructured regressionSharma-Mittal entropytwin Gaussian processes

Mathematics Subject Classification ID

Statistical aspects of information-theoretic topics (62B10) General nonlinear regression (62J02) Measures of information, entropy (94A17) Computing methodologies for image processing (68U10) Machine vision and scene understanding (68T45)

Abstract: There has been a growing interest in mutual information measures due to their wide range of applications in Machine Learning and Computer Vision. In this paper, we present a generalized structured regression framework based on Shama-Mittal divergence, a relative entropy measure, which is introduced to the Machine Learning community in this work. Sharma-Mittal (SM) divergence is a generalized mutual information measure for the widely used R'enyi, Tsallis, Bhattacharyya, and Kullback-Leibler (KL) relative entropies. Specifically, we study Sharma-Mittal divergence as a cost function in the context of the Twin Gaussian Processes (TGP)~citep{Bo:2010}, which generalizes over the KL-divergence without computational penalty. We show interesting properties of Sharma-Mittal TGP (SMTGP) through a theoretical analysis, which covers missing insights in the traditional TGP formulation. However, we generalize this theory based on SM-divergence instead of KL-divergence which is a special case. Experimentally, we evaluated the proposed SMTGP framework on several datasets. The results show that SMTGP reaches better predictions than KL-based TGP, since it offers a bigger class of models through its parameters that we learn from the data.

Recommendations

• Machine learning with squared-loss mutual information
• Novel generalized divergence measures and multimodal medical image registration
• Information measures of kernel estimation
• A closed-form expression for the Sharma-Mittal entropy of exponential families
• scientific article; zbMATH DE number 2218582

Cites work

• scientific article; zbMATH DE number 48436 (Why is no real title available?)
• scientific article; zbMATH DE number 1012640 (Why is no real title available?)
• scientific article; zbMATH DE number 1560711 (Why is no real title available?)
• scientific article; zbMATH DE number 3187999 (Why is no real title available?)
• 10.1162/jmlr.2003.4.7-8.1271
• A Generalized Divergence Measure for Nonnegative Matrix Factorization
• A closed-form expression for the Sharma-Mittal entropy of exponential families
• A new family of \(k\)-Fibonacci numbers
• A note on curvature of \(\alpha\)-connections of a statistical manifold
• A step beyond Tsallis and Rényi entropies
• Algorithmic Learning Theory
• An introduction to support vector machines and other kernel-based learning methods.
• Can Sobolev inequality be written for Sharma-Mittal entropy?
• Clustering with Bregman divergences.
• Constrained Subspace ICA Based on Mutual Information Optimization Directly
• Delay-dependent stabilization of stochastic interval delay systems with nonlinear disturbances
• Divergence Function, Duality, and Convex Analysis
• Escort mean values and the characterization of power-law-decaying probability densities
• Families of alpha-, beta- and gamma-divergences: flexible and robust measures of similarities
• Gaussian processes for machine learning.
• Information theoretic hierarchical clustering
• Information, divergence and risk for binary experiments
• Nonparametric information geometry: from divergence function to referential-representational biduality on statistical manifolds
• Possible generalization of Boltzmann-Gibbs statistics.
• Uncertainty, Information, and Sequential Experiments

Describes a project that uses

Uses Software

• HumanEva

This page was built for publication: Generalized twin Gaussian processes using Sharma-Mittal divergence

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q747261)