Generalized twin Gaussian processes using Sharma-Mittal divergence

DOI10.1007/S10994-015-5497-9zbMATH Open1341.62203arXiv1409.7480OpenAlexW1576564833MaRDI QIDQ747261FDOQ747261

Authors: Mohamed Elhoseiny, Ahmed Elgammal 

Publication date: 23 October 2015

Published in: Machine Learning (Search for Journal in Brave)

Abstract: There has been a growing interest in mutual information measures due to their wide range of applications in Machine Learning and Computer Vision. In this paper, we present a generalized structured regression framework based on Shama-Mittal divergence, a relative entropy measure, which is introduced to the Machine Learning community in this work. Sharma-Mittal (SM) divergence is a generalized mutual information measure for the widely used R'enyi, Tsallis, Bhattacharyya, and Kullback-Leibler (KL) relative entropies. Specifically, we study Sharma-Mittal divergence as a cost function in the context of the Twin Gaussian Processes (TGP)~citep{Bo:2010}, which generalizes over the KL-divergence without computational penalty. We show interesting properties of Sharma-Mittal TGP (SMTGP) through a theoretical analysis, which covers missing insights in the traditional TGP formulation. However, we generalize this theory based on SM-divergence instead of KL-divergence which is a special case. Experimentally, we evaluated the proposed SMTGP framework on several datasets. The results show that SMTGP reaches better predictions than KL-based TGP, since it offers a bigger class of models through its parameters that we learn from the data.

Full work available at URL: https://arxiv.org/abs/1409.7480

Recommendations

• Machine learning with squared-loss mutual information
• Novel generalized divergence measures and multimodal medical image registration
• Information measures of kernel estimation
• A closed-form expression for the Sharma-Mittal entropy of exponential families
• scientific article; zbMATH DE number 2218582

zbMATH Keywords

image reconstructionpose estimationstructured regressionSharma-Mittal entropytwin Gaussian processes

Mathematics Subject Classification ID

Statistical aspects of information-theoretic topics (62B10) General nonlinear regression (62J02) Measures of information, entropy (94A17) Computing methodologies for image processing (68U10) Machine vision and scene understanding (68T45)

Cites Work

• Gaussian processes for machine learning.
• An introduction to support vector machines and other kernel-based learning methods.
• Algorithmic Learning Theory
• Clustering with Bregman divergences.
• Possible generalization of Boltzmann-Gibbs statistics.
• Title not available (Why is that?)
• Title not available (Why is that?)
• Nonparametric information geometry: from divergence function to referential-representational biduality on statistical manifolds
• A Generalized Divergence Measure for Nonnegative Matrix Factorization
• Divergence Function, Duality, and Convex Analysis
• Families of alpha-, beta- and gamma-divergences: flexible and robust measures of similarities
• A note on curvature of \(\alpha\)-connections of a statistical manifold
• A closed-form expression for the Sharma-Mittal entropy of exponential families
• A new family of \(k\)-Fibonacci numbers
• Escort mean values and the characterization of power-law-decaying probability densities
• Uncertainty, Information, and Sequential Experiments
• Title not available (Why is that?)
• Title not available (Why is that?)
• Information theoretic hierarchical clustering
• 10.1162/jmlr.2003.4.7-8.1271
• Can Sobolev inequality be written for Sharma-Mittal entropy?
• Delay-dependent stabilization of stochastic interval delay systems with nonlinear disturbances
• Information, Divergence and Risk for Binary Experiments
• A step beyond Tsallis and Rényi entropies
• Constrained Subspace ICA Based on Mutual Information Optimization Directly

Uses Software

• HumanEva

This page was built for publication: Generalized twin Gaussian processes using Sharma-Mittal divergence

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q747261)