Graph-based composite local Bregman divergences on discrete sample spaces

DOI10.1016/J.NEUNET.2017.06.005zbMATH Open1439.62095DBLPjournals/nn/KanamoriT17arXiv1604.06568OpenAlexW2963773379WikidataQ47957952 ScholiaQ47957952MaRDI QIDQ2179070FDOQ2179070

Authors: Takafumi Kanamori, Takashi Takenouchi 

Publication date: 12 May 2020

Published in: Neural Networks (Search for Journal in Brave)

Abstract: One of the most common methods for statistical inference is the maximum likelihood estimator (MLE). The MLE needs to compute the normalization constant in statistical models, and it is often intractable. Using unnormalized statistical models and replacing the likelihood with the other scoring rule are a good way to circumvent such high computation cost, where the scoring rule measures the goodness of fit of the model to observed samples. The scoring rule is closely related to the Bregman divergence, which is a discrepancy measure between two probability distributions. In this paper, the purpose is to provide a general framework of statistical inference using unnormalized statistical models on discrete sample spaces. A localized version of scoring rules is important to obtain computationally efficient estimators. We show that the local scoring rules are related to the localized version of Bregman divergences. Through the localized Bregman divergence, we investigate the statistical consistency of local scoring rules. We show that the consistency is determined by the structure of neighborhood system defined on discrete sample spaces. In addition, we show a way of applying local scoring rules to classification problems. In numerical experiments, we investigated the relation between the neighborhood system and the estimation accuracy.

Full work available at URL: https://arxiv.org/abs/1604.06568

Recommendations

• Proper local scoring rules on discrete sample spaces
• Statistical inference with unnormalized discrete models and localized homogeneous divergences
• Regression using localised functional Bregman divergence
• Proper scoring rules and Bregman divergence
• Asymptotic theory for local estimators based on Bregman divergence

zbMATH Keywords

robustnessBregman divergencelocalityscoring rulecoincidence axiom

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Nonparametric robustness (62G35) Applications of graph theory (05C90) Neural nets and related approaches to inference from stochastic processes (62M45)

Cites Work

• Title not available (Why is that?)
• Strictly Proper Scoring Rules, Prediction, and Estimation
• Elements of Information Theory
• Title not available (Why is that?)
• Title not available (Why is that?)
• Asymptotic Statistics
• Clustering with Bregman divergences.
• Title not available (Why is that?)
• Some extensions of score matching
• Robust and efficient estimation by minimising a density power divergence
• Robust parameter estimation with a small bias against heavy contamination
• MEASURES OF THE VALUE OF INFORMATION
• Proper local scoring rules
• Proper local scoring rules on discrete sample spaces
• Proper Scores for Probability Forecasters
• Information Geometry of U-Boost and Bregman Divergence
• An introduction to MCMC for machine learning
• Training Products of Experts by Minimizing Contrastive Divergence
• Affine invariant divergences associated with proper composite scoring rules and their applications

Cited In (1)

• Statistical inference with unnormalized discrete models and localized homogeneous divergences

This page was built for publication: Graph-based composite local Bregman divergences on discrete sample spaces

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2179070)