Nonparametric Bayesian topic modelling with the hierarchical Pitman-Yor processes

DOI10.1016/J.IJAR.2016.07.007zbMATH OpenNonearXiv1609.06783OpenAlexW2483632691WikidataQ28111781 ScholiaQ28111781MaRDI QIDQ324682FDOQ324682

Lan Du, Wray Buntine, Kar Wai Lim, Changyou Chen

Publication date: 17 October 2016

Published in: International Journal of Approximate Reasoning (Search for Journal in Brave)

Abstract: The Dirichlet process and its extension, the Pitman-Yor process, are stochastic processes that take probability distributions as a parameter. These processes can be stacked up to form a hierarchical nonparametric Bayesian model. In this article, we present efficient methods for the use of these processes in this hierarchical context, and apply them to latent variable models for text analytics. In particular, we propose a general framework for designing these Bayesian models, which are called topic models in the computer science community. We then propose a specific nonparametric Bayesian topic model for modelling text from social media. We focus on tweets (posts on Twitter) in this article due to their ease of access. We find that our nonparametric model performs better than existing parametric models in both goodness of fit and real world applications.

Full work available at URL: https://arxiv.org/abs/1609.06783

zbMATH Keywords

Markov chain Monte CarloBayesian nonparametric methodshierarchical Pitman-Yor processestopic models

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Generalized stochastic processes (60G20)

Cites Work

• Reversible jump Markov chain Monte Carlo computation and Bayesian model determination
• The Collapsed Gibbs Sampler in Bayesian Computations with Applications to a Gene Regulation Problem
• A Bayesian analysis of some nonparametric problems
• Delayed rejection in reversible jump Metropolis-Hastings.
• Title not available (Why is that?)
• Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
• Monte Carlo sampling methods using Markov chains and their applications
• 10.1162/jmlr.2003.3.4-5.993
• Equation of State Calculations by Fast Computing Machines
• The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator
• Combinatorial stochastic processes. Ecole d'Eté de Probabilités de Saint-Flour XXXII -- 2002.
• Hierarchical Dirichlet Processes
• Introduction to Information Retrieval
• Bayesian Nonparametrics
• Gibbs Sampling Methods for Stick-Breaking Priors
• Title not available (Why is that?)
• Title not available (Why is that?)
• Thermodynamic limits of macroeconomic or financial models: one- and two-parameter Poisson-Dirichlet models
• On Metropolis-Hastings algorithms with delayed rejection
• The nested chinese restaurant process and bayesian nonparametric inference of topic hierarchies
• Bayesian Non-Parametric Inference for Species Variety with a Two-Parameter Poisson–Dirichlet Process Prior
• Title not available (Why is that?)
• Title not available (Why is that?)

Cited In (7)

• Fast approximation of variational Bayes Dirichlet process mixture using the maximization-maximization algorithm
• Accurate parameter estimation for Bayesian network classifiers using hierarchical Dirichlet processes
• A segmented topic model based on the two-parameter Poisson-Dirichlet process
• Hierarchical topic modeling with nested hierarchical Dirichlet process
• A three-way approach for learning rules in automatic knowledge-based topic models
• Dynamic hierarchical Dirichlet processes topic model using the power prior approach
• Hierarchical species sampling models

Uses Software

• WinBUGS
• JAGS
• BayesDA
• Equator
• langid.py

This page was built for publication: Nonparametric Bayesian topic modelling with the hierarchical Pitman-Yor processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q324682)