Visualizing and understanding sum-product networks

DOI10.1007/S10994-018-5760-YzbMATH Open1482.68208DBLPjournals/ml/VergariME19arXiv1608.08266OpenAlexW3101213140WikidataQ62045627 ScholiaQ62045627MaRDI QIDQ669309FDOQ669309

Authors: Antonio Vergari, Nicola Di Mauro, Floriana Esposito 

Publication date: 15 March 2019

Published in: Machine Learning (Search for Journal in Brave)

Abstract: Sum-Product Networks (SPNs) are recently introduced deep tractable probabilistic models by which several kinds of inference queries can be answered exactly and in a tractable time. Up to now, they have been largely used as black box density estimators, assessed only by comparing their likelihood scores only. In this paper we explore and exploit the inner representations learned by SPNs. We do this with a threefold aim: first we want to get a better understanding of the inner workings of SPNs; secondly, we seek additional ways to evaluate one SPN model and compare it against other probabilistic models, providing diagnostic tools to practitioners; lastly, we want to empirically evaluate how good and meaningful the extracted representations are, as in a classic Representation Learning framework. In order to do so we revise their interpretation as deep neural networks and we propose to exploit several visualization techniques on their node activations and network outputs under different types of inference queries. To investigate these models as feature extractors, we plug some SPNs, learned in a greedy unsupervised fashion on image datasets, in supervised classification learning tasks. We extract several embedding types from node activations by filtering nodes by their type, by their associated feature abstraction level and by their scope. In a thorough empirical comparison we prove them to be competitive against those generated from popular feature extractors as Restricted Boltzmann Machines. Finally, we investigate embeddings generated from random probabilistic marginal queries as means to compare other tractable probabilistic models on a common ground, extending our experiments to Mixtures of Trees.

Full work available at URL: https://arxiv.org/abs/1608.08266

Recommendations

• Sum-product graphical models
• Robustifying sum-product networks
• On converting sum-product networks into Bayesian networks
• Resolving inconsistencies of scope interpretations in sum-product networks
• Discriminative training of feed-forward and recurrent sum-product networks by extended Baum-Welch

zbMATH Keywords

representation learningsum-product networkstractable density estimation

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Probabilistic graphical models (62H22)

Cites Work

• Reducing the Dimensionality of Data with Neural Networks
• Probabilistic graphical models.
• A differential approach to inference in Bayesian networks
• Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion
• An efficient learning procedure for deep Boltzmann machines
• On the expressive power of deep architectures

Cited In (8)

• Resolving inconsistencies of scope interpretations in sum-product networks
• Robustifying sum-product networks
• Efficient algorithms for robustness analysis of maximum a posteriori inference in selective sum-product networks
• On converting sum-product networks into Bayesian networks
• \textsc{Strudel}: A fast and accurate learner of structured-decomposable probabilistic circuits
• A hierarchy of sum-product networks using robustness
• Sum-product graphical models
• Learning directed acyclic graph SPNs in sub-quadratic time

Uses Software

• darch
• MADE
• STL-10 dataset

This page was built for publication: Visualizing and understanding sum-product networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q669309)