On the relative expressiveness of Bayesian and neural networks

DOI10.1016/J.IJAR.2019.07.008MaRDI QIDQ2302782zbMATH OpenOpenAlexFDO

Authors Arthur Choi, Ruocheng Wang, Adnan Darwiche

Publication date 26 February 2020

Published in International Journal of Approximate Reasoning (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1812.08957

zbMATH Keywords

Bayesian networksneural networksarithmetic circuitsfunction approximation

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Probabilistic graphical models (62H22) Networks and circuits as models of computation; circuit complexity (68Q06)

Abstract: A neural network computes a function. A central property of neural networks is that they are "universal approximators:" for a given continuous function, there exists a neural network that can approximate it arbitrarily well, given enough neurons (and some additional assumptions). In contrast, a Bayesian network is a model, but each of its queries can be viewed as computing a function. In this paper, we identify some key distinctions between the functions computed by neural networks and those by marginal Bayesian network queries, showing that the former are more expressive than the latter. Moreover, we propose a simple augmentation to Bayesian networks (a testing operator), which enables their marginal queries to become "universal approximators."

Recommendations

• Bayesian inference in neural networks
• Expressivity of Deep Neural Networks
• Bayesian methods for neural networks and related models
• Deep belief networks are compact universal approximators
• Deep, Narrow Sigmoid Belief Networks Are Universal Approximators

Cites work

• scientific article; zbMATH DE number 1149408 (Why is no real title available?)
• scientific article; zbMATH DE number 1149420 (Why is no real title available?)
• A Fast Learning Algorithm for Deep Belief Nets
• A differential approach to inference in Bayesian networks
• A logical calculus of the ideas immanent in nervous activity
• An analysis of first-order logics of probability
• Approximation by superpositions of a sigmoidal function
• Connectionist learning of belief networks
• Deep learning
• Modeling and Reasoning with Bayesian Networks
• Multilayer feedforward networks are universal approximators
• Noisy-or classifier
• On the revision of probabilistic beliefs using uncertain evidence
• Probabilistic logic

Cited in

(4)

• Towards an effective practice of learning from data and knowledge
• Comparison of the representational power of random forests, binary decision diagrams, and neural networks
• A compositional approach to probabilistic knowledge compilation
• Tucker network: expressive power and comparison

Describes a project that uses

Uses Software

• darch

This page was built for publication: On the relative expressiveness of Bayesian and neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2302782)