On the relative expressiveness of Bayesian and neural networks

DOI10.1016/J.IJAR.2019.07.008zbMATH Open1468.68097arXiv1812.08957OpenAlexW2966309298MaRDI QIDQ2302782FDOQ2302782

Adnan Darwiche, Arthur Choi, Ruocheng Wang

Publication date: 26 February 2020

Published in: International Journal of Approximate Reasoning (Search for Journal in Brave)

Abstract: A neural network computes a function. A central property of neural networks is that they are "universal approximators:" for a given continuous function, there exists a neural network that can approximate it arbitrarily well, given enough neurons (and some additional assumptions). In contrast, a Bayesian network is a model, but each of its queries can be viewed as computing a function. In this paper, we identify some key distinctions between the functions computed by neural networks and those by marginal Bayesian network queries, showing that the former are more expressive than the latter. Moreover, we propose a simple augmentation to Bayesian networks (a testing operator), which enables their marginal queries to become "universal approximators."

Full work available at URL: https://arxiv.org/abs/1812.08957

Recommendations

• Bayesian inference in neural networks
• Expressivity of Deep Neural Networks
• Bayesian methods for neural networks and related models
• Deep belief networks are compact universal approximators
• Deep, Narrow Sigmoid Belief Networks Are Universal Approximators

zbMATH Keywords

Bayesian networksneural networksarithmetic circuitsfunction approximation

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Probabilistic graphical models (62H22) Networks and circuits as models of computation; circuit complexity (68Q06)

Cites Work

• Title not available (Why is that?)
• A differential approach to inference in Bayesian networks
• Title not available (Why is that?)
• Modeling and Reasoning with Bayesian Networks
• Probabilistic logic
• A logical calculus of the ideas immanent in nervous activity
• Multilayer feedforward networks are universal approximators
• Approximation by superpositions of a sigmoidal function
• An analysis of first-order logics of probability
• A Fast Learning Algorithm for Deep Belief Nets
• On the revision of probabilistic beliefs using uncertain evidence
• Title not available (Why is that?)
• Connectionist learning of belief networks
• Noisy-or classifier

Cited In (3)

• Towards an effective practice of learning from data and knowledge
• A compositional approach to probabilistic knowledge compilation
• Tucker network: expressive power and comparison

Uses Software

• darch

This page was built for publication: On the relative expressiveness of Bayesian and neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2302782)