Generative adversarial networks are special cases of artificial curiosity (1990) and also closely related to predictability minimization (1991)

DOI10.1016/J.NEUNET.2020.04.008MaRDI QIDQ1982402zbMATH OpenOpenAlexDBLPWikidataFDO

Authors Jürgen Schmidhuber

Publication date 8 September 2021

Published in Neural Networks (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1906.04493

zbMATH Keywords

artificial curiositygenerative adversarial networkspredictability minimization

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Applications of game theory (91A80)

Abstract: I review unsupervised or self-supervised neural networks playing minimax games in game-theoretic settings: (i) Artificial Curiosity (AC, 1990) is based on two such networks. One network learns to generate a probability distribution over outputs, the other learns to predict effects of the outputs. Each network minimizes the objective function maximized by the other. (ii) Generative Adversarial Networks (GANs, 2010-2014) are an application of AC where the effect of an output is 1 if the output is in a given set, and 0 otherwise. (iii) Predictability Minimization (PM, 1990s) models data distributions through a neural encoder that maximizes the objective function minimized by a neural predictor of the code components. I correct a previously published claim that PM is not based on a minimax game.

Recommendations

• scientific article; zbMATH DE number 7626743
• scientific article; zbMATH DE number 7267187
• Understanding Estimation and Generalization Error of Generative Adversarial Networks
• Deconstructing Generative Adversarial Networks
• Generative Adversarial Network for Probabilistic Forecast of Random Dynamical Systems
• A spin glass model for the loss surfaces of generative adversarial networks
• Generative adversarial networks with decoder-encoder output noises

Cites work

• A Mathematical Theory of Communication
• Convergence rate of linear two-time-scale stochastic approximation.
• Domain-adversarial training of neural networks
• scientific article; zbMATH DE number 1306865 (Why is no real title available?)
• scientific article; zbMATH DE number 194374 (Why is no real title available?)
• scientific article; zbMATH DE number 3106184 (Why is no real title available?)
• On Information and Sufficiency
• Simple Algorithmic Principles of Discovery, Subjective Beauty, Selective Attention, Curiosity and Creativity
• Some aspects of the sequential design of experiments
• Stochastic approximation with two time scales
• Taylor expansion of the accumulated rounding error
• The application of the genetic algorithm to the minimization of potential energy functions

Cited in

(1)

• High-dimensional density estimation with tensorizing flow

Describes a project that uses

Uses Software

• InfoGAN
• Wasserstein GAN
• f-GAN
• StyleGAN

This page was built for publication: Generative adversarial networks are special cases of artificial curiosity (1990) and also closely related to predictability minimization (1991)

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1982402)