Categorical semantics of a simple differential programming language

zbMATH Open1477.68262arXiv2101.10491MaRDI QIDQ5019684FDOQ5019684

Authors: Jonathan Gallagher, Dorette Pronk 

Publication date: 10 January 2022

Abstract: With the increased interest in machine learning, and deep learning in particular, the use of automatic differentiation has become more wide-spread in computation. There have been two recent developments to provide the theoretical support for this types of structure. One approach, due to Abadi and Plotkin, provides a simple differential programming language. Another approach is the notion of a reverse differential category. In the present paper we bring these two approaches together. In particular, we show how an extension of reverse derivative categories models Abadi and Plotkin's language, and describe how this categorical model allows one to consider potential improvements to the operational semantics of the language.

Full work available at URL: https://arxiv.org/abs/2101.10491

Recommendations

• Monoidal reverse differential categories
• Differential categories revisited
• Reverse AD at higher types: pure, principled and denotationally correct
• What is a categorical model of the differential and the resource \(\lambda \)-calculi?
• Categories of Differentiable Polynomial Circuits for Machine Learning

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Theory of programming languages (68N15) Categorical semantics of formal languages (18C50) Semantics in the theory of computing (68Q55) Synthetic differential geometry, tangent categories, differential categories (18F40)

Cites Work

• Deep learning
• LCF considered as a programming language
• A lattice-theoretical fixpoint theorem and its applications
• Title not available (Why is that?)
• Cohesive categories and manifolds
• Total maps of Turing categories
• Timed Sets, Functional Complexity, and Computability
• Restriction categories. I: Categories of partial maps
• Differential calculus over general base fields and rings.
• A type-theoretical alternative to ISWIM, CUCH, OWHY
• Title not available (Why is that?)
• Differential categories
• Title not available (Why is that?)
• Title not available (Why is that?)
• Restriction categories III: colimits, partial limits and extensivity
• Title not available (Why is that?)
• Differential structure, tangent structure, and SDG
• Cartesian differential categories
• The Faà di Bruno construction
• Title not available (Why is that?)
• Range categories II: Towards regularity
• Boolean and classical restriction categories
• On 1-form classifiers^∗
• The delay monad and restriction categories
• Differential restriction categories
• Differential bundles and fibrations for tangent categories
• Elgot algebras (extended abstract)

Cited In (9)

• Categorical foundations of gradient-based learning
• Learners' languages
• Monoidal reverse differential categories
• Categories of Differentiable Polynomial Circuits for Machine Learning
• Title not available (Why is that?)
• Deterministic Caterpillar Expressions
• Latent fibrations: fibrations for categories of partial maps
• Reverse derivative ascent: a categorical approach to learning Boolean circuits
• A simply typed \(\lambda\)-calculus of forward automatic differentiation

Uses Software

• Theano
• SciPy
• TensorFlow
• PyTorch
• Autograd
• Lantern

This page was built for publication: Categorical semantics of a simple differential programming language

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5019684)