Backprop as Functor: A compositional perspective on supervised learning

MaRDI QIDQ6294520zbMATH OpenFDO

Authors Brendan Fong, David I. Spivak, Rémy Tuyéras

Publication date 28 November 2017

Has companion code repository https://github.com/darrenjw/isba2021

Abstract: A supervised learning algorithm searches over a set of functions $AoB$ parametrised by a space $P$ to find the best approximation to some ideal function $fcolonAoB$. It does this by taking examples $(a,f(a))inAimesB$, and updating the parameter according to some rule. We define a category where these update rules may be composed, and show that gradient descent---with respect to a fixed step size and an error function satisfying a certain property---defines a monoidal functor from a category of parametrised functions to this category of update rules. This provides a structural perspective on backpropagation, as well as a broad generalisation of neural networks.

This page was built for publication: Backprop as Functor: A compositional perspective on supervised learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6294520)