A network that learns Strassen multiplication

Publication date 16 August 2016

Full work available at URL https://arxiv.org/abs/1601.07227, http://jmlr.csail.mit.edu/papers/v17/16-074.html

Strassen multiplication sum-product networks tensor decomposition

Learning and adaptive systems in artificial intelligence (68T05) Symbolic computation and algebraic computation (68W30)

Abstract: We study neural networks whose only non-linear components are multipliers, to test a new training rule in a context where the precise representation of data is paramount. These networks are challenged to discover the rules of matrix multiplication, given many examples. By limiting the number of multipliers, the network is forced to discover the Strassen multiplication rules. This is the mathematical equivalent of finding low rank decompositions of the

n i m e s n

matrix multiplication tensor,

M_{n}

. We train these networks with the conservative learning rule, which makes minimal changes to the weights so as to give the correct output for each input at the time the input-output pair is received. Conservative learning needs a few thousand examples to find the rank 7 decomposition of

M_{2}

, and

1 0^{5}

for the rank 23 decomposition of

M_{3}

(the lowest known). High precision is critical, especially for

M_{3}

, to discriminate between true decompositions and "border approximations".

Recommendations

Cited in

(2)

This page was built for publication: A network that learns Strassen multiplication

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3188053)