Attention is Turing-complete

MaRDI QIDQ4998978zbMATH OpenFDO

Authors

Publication date 9 July 2021

Full work available at URL https://jmlr.csail.mit.edu/papers/v22/20-302.html

zbMATH Keywords

arbitrary precisionneural networksself-attentiontransformersTuring completeness

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Recommendations

• scientific article; zbMATH DE number 953043
• Turing computability with neural nets
• Expressive power of first-order recurrent neural networks determined by their attractor dynamics
• Synthesis and analysis of elementary algorithms for a differential neural computer
• Computational power of neural networks: a characterization in terms of Kolmogorov complexity

Cites work

• Approximation by superpositions of a sigmoidal function
• Deep learning
• scientific article; zbMATH DE number 5595162 (Why is no real title available?)
• On the computational power of neural nets

Cited in

(5)

• Priors in Bayesian Deep Learning: A Review
• DCT-former: efficient self-attention with discrete cosine transform
• A provably stable neural network Turing machine with finite precision and time
• Neural networks generalize on low complexity data
• The role of logic and automata in understanding transformers

Describes a project that uses

Uses Software

• Tensor2Tensor
• BERT

This page was built for publication: Attention is Turing-complete

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4998978)