The quarks of attention: structure and capacity of neural attention building blocks

DOI10.1016/j.artint.2023.103901MaRDI QIDQ6161491

Pierre Baldi, R. V. Vershinin

Publication date: 27 June 2023

Published in: Artificial Intelligence (Search for Journal in Brave)

zbMATH Keywords

attentionneural networkscapacitycircuit complexitydeep learninggatingtransformerssynaptic modulation

Mathematics Subject Classification ID

Artificial intelligence (68Txx)

Cites Work

• Unnamed Item
• Unnamed Item
• The capacity of feedforward neural networks
• Discrete Mathematics of Neural Networks
• Neural networks, orientations of the hypercube, and algebraic threshold functions
• On the Probability That a Random ± 1-Matrix Is Singular
• Polynomial Threshold Functions, Hyperplane Arrangements, and Random Tensors
• Deep Learning in Science

This page was built for publication: The quarks of attention: structure and capacity of neural attention building blocks