attention

CRANattentionMaRDI QIDQ102016

Self-Attention Algorithm

Bastiaan Quast

Last update: 10 November 2023

Copyright license: GNU General Public License

Software version identifier: 0.2.0, 0.3.0, 0.1.0, 0.4.0

Description

Self-Attention algorithm helper functions and demonstration vignettes of increasing depth on how to construct the Self-Attention algorithm, this is based on Vaswani et al. (2017) <doi:10.48550/arXiv.1706.03762>, Dan Jurafsky and James H. Martin (2022, ISBN:978-0131873216) <https://web.stanford.edu/~jurafsky/slp3/> "Speech and Language Processing (3rd ed.)" and Alex Graves (2020) <https://www.youtube.com/watch?v=AIiwuClvH6k> "Attention and Memory in Deep Learning".

Cites work

• Attention Is All You Need

Related Items (2)

rnn ⋮ transformer

This page was built for software: attention