Formalizing BPE tokenization

DOI10.4204/EPTCS.388.4MaRDI QIDQ6940574zbMATH OpenFDO

Authors Martin Berglund, Brink van der Merwe

Publication date 19 August 2025

Mathematics Subject Classification ID

Formal languages and automata (68Q45) Natural language processing (68T50)

This page was built for publication: Formalizing BPE tokenization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6940574)