An enriched category theory of language: from syntax to semantics

DOI10.1007/S44007-022-00021-2zbMATH Open1491.18008arXiv2106.07890OpenAlexW3167361329MaRDI QIDQ2153136FDOQ2153136

Yiannis Vlassopoulos, John Terilla, Tai-Danae Bradley

Publication date: 1 July 2022

Published in: La Matematica (Search for Journal in Brave)

Abstract: State of the art language models return a natural language text continuation from any piece of input text. This ability to generate coherent text extensions implies significant sophistication, including a knowledge of grammar and semantics. In this paper, we propose a mathematical framework for passing from probability distributions on extensions of given texts, such as the ones learned by today's large language models, to an enriched category containing semantic information. Roughly speaking, we model probability distributions on texts as a category enriched over the unit interval. Objects of this category are expressions in language, and hom objects are conditional probabilities that one expression is an extension of another. This category is syntactical -- it describes what goes with what. Then, via the Yoneda embedding, we pass to the enriched category of unit interval-valued copresheaves on this syntactical category. This category of enriched copresheaves is semantic -- it is where we find meaning, logical operations such as entailment, and the building blocks for more elaborate semantic concepts.

Full work available at URL: https://arxiv.org/abs/2106.07890

Recommendations

• Language modeling with reduced densities
• scientific article; zbMATH DE number 7453969
• A generalised quantifier theory of natural language in categorical compositional distributional semantics with bialgebras
• Ambiguity and incomplete information in categorical models of language
• Open system categorical quantum semantics in natural language processing
• Lambek vs. Lambek: functorial vector space semantics and string diagrams for Lambek calculus
• Compositionality for recursive neural networks
• Generalized relations in linguistics and cognition
• Categorical vector space semantics for Lambek calculus with a relevant modality (extended abstract)
• Coherent diagrammatic reasoning in compositional distributional semantics

zbMATH Keywords

probabilitycategory theorylogiccompositionalitynatural languageYoneda embedding

Mathematics Subject Classification ID

Categories admitting limits (complete categories), functors preserving limits, completions (18A35) Topoi (18B25) Functor categories, comma categories (18A25) Limits and colimits (products, sums, directed limits, pushouts, fiber products, equalizers, kernels, ends and coends, etc.) (18A30) Enriched categories (over closed or monoidal categories) (18D20) Preorders, orders, domains and lattices (viewed as categories) (18B35)

Cites Work

• Tropical convexity
• Sheaves in geometry and logic: a first introduction to topos theory
• Title not available (Why is that?)
• Title not available (Why is that?)
• Metric spaces, generalized logic, and closed categories
• Title not available (Why is that?)
• The tropical Grassmannian
• From Frequency to Meaning: Vector Space Models of Semantics
• Categorical homotopy theory
• Brief introduction to tropical geometry
• Adjointness in Foundations
• Title not available (Why is that?)
• Introduction to tropical algebraic geometry
• Basic Category Theory
• Category theory in context
• Tight spans, Isbell completions and semi-tropical modules
• Title not available (Why is that?)
• An Invitation to Applied Category Theory
• Semantic Unification
• Lambek pregroups are Frobenius spiders in preorders

Cited In (4)

• Title not available (Why is that?)
• Title not available (Why is that?)
• A Study of Entanglement in a Categorical Framework of Natural Language
• A generalised quantifier theory of natural language in categorical compositional distributional semantics with bialgebras

Uses Software

• Tensor2Tensor
• GPT-3

This page was built for publication: An enriched category theory of language: from syntax to semantics

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2153136)