Designing universal causal deep learning models: The geometric (Hyper)transformer

DOI10.1111/MAFI.12389arXiv2201.13094OpenAlexW4367052271MaRDI QIDQ6196301FDOQ6196301

Authors: B. Acciaio, Anastasis Kratsios, Gudmund Pammer 

Publication date: 14 March 2024

Published in: Mathematical Finance (Search for Journal in Brave)

Abstract: Several problems in stochastic analysis are defined through their geometry, and preserving that geometric structure is essential to generating meaningful predictions. Nevertheless, how to design principled deep learning (DL) models capable of encoding these geometric structures remains largely unknown. We address this open problem by introducing a universal causal geometric DL framework in which the user specifies a suitable pair of metric spaces $mathscrX$ and $mathscrY$ and our framework returns a DL model capable of causally approximating any ``regular map sending time series in $mathscr{X}^{mathbbZ}$ to time series in $mathscr{Y}^{mathbbZ}$ while respecting their forward flow of information throughout time. Suitable geometries on $mathscrY$ include various (adapted) Wasserstein spaces arising in optimal stopping problems, a variety of statistical manifolds describing the conditional distribution of continuous-time finite state Markov chains, and all Fr'{e}chet spaces admitting a Schauder basis, e.g. as in classical finance. Suitable spaces $mathscrX$ are compact subsets of any Euclidean space. Our results all quantitatively express the number of parameters needed for our DL model to achieve a given approximation error as a function of the target map's regularity and the geometric structure both of $mathscrX$ and of $mathscrY$. Even when omitting any temporal structure, our universal approximation theorems are the first guarantees that H"{o}lder functions, defined between such $mathscrX$ and $mathscrY$ can be approximated by DL models.

Full work available at URL: https://arxiv.org/abs/2201.13094

Recommendations

• Universal regular conditional distributions via probabilistic transformers
• Generative OrnsteinUhlenbeck markets via geometric deep learning
• Structure-preserving deep learning
• Deep learning via dynamical systems: an approximation perspective
• Quant GANs: deep generation of financial time series

zbMATH Keywords

metric geometrystochastic processesrandom projectionuniversal approximationgeometric deep learninghypernetworksadapted optimal transporttransformer networks

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Financial applications of other theories (91G80) Optimal stochastic control (93E20)

Cites Work

• DeepLOB: Deep Convolutional Neural Networks for Limit Order Books
• Measure theory. Vol. I and II
• Title not available (Why is that?)
• Theory of Reproducing Kernels
• Barycenters in the Wasserstein space
• Bond Pricing and the Term Structure of Interest Rates: A New Methodology for Contingent Claims Valuation
• Learning representations by back-propagating errors
• Title not available (Why is that?)
• Lectures on analysis on metric spaces
• Functional Itō calculus and stochastic integral representation of martingales
• Nagata dimension, quasisymmetric embeddings, and Lipschitz extensions
• Title not available (Why is that?)
• One-Parameter Semigroups for Linear Evolution Equations
• Martingale Hardy spaces and their applications in Fourier analysis
• Measured descent: A new embedding method for finite metrics
• Optimal stochastic control, stochastic target problems, and backward SDE.
• Title not available (Why is that?)
• A logical calculus of the ideas immanent in nervous activity
• Consistency problems for Heath-Jarrow-Morton interest rate models
• Bilipschitz embeddings of metric spaces into space forms
• Measure density and extension of Besov and Triebel-Lizorkin functions
• A functional extension of the Ito formula
• Information geometry and sufficient statistics
• Convex geodesic bicombings and hyperbolicity
• Fixed point theorems for metric spaces with a conical geodesic bicombing
• Fuzzy Set Theory and Topos Theory
• Title not available (Why is that?)
• On a Heath-Jarrow-Morton approach for stock options
• Title not available (Why is that?)
• HJM: A Unified Approach to Dynamic Models for Fixed Income, Credit and Equity Markets
• On the computational power of neural nets
• Differential equations driven by rough signals. I: An extension of an inequality of L. C. Young
• Approximation rates for neural networks with general activation functions
• Stochastic differential inclusions and applications.
• Fading memory and the problem of approximating nonlinear operators with Volterra series
• Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations
• The Wasserstein distance and approximation theorems
• Stochastic Gradient Descent on Riemannian Manifolds
• Multivariate normal distributions parametrized as a Riemannian symmetric space
• Reservoir computing approaches to recurrent neural network training
• Exponential families of stochastic processes
• Statistical analysis of counting processes
• Regression on fixed-rank positive semidefinite matrices: a Riemannian approach
• A phase transition phenomenon between the isometric and isomorphic extension problems for Hölder functions betweenL_pspaces
• Wasserstein Riemannian geometry of Gaussian densities
• Theory of reproducing kernels and applications
• Title not available (Why is that?)
• Information geometry and its applications
• Linear Lipschitz and \(C^1\) extension operators through random projection
• A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations
• Information geometry
• Adapted Wasserstein distances and stability in mathematical finance
• Solving high-dimensional optimal stopping problems using deep learning
• Deep optimal stopping
• Memory Capacity of Neural Networks with Threshold and Rectified Linear Unit Activations
• Bishop's generalized Stone-Weierstraß theorem for weighted spaces
• Deep network with approximation error being reciprocal of width to power of square root of depth
• Equal risk pricing of derivatives with deep hedging
• A generalized modulus of smoothness
• Quant GANs: deep generation of financial time series
• Stochastic calculus and applications
• A random matrix approach to neural networks
• Uniform decomposition of probability measures: quantization, clustering and rate of convergence
• Provable approximation properties for deep neural networks
• The Cartan-Hadamard theorem for metric spaces with local geodesic bicombings
• Title not available (Why is that?)
• Echo state networks are universal
• Randomized Wasserstein barycenter computation: resampling with statistical guarantees
• Title not available (Why is that?)
• Deep hedging
• Neural network approximation: three hidden layers are enough
• Universal features of price formation in financial markets: perspectives from deep learning
• Universal discrete-time reservoir computers with stochastic inputs and linear readouts using non-homogeneous state-affine systems
• Title not available (Why is that?)
• Fading memory echo state networks are universal
• Title not available (Why is that?)
• All adapted topologies are equal
• Echo state property linked to an input: exploring a fundamental characteristic of recurrent neural networks
• Conical geodesic bicombings on subsets of normed vector spaces
• Metric entropy limits on recurrent neural network learning of linear dynamical systems
• Book Review: Optimal transport: old and new
• Neural networks in Fréchet spaces
• Convergence rate of optimal quantization and application to the clustering performance of the empirical measure
• Stability and memory-loss go hand-in-hand: three results in dynamics and computation

Cited In (2)

• Out-of-distributional risk bounds for neural operators with applications to the Helmholtz equation
• Universal regular conditional distributions via probabilistic transformers

This page was built for publication: Designing universal causal deep learning models: The geometric (Hyper)transformer

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6196301)