Designing universal causal deep learning models: The geometric (Hyper)transformer

DOI10.1111/MAFI.12389MaRDI QIDQ6196301zbMATH OpenOpenAlexFDO

Authors B. Acciaio, Anastasis Kratsios, Gudmund Pammer

Publication date 14 March 2024

Published in Mathematical Finance (Search for Journal in Brave)

Copyright license Creative Commons Attribution-NonCommercial 4.0 International

Full work available at URL https://arxiv.org/abs/2201.13094

zbMATH Keywords

metric geometrystochastic processesrandom projectionuniversal approximationgeometric deep learninghypernetworksadapted optimal transporttransformer networks

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Financial applications of other theories (91G80) Optimal stochastic control (93E20)

Abstract: Several problems in stochastic analysis are defined through their geometry, and preserving that geometric structure is essential to generating meaningful predictions. Nevertheless, how to design principled deep learning (DL) models capable of encoding these geometric structures remains largely unknown. We address this open problem by introducing a universal causal geometric DL framework in which the user specifies a suitable pair of metric spaces $mathscrX$ and $mathscrY$ and our framework returns a DL model capable of causally approximating any ``regular map sending time series in $mathscr{X}^{mathbbZ}$ to time series in $mathscr{Y}^{mathbbZ}$ while respecting their forward flow of information throughout time. Suitable geometries on $mathscrY$ include various (adapted) Wasserstein spaces arising in optimal stopping problems, a variety of statistical manifolds describing the conditional distribution of continuous-time finite state Markov chains, and all Fr'{e}chet spaces admitting a Schauder basis, e.g. as in classical finance. Suitable spaces $mathscrX$ are compact subsets of any Euclidean space. Our results all quantitatively express the number of parameters needed for our DL model to achieve a given approximation error as a function of the target map's regularity and the geometric structure both of $mathscrX$ and of $mathscrY$. Even when omitting any temporal structure, our universal approximation theorems are the first guarantees that H"{o}lder functions, defined between such $mathscrX$ and $mathscrY$ can be approximated by DL models.

Recommendations

• Universal regular conditional distributions via probabilistic transformers
• Generative OrnsteinUhlenbeck markets via geometric deep learning
• Structure-preserving deep learning
• Deep learning via dynamical systems: an approximation perspective
• Quant GANs: deep generation of financial time series

Cites work

• scientific article; zbMATH DE number 3826915 (Why is no real title available?)
• scientific article; zbMATH DE number 3761167 (Why is no real title available?)
• scientific article; zbMATH DE number 477682 (Why is no real title available?)
• scientific article; zbMATH DE number 1461253 (Why is no real title available?)
• scientific article; zbMATH DE number 1405266 (Why is no real title available?)
• scientific article; zbMATH DE number 7626805 (Why is no real title available?)
• scientific article; zbMATH DE number 7306919 (Why is no real title available?)
• scientific article; zbMATH DE number 7164779 (Why is no real title available?)
• scientific article; zbMATH DE number 7064043 (Why is no real title available?)
• scientific article; zbMATH DE number 3195732 (Why is no real title available?)
• scientific article; zbMATH DE number 6541941 (Why is no real title available?)
• A functional extension of the Ito formula
• A generalized modulus of smoothness
• A logical calculus of the ideas immanent in nervous activity
• A phase transition phenomenon between the isometric and isomorphic extension problems for Hölder functions betweenL_pspaces
• A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations
• A random matrix approach to neural networks
• Adapted Wasserstein distances and stability in mathematical finance
• All adapted topologies are equal
• Approximation rates for neural networks with general activation functions
• Barycenters in the Wasserstein space
• Bilipschitz embeddings of metric spaces into space forms
• Bishop's generalized Stone-Weierstraß theorem for weighted spaces
• Bond Pricing and the Term Structure of Interest Rates: A New Methodology for Contingent Claims Valuation
• Book Review: Optimal transport: old and new
• Conical geodesic bicombings on subsets of normed vector spaces
• Consistency problems for Heath-Jarrow-Morton interest rate models
• Convergence rate of optimal quantization and application to the clustering performance of the empirical measure
• Convex geodesic bicombings and hyperbolicity
• Deep hedging
• Deep network with approximation error being reciprocal of width to power of square root of depth
• Deep optimal stopping
• DeepLOB: Deep Convolutional Neural Networks for Limit Order Books
• Differential equations driven by rough signals. I: An extension of an inequality of L. C. Young
• Echo state networks are universal
• Echo state property linked to an input: exploring a fundamental characteristic of recurrent neural networks
• Equal risk pricing of derivatives with deep hedging
• Exponential families of stochastic processes
• Fading memory and the problem of approximating nonlinear operators with Volterra series
• Fading memory echo state networks are universal
• Fixed point theorems for metric spaces with a conical geodesic bicombing
• Functional Itō calculus and stochastic integral representation of martingales
• Fuzzy Set Theory and Topos Theory
• HJM: A Unified Approach to Dynamic Models for Fixed Income, Credit and Equity Markets
• Information geometry
• Information geometry and its applications
• Information geometry and sufficient statistics
• Learning representations by back-propagating errors
• Lectures on analysis on metric spaces
• Linear Lipschitz and \(C^1\) extension operators through random projection
• Martingale Hardy spaces and their applications in Fourier analysis
• Measure density and extension of Besov and Triebel-Lizorkin functions
• Measure theory. Vol. I and II
• Measured descent: A new embedding method for finite metrics
• Memory Capacity of Neural Networks with Threshold and Rectified Linear Unit Activations
• Metric entropy limits on recurrent neural network learning of linear dynamical systems
• Multivariate normal distributions parametrized as a Riemannian symmetric space
• Nagata dimension, quasisymmetric embeddings, and Lipschitz extensions
• Neural network approximation: three hidden layers are enough
• Neural networks in Fréchet spaces
• On a Heath-Jarrow-Morton approach for stock options
• On the computational power of neural nets
• One-Parameter Semigroups for Linear Evolution Equations
• Optimal stochastic control, stochastic target problems, and backward SDE.
• Provable approximation properties for deep neural networks
• Quant GANs: deep generation of financial time series
• Randomized Wasserstein barycenter computation: resampling with statistical guarantees
• Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations
• Regression on fixed-rank positive semidefinite matrices: a Riemannian approach
• Reservoir computing approaches to recurrent neural network training
• Solving high-dimensional optimal stopping problems using deep learning
• Stability and memory-loss go hand-in-hand: three results in dynamics and computation
• Statistical analysis of counting processes
• Stochastic Gradient Descent on Riemannian Manifolds
• Stochastic calculus and applications
• Stochastic differential inclusions and applications.
• The Cartan-Hadamard theorem for metric spaces with local geodesic bicombings
• The Wasserstein distance and approximation theorems
• Theory of Reproducing Kernels
• Theory of reproducing kernels and applications
• Uniform decomposition of probability measures: quantization, clustering and rate of convergence
• Universal discrete-time reservoir computers with stochastic inputs and linear readouts using non-homogeneous state-affine systems
• Universal features of price formation in financial markets: perspectives from deep learning
• Wasserstein Riemannian geometry of Gaussian densities

Cited in

(2)

• Out-of-distributional risk bounds for neural operators with applications to the Helmholtz equation
• Universal regular conditional distributions via probabilistic transformers

This page was built for publication: Designing universal causal deep learning models: The geometric (Hyper)transformer

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6196301)