Compositional sparsity, approximation classes, and parametric transport equations
approximation classescompositional approximationdeep neural networksnonlinear widthsoperator learningparametric transport equationssolution manifoldstamed compositions
Theoretical approximation in context of PDEs (35A35) Dependence of solutions to PDEs on initial and/or boundary data and/or on parameters of PDEs (35B30) Initial-boundary value problems for first-order hyperbolic equations (35L04) Rate of convergence, degree of approximation (41A25) Approximation by arbitrary nonlinear expressions; widths and entropy (41A46) Multidimensional problems (41A63)
There are many good methods to approximate functions of many variables, even when the number of variables, that is, the dimension of the ambient space is large, such as multivariate splines or radial basis functions (less so multivariable polynomials because they have inherent computational and stability problems).\N\NNonetheless, there are limits to the approximation types however useful they are in theory when the dimension of the real space is extremely large. This phenomenon is common, and it is called the curse of dimensionality. It is not so problematic if the approximated functions (the approximands) are very smooth because the standard error estimates usually have factors that depend on the differentiability (or being an element of some very specific function class or space) and they may become small with higher smoothness. But this observation is not a complete remedy for the curse of dimensionality because in the asymptotic estimates there is always some type of power of distances between knots, and those provide a quickly growing factor that is always dimension-dependend.\N\NOn the other hand, it is occasionally possible to reduce the space dimensions of the approximation process to certain manifolds of much lower dimension on which the approximation really should take place because the data are in fact not as high-dimensional as they appear. From time to time, this idea gives satisfactory results but of course one would first have to identify the said manifold. A typical (though simple) example is the approximation on a sphere rather than the whole ambient space.\N\NSo from the classical point of view, there is no obvious solution, but as Wolfgang Dahmen shows in this excellent and deep paper there are other approaches that overcome the CoD, namely by forming approximations as relatively simple compositions (so these are not from a standard linear approximation space, Sobolev space, Besov space etc.) similar to neural network approximations.\N\NThe article is a tour d'horizon of such novel ways to deal with the CoD. The mentioned compositions are formed with -- in the language of neural networks and deep learning -- decoder and encoders which are restricted by stability conditions. Otherwise, since there is now great freedom in choices of the decoder/encoder pairs, trivial solutions may occur. The way the approximands are always written are called ``compositional representations. These representations are essential for this new Ansatz. Also, some natural technical conditions on the approximands are usual.\N\NThose approximations are now called ``compositional approximations, and both concepts and examples are presented in great detail. A very useful approximation problem that the author presents is that of solving transport equations; such an example is highly suitable for the presentation because it is a non-trivial practical problem and it otherwise would suffer severely from the curse of dimensionality. In all contexts and examples, especially those linked to compositions, sparsity is of the essence because otherwise due to the complicated computation, the efficiency would suffer deeply. Of course there are restrictions of the function spaces where the approximands come from because there is no reasonable solution for every approximation problem simultaneously without any conditions.\N\NBoth continuous and discrete situations are addressed which is relevant as soon as the suggested methods are to be implemented in some software. The theorems include, among many other things, dimension-independent error estimates (this is of course what we are looking for) and existence proofs. The results in particular in unison with the illustrative example are most impressive.
- A Kinetic Formulation of Multidimensional Scalar Conservation Laws and Related Equations
- A Proof that Artificial Neural Networks Overcome the Curse of Dimensionality in the Numerical Approximation of Black–Scholes Partial Differential Equations
- A theoretical analysis of deep neural networks and parametric PDEs
- Approximation and learning by greedy algorithms
- Approximation of high-dimensional parametric PDEs
- Approximation of infinitely differentiable multivariate functions is intractable
- Approximation rates for neural networks with general activation functions
- Approximation spaces of deep neural networks
- Deep Network Approximation for Smooth Functions
- Efficient approximation of solutions of parametric linear transport equations by ReLU DNNs
- Error bounds for approximations with deep ReLU networks
- Error bounds for approximations with deep ReLU neural networks in \(W^{s , p}\) norms
- Exponential ReLU DNN expression of holomorphic maps in high dimension
- Function approximation by deep networks
- scientific article; zbMATH DE number 1215245 (Why is no real title available?)
- Kinetic Semidiscretization of Scalar Conservation Laws and Convergence by Using Averaging Lemmas
- Metric Entropy, Widths, and Superpositions of Functions
- Neural network approximation
- Nonlinear approximation and (deep) ReLU networks
- Nonparametric regression using deep neural networks with ReLU activation function
- Optimal approximation of piecewise smooth functions using deep ReLU neural networks
- Optimal approximation rate of ReLU networks in terms of width and depth
- Optimal nonlinear approximation
- Optimal stable nonlinear approximation
- Proof of the theory-to-practice gap in deep learning via sampling complexity bounds for neural network approximation spaces
- Reduced Basis Greedy Selection Using Random Training Sets
- Solving the Kolmogorov PDE by means of deep learning
- Sparse adaptive approximation of high dimensional parametric initial value problems
- Tensor-sparsity of solutions to high-dimensional elliptic partial differential equations
- The Barron space and the flow-induced function spaces for neural network models
- Universal approximation bounds for superpositions of a sigmoidal function
This page was built for publication: Compositional sparsity, approximation classes, and parametric transport equations
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q7006567)
