Compositional sparsity, approximation classes, and parametric transport equations (Q7006567)

There are many good methods to approximate functions of many variables, even when the number of variables, that is, the dimension of the ambient space is large, such as multivariate splines or radial basis functions (less so multivariable polynomials because they have inherent computational and stability problems).\N\NNonetheless, there are limits to the approximation types however useful they are in theory when the dimension of the real space is extremely large. This phenomenon is common, and it is called the curse of dimensionality. It is not so problematic if the approximated functions (the approximands) are very smooth because the standard error estimates usually have factors that depend on the differentiability (or being an element of some very specific function class or space) and they may become small with higher smoothness. But this observation is not a complete remedy for the curse of dimensionality because in the asymptotic estimates there is always some type of power of distances between knots, and those provide a quickly growing factor that is always dimension-dependend.\N\NOn the other hand, it is occasionally possible to reduce the space dimensions of the approximation process to certain manifolds of much lower dimension on which the approximation ''really'' should take place because the data are in fact not as high-dimensional as they appear. From time to time, this idea gives satisfactory results but of course one would first have to identify the said manifold. A typical (though simple) example is the approximation on a sphere rather than the whole ambient space.\N\NSo from the classical point of view, there is no obvious solution, but as Wolfgang Dahmen shows in this excellent and deep paper there are other approaches that overcome the CoD, namely by forming approximations as relatively simple compositions (so these are not from a standard linear approximation space, Sobolev space, Besov space etc.) similar to neural network approximations.\N\NThe article is a tour d'horizon of such novel ways to deal with the CoD. The mentioned compositions are formed with -- in the language of neural networks and deep learning -- decoder and encoders which are restricted by stability conditions. Otherwise, since there is now great freedom in choices of the decoder/encoder pairs, trivial solutions may occur. The way the approximands are always written are called ``compositional representations''. These representations are essential for this new Ansatz. Also, some natural technical conditions on the approximands are usual.\N\NThose approximations are now called ``compositional approximations'', and both concepts and examples are presented in great detail. A very useful approximation problem that the author presents is that of solving transport equations; such an example is highly suitable for the presentation because it is a non-trivial practical problem and it otherwise would suffer severely from the curse of dimensionality. In all contexts and examples, especially those linked to compositions, sparsity is of the essence because otherwise due to the complicated computation, the efficiency would suffer deeply. Of course there are restrictions of the function spaces where the approximands come from because there is no reasonable solution for every approximation problem simultaneously without any conditions.\N\NBoth continuous and discrete situations are addressed which is relevant as soon as the suggested methods are to be implemented in some software. The theorems include, among many other things, dimension-independent error estimates (this is of course what we are looking for) and existence proofs. The results in particular in unison with the illustrative example are most impressive.