Nonparametric regression using deep neural networks with ReLU activation function (Q2215715)

The author considers the already classical nonparametric regression model $Y_i =f_0(\mathbf{X} _i) +\epsilon _i $, where $\mathbf{X}_i\in[0,1]^d$, $Y_i\in \mathbb R$, $i=1,\dots,n$, $\mathbf{X} _i$ are the observed vectors, $Y_i$ are the responses and $\epsilon _i$ are the noise variables. The statistical problem is to recover $f_0$ from the samples ($\mathbf{X}_i$, $Y_i$). In the present paper, the estimation of the regression function is done by means of neural networks technology. In the second section, multilayer feedforward artificial neural networks are introduced and concepts as deep neural networks, deep learning, sparsity of the neural network, rectified linear unit (ReLU) are explained. The third section is devoted to the presentation of main results. The regression function is assumed to be a composition of several functions. The author proves that the estimators based on sparsely connected deep neural networks with ReLU activation function and for an appropriate network architecture can achieve minimax rates of convergence. Special cases of compositional constrains on the regression function are discussed in the fourth section. Suboptimality of wavelet series estimators is discussed in the fifth section and a summary on some achievements on statistical theory and neural networks can be found in the sixth section. The seventh section contains the proofs of results. The author reports that additional proofs are to be found in a supplementary material: ``Nonparametric regression using deep neural networks with ReLU activation function'', \url{doi:10.1214/19-AOS1875SUPP}.