Two-layer networks with the \(\text{ReLU}^k\) activation function: Barron spaces and derivative approximation (Q6191372)

The paper studies the approximation of a function and its derivatives using a two-layer neural network. The activation function is chosen to be a rectified power unit \[ \mbox{ReLU}^k(x)=(\max(0,x))^k \] so that the approximation of derivatives is possible. By investigating the corresponding Barron space, the authors show that two-layer networks with the \(\mbox{ReLU}^k\) activation function are able to simultaneously approximate an unknown function and its derivatives. A Tikhonov type regularization method is proposed to fulfill the purpose. Error bounds are established and several numerical examples to support the efficiency of the proposed approach are provided.