Deep learning in random neural fields: numerical experiments via neural tangent kernel (Q6053432)

The model formulate a \(d\)-dimensional random neural field (\(d\)-RNF). Typically, \(d = 2\) for image processing. The positions of neurons in the \(l\)th layer are denoted by \(\mathbf{z}_l \in \mathbb{T}^d\). The behavior of the model is described as \[ \begin{cases} h^l(\mathbf{z}_l) = \int_{\mathbb{T}^d} \omega^l (\mathbf{z}_l, \mathbf{z}_{l-1}) x^{l-1}(\mathbf{z}_{l-1})d \mathbf{z}_{l-1} +b^l(\mathbf{z}_l)\\ x^l(\mathbf{z}_l = \varphi (h^l(\mathbf{z}_l)) \end{cases} \tag{1} \] where \(\omega^l(\mathbf{z}_l, \mathbf{z}_{l-1})\) is a scalar weight connecting a neuron at position \(\mathbf{z}_{l-1}\) in the \((l-1)\)-th layer of this to a neuron at position \(\mathbf{z}_l\) in the \(l\)th layer, \(h^l(\mathbf{z}^l)\) is the pre-activation of the neuron at \(\mathbf{z}_l, b^l(\mathbf{z}_l)\) is a bias function, and \(x^l(\mathbf{z}_l)\) is the output of this neuron, which becomes the input to the next \((l+1)\)-th layer. Here, \(\varphi (\cdot)\) is an activation function. In the present paper, first formulate introduce model mathematically. Next, address three research questions: \((1)\) How do the performances of neural fields depend on the intensity of the correlation in the initial random connections and also the size of the receptive field? \((2)\) Are neural fields with correlated neurons and receptive fields still governed by the NTK regime? \((3)\) Does our model improve robustness to perturbations? We aim to investigate these questions with numerical simulations, although the simulations are preliminary. The main results in the paper are as follows. Formulate and investigate the supervised learning of multilayer random neural fields. We numerically confirm that our model of multilayer random neural fields with correlated neurons and receptive fields is governed by the NTK regime. Find that our model of random neural fields is robust under the disturbances of both random noise and deformation of training samples. The generalization ability is slightly superior to those of conventional models of neural networks.