Deep learning in random neural fields: numerical experiments via neural tangent kernel (Q6053432)

From MaRDI portal
scientific article; zbMATH DE number 7752115
Language Label Description Also known as
English
Deep learning in random neural fields: numerical experiments via neural tangent kernel
scientific article; zbMATH DE number 7752115

    Statements

    Deep learning in random neural fields: numerical experiments via neural tangent kernel (English)
    0 references
    0 references
    0 references
    0 references
    0 references
    0 references
    18 October 2023
    0 references
    The model formulate a \(d\)-dimensional random neural field (\(d\)-RNF). Typically, \(d = 2\) for image processing. The positions of neurons in the \(l\)th layer are denoted by \(\mathbf{z}_l \in \mathbb{T}^d\). The behavior of the model is described as \[ \begin{cases} h^l(\mathbf{z}_l) = \int_{\mathbb{T}^d} \omega^l (\mathbf{z}_l, \mathbf{z}_{l-1}) x^{l-1}(\mathbf{z}_{l-1})d \mathbf{z}_{l-1} +b^l(\mathbf{z}_l)\\ x^l(\mathbf{z}_l = \varphi (h^l(\mathbf{z}_l)) \end{cases} \tag{1} \] where \(\omega^l(\mathbf{z}_l, \mathbf{z}_{l-1})\) is a scalar weight connecting a neuron at position \(\mathbf{z}_{l-1}\) in the \((l-1)\)-th layer of this to a neuron at position \(\mathbf{z}_l\) in the \(l\)th layer, \(h^l(\mathbf{z}^l)\) is the pre-activation of the neuron at \(\mathbf{z}_l, b^l(\mathbf{z}_l)\) is a bias function, and \(x^l(\mathbf{z}_l)\) is the output of this neuron, which becomes the input to the next \((l+1)\)-th layer. Here, \(\varphi (\cdot)\) is an activation function. In the present paper, first formulate introduce model mathematically. Next, address three research questions: \((1)\) How do the performances of neural fields depend on the intensity of the correlation in the initial random connections and also the size of the receptive field? \((2)\) Are neural fields with correlated neurons and receptive fields still governed by the NTK regime? \((3)\) Does our model improve robustness to perturbations? We aim to investigate these questions with numerical simulations, although the simulations are preliminary. The main results in the paper are as follows. Formulate and investigate the supervised learning of multilayer random neural fields. We numerically confirm that our model of multilayer random neural fields with correlated neurons and receptive fields is governed by the NTK regime. Find that our model of random neural fields is robust under the disturbances of both random noise and deformation of training samples. The generalization ability is slightly superior to those of conventional models of neural networks.
    0 references
    0 references
    random neural field
    0 references
    neural tangent kernel
    0 references
    supervised learning
    0 references
    reproducing kernel Hilbert space
    0 references

    Identifiers