Spectral complexity-scaled generalisation bound of complex-valued neural networks

DOI10.1016/J.ARTINT.2023.103951arXiv2112.03467OpenAlexW4379472738MaRDI QIDQ6136092FDOQ6136092

Shiye Lei, Dacheng Tao, Fengxiang He, Haowen Chen

Publication date: 28 August 2023

Published in: Artificial Intelligence (Search for Journal in Brave)

Abstract: Complex-valued neural networks (CVNNs) have been widely applied to various fields, especially signal processing and image recognition. However, few works focus on the generalization of CVNNs, albeit it is vital to ensure the performance of CVNNs on unseen data. This paper is the first work that proves a generalization bound for the complex-valued neural network. The bound scales with the spectral complexity, the dominant factor of which is the spectral norm product of weight matrices. Further, our work provides a generalization bound for CVNNs when training data is sequential, which is also affected by the spectral complexity. Theoretically, these bounds are derived via Maurey Sparsification Lemma and Dudley Entropy Integral. Empirically, we conduct experiments by training complex-valued convolutional neural networks on different datasets: MNIST, FashionMNIST, CIFAR-10, CIFAR-100, Tiny ImageNet, and IMDB. Spearman's rank-order correlation coefficients and the corresponding p values on these datasets give strong proof that the spectral complexity of the network, measured by the weight matrices spectral norm product, has a statistically significant correlation with the generalization ability.

Full work available at URL: https://arxiv.org/abs/2112.03467

zbMATH Keywords

complex-valued neural networksgeneralisationspectral complexity

Mathematics Subject Classification ID

Artificial intelligence (68Txx)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Orthogonality of Decision Boundaries in Complex-Valued Neural Networks
• Complex-valued neural networks
• Sequential complexities and uniform martingale laws of large numbers
• Nonlinear adaptive prediction of complex-valued signals by complex-valued PRNN
• On the inherent property of the decision boundary in complex-valued neural networks
• Redundancy of the parameters of the complex-valued neural network.
• Title not available (Why is that?)

Cited In (1)

• The universal approximation theorem for complex-valued neural networks

This page was built for publication: Spectral complexity-scaled generalisation bound of complex-valued neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6136092)