Spectral complexity-scaled generalisation bound of complex-valued neural networks

DOI10.1016/J.ARTINT.2023.103951MaRDI QIDQ6136092zbMATH OpenOpenAlexFDO

Authors Haowen Chen, Fengxiang He, Shiye Lei, Dacheng Tao

Publication date 28 August 2023

Published in Artificial Intelligence (Search for Journal in Brave)

Copyright license Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International

Full work available at URL https://arxiv.org/abs/2112.03467

zbMATH Keywords

complex-valued neural networksgeneralisationspectral complexity

Mathematics Subject Classification ID

Artificial intelligence (68Txx)

Abstract: Complex-valued neural networks (CVNNs) have been widely applied to various fields, especially signal processing and image recognition. However, few works focus on the generalization of CVNNs, albeit it is vital to ensure the performance of CVNNs on unseen data. This paper is the first work that proves a generalization bound for the complex-valued neural network. The bound scales with the spectral complexity, the dominant factor of which is the spectral norm product of weight matrices. Further, our work provides a generalization bound for CVNNs when training data is sequential, which is also affected by the spectral complexity. Theoretically, these bounds are derived via Maurey Sparsification Lemma and Dudley Entropy Integral. Empirically, we conduct experiments by training complex-valued convolutional neural networks on different datasets: MNIST, FashionMNIST, CIFAR-10, CIFAR-100, Tiny ImageNet, and IMDB. Spearman's rank-order correlation coefficients and the corresponding p values on these datasets give strong proof that the spectral complexity of the network, measured by the weight matrices spectral norm product, has a statistically significant correlation with the generalization ability.

Cites work

• scientific article; zbMATH DE number 3772326 (Why is no real title available?)
• Complex-valued neural networks
• Foundations of machine learning
• Nonlinear adaptive prediction of complex-valued signals by complex-valued PRNN
• On the inherent property of the decision boundary in complex-valued neural networks
• Orthogonality of Decision Boundaries in Complex-Valued Neural Networks
• Redundancy of the parameters of the complex-valued neural network.
• Sequential complexities and uniform martingale laws of large numbers
• Statistical analysis of some multi-category large margin classification methods

Cited in

(1)

• The universal approximation theorem for complex-valued neural networks

This page was built for publication: Spectral complexity-scaled generalisation bound of complex-valued neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6136092)