An Interpretive Constrained Linear Model for ResNet and MgNet

DOI10.1016/J.NEUNET.2023.03.011MaRDI QIDQ6385646zbMATH OpenOpenAlexFDO

Authors Juncai He, Jinchao Xu, Lian Zhang, Jianqing Zhu

Publication date 14 December 2021

Full work available at URL https://doi.org/10.1016/j.neunet.2023.03.011

Mathematics Subject Classification ID

Artificial intelligence (68Txx)

Abstract: We propose a constrained linear data-feature-mapping model as an interpretable mathematical model for image classification using a convolutional neural network (CNN). From this viewpoint, we establish detailed connections between the traditional iterative schemes for linear systems and the architectures of the basic blocks of ResNet- and MgNet-type models. Using these connections, we present some modified ResNet models that compared with the original models have fewer parameters and yet can produce more accurate results, thereby demonstrating the validity of this constrained learning data-feature-mapping assumption. Based on this assumption, we further propose a general data-feature iterative scheme to show the rationality of MgNet. We also provide a systematic numerical study on MgNet to show its success and advantages in image classification problems and demonstrate its advantages in comparison with established networks.

Recommendations

• MgNet: a unified framework of multigrid and convolutional neural network
• Residual networks as flows of diffeomorphisms
• Deep neural networks motivated by partial differential equations
• Convergence analysis of deep residual networks

Cites work

• Algebraic multigrid methods
• Approximation properties of deep ReLU CNNs
• scientific article; zbMATH DE number 6378127 (Why is no real title available?)
• scientific article; zbMATH DE number 1324524 (Why is no real title available?)
• Iterative Methods by Space Decomposition and Subspace Correction
• Neural network with unbounded activation functions is universal approximator
• The method of alternating projections and the method of subspace corrections in Hilbert space

Cited in

(3)

• MGIC: Multigrid-in-Channels Neural Network Architectures
• An enhanced V-cycle MgNet model for operator learning in numerical partial differential equations
• DeepONet based preconditioning strategies for solving parametric linear systems of equations

This page was built for publication: An Interpretive Constrained Linear Model for ResNet and MgNet

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6385646)