MGIC: Multigrid-in-Channels Neural Network Architectures

DOI10.1137/21M1430194MaRDI QIDQ6108155zbMATH OpenOpenAlexFDO

Authors Moshe Eliasof, Lars Ruthotto, Eran Treister

Publication date 29 June 2023

Published in SIAM Journal on Scientific Computing (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2011.09128

zbMATH Keywords

alternative CNN architecturescompact and lightweight neural networksmultilevel neural networks

Mathematics Subject Classification ID

Multigrid methods; domain decomposition for boundary value problems involving PDEs (65N55) Artificial neural networks and deep learning (68T07) Machine vision and scene understanding (68T45)

Abstract: We present a multigrid-in-channels (MGIC) approach that tackles the quadratic growth of the number of parameters with respect to the number of channels in standard convolutional neural networks (CNNs). Thereby our approach addresses the redundancy in CNNs that is also exposed by the recent success of lightweight CNNs. Lightweight CNNs can achieve comparable accuracy to standard CNNs with fewer parameters; however, the number of weights still scales quadratically with the CNN's width. Our MGIC architectures replace each CNN block with an MGIC counterpart that utilizes a hierarchy of nested grouped convolutions of small group size to address this. Hence, our proposed architectures scale linearly with respect to the network's width while retaining full coupling of the channels as in standard CNNs. Our extensive experiments on image classification, segmentation, and point cloud classification show that applying this strategy to different architectures like ResNet and MobileNetV3 reduces the number of parameters while obtaining similar or better accuracy.

Recommendations

• MgNet: a unified framework of multigrid and convolutional neural network
• An Interpretive Constrained Linear Model for ResNet and MgNet
• scientific article; zbMATH DE number 7255154
• Multilevel Artificial Neural Network Training for Spatially Correlated Learning
• Structure optimization of convolutional neural networks: a survey

Cites work

• A Multigrid Tutorial, Second Edition
• A multilevel framework for sparse optimization with application to inverse covariance estimation and logistic regression
• Algebraic multigrid by smoothed aggregation for second and fourth order elliptic problems
• An efficient multigrid method for graph Laplacian systems. II: Robust aggregation
• DeepXDE: a deep learning library for solving differential equations
• fPINNs: Fractional Physics-Informed Neural Networks
• scientific article; zbMATH DE number 2068044 (Why is no real title available?)
• scientific article; zbMATH DE number 1843025 (Why is no real title available?)
• Layer-Parallel Training of Deep Residual Neural Networks
• Lean algebraic multigrid (LAMG): fast graph Laplacian linear solver
• MgNet: a unified framework of multigrid and convolutional neural network
• Multigrid Methods for PDE Optimization
• Multilevel Adaptive Aggregation for Markov Chains, with Application to Web Ranking
• Parallel time integration with multigrid
• Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
• Strong Solutions for PDE-Based Tomography by Unsupervised Learning
• SwitchNet: a neural Network Model for forward and inverse scattering problems

Cited in

(2)

• MgNet: a unified framework of multigrid and convolutional neural network
• Deep learning for PDE-based inverse problems. Abstracts from the workshop held October 27 -- November 1, 2024

This page was built for publication: MGIC: Multigrid-in-Channels Neural Network Architectures

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6108155)