A weight initialization based on the linear product structure for neural networks

DOI10.1016/J.AMC.2021.126722MaRDI QIDQ2247166zbMATH OpenOpenAlexFDO

Authors Qipin Chen, Wenrui Hao, Juncai He

Publication date 16 November 2021

Published in Applied Mathematics and Computation (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2109.00125

zbMATH Keywords

neural networksnonlinear computationlinear product structureweight initialization

Mathematics Subject Classification ID

Numerical analysis (65-XX) Probabilistic methods, stochastic differential equations (65Cxx) Nonlinear algebraic or transcendental equations (65Hxx)

Recommendations

• Structural, Syntactic, and Statistical Pattern Recognition
• Dying ReLU and initialization: theory and numerical examples
• Solving the linear interval tolerance problem for weight initialization of neural networks
• New Approach of Recurrent Neural Network Weight Initialization
• New weight learning strategy for fully connected feedforward neural networks

Cites work

• A gradient descent method for solving a system of nonlinear equations
• A homotopy training algorithm for fully connected neural networks
• An adaptive homotopy method for computing bifurcations of nonlinear parametric systems
• Dying ReLU and initialization: theory and numerical examples
• Numerically solving polynomial systems with Bertini
• The Numerical Solution of Systems of Polynomials Arising in Engineering and Science
• The Wiener--Askey Polynomial Chaos for Stochastic Differential Equations

Cited in

(11)

• Improved weight initialization for deep and narrow feedforward neural network
• Efficient Initialization of Artificial Neural Network Weights for Electrical Component Models
• Dying ReLU and initialization: theory and numerical examples
• A new initialization method for neural networks with weight sharing
• Global exponential stability for delayed Clifford-valued coupled neural networks with impulsive effects
• Structural, Syntactic, and Statistical Pattern Recognition
• A mathematical framework for improved weight initialization of neural networks using Lagrange multipliers
• A Weight Initialization Based on the Linear Product Structure for Neural Networks
• Solving the linear interval tolerance problem for weight initialization of neural networks
• Side effects of learning from low-dimensional data embedded in a Euclidean space
• Exponential synchronisation for delayed Clifford-valued coupled switched neural networks via static event-triggering rule

Describes a project that uses

Uses Software

• Adam
• ImageNet
• CIFAR
• MNIST
• Bertini

This page was built for publication: A weight initialization based on the linear product structure for neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2247166)