Ill-Conditioning in Neural Network Training Problems

DOI10.1137/0914044MaRDI QIDQ4695054zbMATH OpenOpenAlexFDO

Authors Sirpa Saarinen, Randall Bramley, George Cybenko

Publication date 18 June 1995

Published in SIAM Journal on Scientific Computing (Search for Journal in Brave)

Full work available at URL https://semanticscholar.org/paper/a62f69f42d0bade0980419741d4eea1fc7060c49

zbMATH Keywords

experimental resultsfeedforward neural networksill-conditioningneural network trainingnonlinear parameter estimationoptimization methods

Mathematics Subject Classification ID

Numerical optimization and variational techniques (65K10) General systems theory (93A99)

Recommendations

• Variable projections neural network training
• A comparison of nonlinear optimization methods for supervised learning in multilayer feedforward neural networks
• Analysis of gradient descent learning algorithms for multilayer feedforward neural networks
• On the infeasibility of training neural networks with small mean-squared error
• The computational intractability of training sigmoidal neural networks

Cited in

(18)

• A homotopy method for training neural networks
• Feedforward neural nets as discretization schemes for ODEs and DAEs
• Approximation properties of local bases assembled from neural network transfer functions
• OCReP: an optimally conditioned regularization for pseudoinversion based neural training
• Nonlinear black-box modeling in system identification: A unified overview
• Second-order stagewise backpropagation for Hessian-matrix analyses and investigation of negative curvature
• ADAPTIVE ALGORITHMS FOR NEURAL NETWORK SUPERVISED LEARNING: A DETERMINISTIC OPTIMIZATION APPROACH
• scientific article; zbMATH DE number 1191258 (Why is no real title available?)
• scientific article; zbMATH DE number 1203361 (Why is no real title available?)
• Regularization tools for training large feed-forward neural networks using automatic differentiation^∗
• scientific article; zbMATH DE number 1133202 (Why is no real title available?)
• Overtraining, regularization and searching for a minimum, with application to neural networks
• Two Frameworks for Improving Gradient-Based Learning Algorithms
• Improving neural network training solutions using regularisation
• Error estimates and physics informed augmentation of neural networks for thermally coupled incompressible Navier Stokes equations
• Degree-of-freedom and optimization-dynamic effects on the observability of Kuramoto-Sivashinsky systems
• Variable projections neural network training
• Establishing criteria to ensure successful feedforward artificial neural network modelling of mechanical systems

This page was built for publication: Ill-Conditioning in Neural Network Training Problems

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4695054)