Annealing stochastic approximation Monte Carlo algorithm for neural network training

DOI10.1007/s10994-007-5017-7zbMath1470.68135OpenAlexW2170189046MaRDI QIDQ2384162

Faming Liang

Publication date: 20 September 2007

Published in: Machine Learning (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1007/s10994-007-5017-7

zbMATH Keywords

stochastic approximationMarkov chain Monte Carlosimulated annealingconvergence rateback-propagationWang-Landau algorithmmultiple-layer perceptron

Mathematics Subject Classification ID

Monte Carlo methods (65C05) Learning and adaptive systems in artificial intelligence (68T05)

Related Items

Stochastic approximation Hamiltonian Monte Carlo, Bayesian multiple change-point estimation with annealing stochastic approximation Monte Carlo, Parallel and interacting stochastic approximation annealing algorithms for global optimisation, Improving inductive logic programming by using simulated annealing, Multiple change-point detection of multivariate mean vectors with the Bayesian approach, Annealing evolutionary stochastic approximation Monte Carlo for global optimization, On the use of stochastic approximation Monte Carlo for Monte Carlo integration, Simulated Stochastic Approximation Annealing for Global Optimization With a Square-Root Cooling Schedule, Annealing evolutionary stochastic approximation Monte Carlo for global optimization

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Optimization by Simulated Annealing
• Markov chains and stochastic stability
• Asymptotics of the spectral gap with applications to the theory of simulated annealing
• Multilayer feedforward networks are universal approximators
• Convergence of a stochastic approximation version of the EM algorithm
• On approximating weighted sums with exponentially many terms
• Bayesian learning for neural networks
• Rates of convergence of the Hastings and Metropolis algorithms
• Maximum Likelihood Estimation for Spatial Models by Markov Chain Monte Carlo Stochastic Approximation
• Population-Based Reversible Jump Markov Chain Monte Carlo
• Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
• A stochastic approximation algorithm with Markov chain Monte-Carlo method for incomplete data estimation problems
• An Algorithm for Least-Squares Estimation of Nonlinear Parameters
• Multivariate stochastic approximation using a simultaneous perturbation gradient approximation
• Variable Metric Method for Minimization
• Universal approximation bounds for superpositions of a sigmoidal function
• Dynamic weighting in Monte Carlo and optimization
• Two Timescale Analysis of the Alopex Algorithm for Optimization
• Real-Parameter Evolutionary Monte Carlo With Applications to Bayesian Mixture Models
• Evidence Evaluation for Bayesian Neural Networks Using Contour Monte Carlo
• An Effective Bayesian Neural Network Classifier with a Comparison Study to Support Vector Machine
• An Algorithm of Supervised Learning for Multilayer Neural Networks
• Stochastic Approximation in Monte Carlo Computation
• Stability of Stochastic Approximation under Verifiable Conditions
• A Rapidly Convergent Descent Method for Minimization
• Learning representations by back-propagating errors
• A Family of Variable-Metric Methods Derived by Variational Means
• The Convergence of a Class of Double-rank Minimization Algorithms
• A new approach to variable metric algorithms
• Monte Carlo sampling methods using Markov chains and their applications
• The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations
• Conditioning of Quasi-Newton Methods for Function Minimization
• A Generalized Wang–Landau Algorithm for Monte Carlo Computation
• A Useful Convergence Theorem for Probability Distributions
• A Stochastic Approximation Method
• A method for the solution of certain non-linear problems in least squares
• Approximation by superpositions of a sigmoidal function
• The elements of statistical learning. Data mining, inference, and prediction
• Stochastic approximation algorithms: overview and recent trends.