Bayesian learning for neural networks (Q1922287)

The book explores the Bayesian approach to learning flexible statistical models based on what is known as neural networks. The aim of the reported work is to show that the Bayesian approach to learning these models can yield theoretical insights and can be useful in practice as well. In Chapter 1, the Bayesian framework for learning, the examined neural network models, Markov chain Monte Carlo methods and the major themes of the book are introduced. Chapter 2 defines classes of prior distributions for network parameters that reach sensible limits as the size of the network goes to infinity. The properties of prior distributions are examined focusing on the limit as the number of hidden units in the network goes to infinity. The aim is to show that reasonable priors for such infinite networks can be defined so that one can select an appropriate prior for a particular problem. Chapter 3 addresses the computational problems of producing predictions based on Bayesian neural network models. Such predictions involve integrations over the posterior distribution of network parameters estimated by using a Markov chain Monte Carlo method based on the hybrid Monte Carlo algorithm. The hybrid Monte Carlo method makes use of complex Bayesian network models possible in practice, though the computation time required can still be substantial. In Chapter 4, the author evaluates how good the predictions of Bayesian neural network models are. He demonstrates that Bayesian inference does not require limiting the complexity of the model. He also evaluates the effectiveness of hierarchical models, in particular the automatic relevance determination model. Tests on real data sets demonstrate that the Bayesian approach, implemented using hybrid Monte Carlo, can be effectively applied to problems of moderate size. Chapter 5 summarises the contributions of the work, draws some conclusions and indicates possible directions for future research. Appendices A and B give details of the implementation of the Bayesian neural network described in Chapter 3 and the way to obtain the implementation software. Bibliography and Index close the book.