The elements of statistical learning. Data mining, inference, and prediction (Q5943421): Difference between revisions

This book is designed for researchers and students in the fields of statistics, artificial intelligence, engineering, finance and others. Some of the most important learning methods with the underlying concepts are described. The approach is statistical, though written in a more intuitive fashion emphasizing concepts rather than mathematical details. The authors use the more modern language of machine learning. The dependent and independent variables in statistical literature are interchangeably called responses and inputs, respectively. The book starts with an overview of the supervisory learning problem, and discusses linear regressions and classifications. For a single predictor, splines, wavelets, penalization and kernel methods are used. Model assessment and selection are covered by concepts of bias, variance, overfitting and cross validation. Other topics include neural networks and show the working principle of vector mnachines. Model inference and averaging is based on maximum likelihood and Bayesian inference. The popular EM algorithm for simplifying difficult maximum likelihood problems is described in the context of a two-component mixture model. Specific methods for supervised learning assume a different structural form for the unknown regression function. Another focus is on boosting methods as those of the most powerful learning principles. Bibliographic notes giving background references for the material, as well as computational considerations and exercises are provided at the end of each chapter. The S-PLUS programming language is used. The website for this book is located at \url{http://www-stat.stanford.edu/ElemStatLearn}, which includes many of the datasets used. Contents: 1. Introduction; 2. Overview of supervised learning; 3. Linear methods for regression; 4. Linear methods for classification; 5. Basic expansions and regularization; 6. Kernel methods; 7. Model assessment and selection; 8. Model inference and averaging; 9. Additive models, trees, and related methods; 10. Boosting and additive trees; 11. Neural networks; 12. Support vector machines and flexible discriminants; 13. Prototype methods and nearest neighbors; 14. Unsupervised learning.