Continuous optimization for data science (Q6950476)

This book presents the theory and algorithms of continuous optimization, with emphasis on applications to data science. The application of nonlinear optimization in this field is particularly valuable, since data science is a rapidly emerging area of research.\N\NThe book is divided into three parts:\N\NPart I consists of two chapters.\N\N\(\bullet\) Chapter 1 addresses optimization problems in \(\mathbb{R}\). It discusses necessary and sufficient optimality conditions, the minimization of convex and quasiconvex functions, and presents maximum likelihood estimation for distribution functions with one parameter as an example of a data science problem corresponding to a one-dimensional optimization task. Classical numerical solution methods are also included.\N\N\(\bullet\) Chapter 2 focuses on the minimization of functions in \(\mathbb{R}^n\). Convexity again plays a central role and is thoroughly examined, together with first- and second-order necessary conditions. Applications include maximum likelihood estimation for multi-parametric distributions such as the normal distribution, as well as linear neural networks.\N\NPart II contains three chapters devoted to constrained nonlinear optimization in data science.\N\N\(\bullet\) Chapter 3 considers the minimization of functions under linear constraints. It begins with two classes of problems: allocation and queueing models. Then, the author introduces the Karush-Kuhn-Tucker (KKT) optimality conditions and apply them to least squares problems with linear constraints and linear programming models.\N\N\(\bullet\) Chapter 5 addresses general equality constraints. Examples such as service capacity distribution, regression, and queueing problems are revisited. Duality in nonlinear programs with equality constraints is presented, and the linear case is analyzed to derive the dual form.\N\NPart III is dedicated to the numerical solution of linear programs. It covers the classical simplex method and examines solution stability under changes in the objective function. The dual simplex method is highlighted as an important hybrid algorithm, particularly useful in applications where additional constraints are introduced and the standard simplex method becomes inefficient.\N\NOverall, this book provides a foundation in optimization and its applications to data science. To complement its coverage, readers should also study the dual simplex method, which is well-documented in classical linear programming texts. The book is recommended for advanced undergraduate and master's students, and serves as a useful guide for professors designing courses for data science students, as it offers concrete examples of optimization models. Researchers beginning in data science will also find in this book a smooth introduction to optimization and its applications across different areas.