A generative model and a generalized trust region Newton method for noise reduction (Q2436691)

Principal component analysis (PCA) is the classical technique for extracting the linear low-dimensional structure from a given dataset, allowing to replace the initial variables by uncorrelated (orthogonal) ones called principal components. Intuitively speaking, the first \(k\) principal components determine the \(k\)-dimensional linear manifold that better fits the dataset, and they are computed by means of a closed formula providing the optimal solution to a suitable quadratic programming problem. When the underlying structure is nonlinear, as it happens in many practical applications (e.g., in machine learning, data mining or pattern recognition), where the user deals with noisy data lying near some low-dimensional manifold, the linear principal components must be replaced by nonlinear manifolds. This paper assumes a generative model for the noisy data and proposes a probabilistic approach for separating the data-generating nonlinear functions from noise. The corresponding numerical problem is solved by means of a generalized trust region Newton method, whose convergence is proved. Numerical experiments on synthetically generated datasets (by means of Gaussian kernels) are reported.