Exploiting Hessian matrix and trust-region algorithm in hyperparameters estimation of Gaussian process
Nonparametric Bayesian approach to Gaussian regression is considered. The training data \((x_i,t_i)_{i=1}^N\) consist of output \(t_i\in\mathbb{R}\) and input vectors \(x_i\in \mathbb{R}^L\). Their distribution is described by \[ \mathbf{P}(t| \;x,\Theta)\propto \exp\left( -{1\over 2}t^TC^{-1}(\vartheta)t \right), \] where \(C\) is a matrix with entries \[ c(x_i,x_j,\Theta)= \alpha\exp \left( -{1\over 2}\sum_{l=1}^L (x_i^{(l)}-x_j^{(l)})^2d_l \right) +\nu\delta_{ij}, \] \(\Theta=(\alpha,d_1,\dots,d_L,\nu)\) being the hyperparameter. It is proposed to estimate \(\Theta\) using maximum likelihood, i.e. to minimize the negative log-likelihood \(L(\Theta)={1\over 2}\log\det C(\Theta)+{1\over 2}t^TC^{-1}(\Theta)t\). The authors describe the form of the Hessian matrix for \(L\) and propose a second-order trust-region algorithm for the minimization of \(L\). Results of numerical simulations are presented.
- Approximate implementation of the logarithm of the matrix determinant in Gaussian process regression
- A matrix-free approach for solving the parametric Gaussian process maximum likelihood problem
- O(N2)-Operation Approximation of Covariance Matrix Inverse in Gaussian Process Regression Based on Quasi-Newton BFGS Method
- Maximum likelihood estimation and uncertainty quantification for Gaussian process approximation of deterministic functions
- Properties of the Bayesian parameter estimation of a regression based on Gaussian processes
- A fast Newton method for entropy maximization in statistical phase estimation
- Analysing Data from Hormone-Receptor Assays
- Approximate solution of the trust region problem by minimization over two-dimensional subspaces
- Bayesian learning for neural networks
- Bayesian regression and classification using mixtures of Gaussian processes
- Computing a Trust Region Step
- Global Convergence of a Class of Trust Region Algorithms for Optimization with Simple Bounds
- scientific article; zbMATH DE number 3928227 (Why is no real title available?)
- scientific article; zbMATH DE number 88930 (Why is no real title available?)
- scientific article; zbMATH DE number 3583145 (Why is no real title available?)
- scientific article; zbMATH DE number 1222285 (Why is no real title available?)
- scientific article; zbMATH DE number 961607 (Why is no real title available?)
- Maximum likelihood estimation of models for residual covariance in spatial regression
- Multilayer feedforward networks are universal approximators
- The Conjugate Gradient Method and Trust Regions in Large Scale Optimization
- Improved gradient-based neural networks for online solution of Lyapunov matrix equation
- Discrete-time noise-tolerant Zhang neural network for dynamic matrix pseudoinversion
- Time-varying minimum-cost portfolio insurance problem via an adaptive fuzzy-power LVI-PDNN
- Improved neural solution for the Lyapunov matrix equation based on gradient search
- Particle swarm optimization for adaptive hyper-parameters acquisition of Gaussian process regression
- Approximate implementation of the logarithm of the matrix determinant in Gaussian process regression
- O(N2)-Operation Approximation of Covariance Matrix Inverse in Gaussian Process Regression Based on Quasi-Newton BFGS Method
- On exponential convergence of nonlinear gradient dynamics system with application to square root finding
- Complex-valued Zhang neural network for online complex-valued time-varying matrix inversion
- A non-intrusive solution to the ill-conditioning problem of the gradient-enhanced Gaussian covariance matrix for Gaussian processes
- Zhang neural network for online solution of time-varying convex quadratic program subject to time-varying linear-equality constraints
- Efficient gradient-enhanced Bayesian optimizer with comparisons to conjugate-gradient and quasi-Newton optimizers for unconstrained local optimization
- Global exponential convergence and stability of gradient-based neural network for online matrix inversion
- Kriging-based infill sampling criterion for constraint handling in multi-objective optimization
This page was built for publication: Exploiting Hessian matrix and trust-region algorithm in hyperparameters estimation of Gaussian process
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2491022)
