Statistical analysis of kernel-based least-squares density-ratio estimation (Q420923)

Assume that two independent sets of independent and identically distributed samples are drawn from populations with respective density function \(q(x)\) and \(p(x)\). This paper considers the estimation of the ratio of \(q(x)\) and \(p(x)\) based on these two samples. Least-square approach to the estimation proposed by \textit{T. Kanamori}, \textit{S. Hido} and \textit{M. Sugiyama} [J. Mach. Learn. Res. 10, 1391--1445 (2009; Zbl 1235.62039)] is first reviewed in the paper. This approach assumes that the density ratio estimate can be expressed as a linear combination of several known base functions with the coefficients estimated minimizing squared errors plus a regularization term. This paper assumes that the model for density ratio is a reproducing kernel Hilbert space endowed with a kernel function and proposes that the density ratio estimate can be obtained by also minimizing the squared error plus a regularization term. Analytical solution to the minimization problem is derived and a leave-one-out cross-validation method is proposed to find an optimal value of regularization parameter. The convergence rate of the proposed estimate is established and connections with other types of kernel-based estimates are explored. Numerical experiments are performed to evaluate the computational efficiency and statistical performance of the proposed approach.