Statistical analysis of kernel-based least-squares density-ratio estimation
Assume that two independent sets of independent and identically distributed samples are drawn from populations with respective density function \(q(x)\) and \(p(x)\). This paper considers the estimation of the ratio of \(q(x)\) and \(p(x)\) based on these two samples. Least-square approach to the estimation proposed by \textit{T. Kanamori}, \textit{S. Hido} and \textit{M. Sugiyama} [J. Mach. Learn. Res. 10, 1391--1445 (2009; Zbl 1235.62039)] is first reviewed in the paper. This approach assumes that the density ratio estimate can be expressed as a linear combination of several known base functions with the coefficients estimated minimizing squared errors plus a regularization term. This paper assumes that the model for density ratio is a reproducing kernel Hilbert space endowed with a kernel function and proposes that the density ratio estimate can be obtained by also minimizing the squared error plus a regularization term. Analytical solution to the minimization problem is derived and a leave-one-out cross-validation method is proposed to find an optimal value of regularization parameter. The convergence rate of the proposed estimate is established and connections with other types of kernel-based estimates are explored. Numerical experiments are performed to evaluate the computational efficiency and statistical performance of the proposed approach.
- On direct kernel estimator of density ratio
- Asymptotic properties of a kernel type estimator of a density ratio
- Density estimation in the two-sample problem with likelihood ratio ordering
- Density Estimation in RKHS with Application to Korobov Spaces in High Dimensions
- The Kernel distribution estimator of functions of random variables
- On kernel estimators of density ratio
- scientific article; zbMATH DE number 758461
- Nonparametric Density Estimation from Biased Data with Unknown Biasing Function
- Integrated squared error of kernel-type estimator of distribution function
- A fast dual algorithm for kernel logistic regression
- A least-squares approach to direct importance estimation
- Computational complexity of kernel-based density-ratio estimation: a condition number analysis
- Consistency of Support Vector Machines and Other Regularized Kernel Classifiers
- Convergence rates of generalization errors for margin-based classification
- Convexity, Classification, and Risk Bounds
- Covariate shift adaptation by importance weighted cross validation
- Density ratio estimation in machine learning. Foreword by Thomas G. Dietterich
- Direct importance estimation for covariate shift adaptation
- Discriminative learning under covariate shift
- Estimating Divergence Functionals and the Likelihood Ratio by Convex Risk Minimization
- scientific article; zbMATH DE number 5759476 (Why is no real title available?)
- scientific article; zbMATH DE number 5654889 (Why is no real title available?)
- scientific article; zbMATH DE number 3933857 (Why is no real title available?)
- scientific article; zbMATH DE number 1332320 (Why is no real title available?)
- scientific article; zbMATH DE number 3252891 (Why is no real title available?)
- scientific article; zbMATH DE number 3322635 (Why is no real title available?)
- scientific article; zbMATH DE number 3383043 (Why is no real title available?)
- scientific article; zbMATH DE number 961607 (Why is no real title available?)
- Improving predictive inference under covariate shift by weighting the log-likelihood function
- Inferences for case-control and semiparametric two-sample density ratio models
- Input-dependent estimation of generalization error under covariate shift
- Linear and nonlinear programming.
- Nonparametric and semiparametric models.
- On the influence of the kernel on the consistency of support vector machines
- On the mathematical foundations of learning
- Remarks on Some Nonparametric Estimates of a Density Function
- Soft margins for AdaBoost
- Some results on Tchebycheffian spline functions and stochastic processes
- Sparseness vs estimating conditional probabilities: some asymptotic results
- Theory of Reproducing Kernels
- Direct importance estimation for covariate shift adaptation
- Dimensionality reduction for density ratio estimation in high-dimensional spaces
- Density-ratio matching under the Bregman divergence: a unified framework of density-ratio estimation
- Computational complexity of kernel-based density-ratio estimation: a condition number analysis
- Least-squares two-sample test
- Semiparametric optimal estimation with nonignorable nonresponse data
- On a regularization of unsupervised domain adaptation in RKHS
- Numerical study of learning algorithms on Stiefel manifold
- Change-point detection in time-series data by relative density-ratio estimation
- A least-squares approach to direct importance estimation
- Density ratio estimation in machine learning. Foreword by Thomas G. Dietterich
- Statistical analysis of distance estimators with density differences and density ratios
- Imitation learning as \(f\)-divergence minimization
- Some heuristics of kernel based estimators of ratio functions
- Semi-supervised learning with density-ratio estimation
- Kernel and Probit Estimates in Quantal Bioassay
- Mode-seeking clustering and density ridge estimation via direct estimation of density-derivative-ratios
- Semi-supervised learning of class balance under class-prior change by distribution matching
- Least-squares independence regression for non-linear causal inference under non-Gaussian noise
- Rejoinder: Learning Optimal Distributionally Robust Individualized Treatment Rules
- Information-theoretic representation learning for positive-unlabeled classification
- Variable selection for nonparametric learning with power series kernels
- Density-difference estimation
- Online direct density-ratio estimation applied to inlier-based outlier detection
- Density estimation in the two-sample problem with likelihood ratio ordering
- On kernel estimators of density ratio
- Direct density-ratio estimation with dimensionality reduction via least-squares hetero-distributional subspace search
- Distributed computation for marginal likelihood based model choice
- Learning under nonstationarity: covariate shift and class-balance change
- The impact of smoothness of kernels and target functions on unsupervised covariate shift adaptation in RKHS
- On recovering the Radon-Nikodym derivative under the big data assumption
- F-measure maximizing logistic regression
- Generative conformal prediction under multiple covariate shifts
- Machine learning with squared-loss mutual information
- Model-based policy gradients with parameter-based exploration by least-squares conditional density estimation
- Analytical plug-in method for kernel density estimator applied to genetic neutrality study
This page was built for publication: Statistical analysis of kernel-based least-squares density-ratio estimation
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q420923)
