Conditional density estimation with dimensionality reduction via squared-loss conditional entropy minimization
From MaRDI portal
Abstract: Regression aims at estimating the conditional mean of output given input. However, regression is not informative enough if the conditional density is multimodal, heteroscedastic, and asymmetric. In such a case, estimating the conditional density itself is preferable, but conditional density estimation (CDE) is challenging in high-dimensional space. A naive approach to coping with high-dimensionality is to first perform dimensionality reduction (DR) and then execute CDE. However, such a two-step process does not perform well in practice because the error incurred in the first DR step can be magnified in the second CDE step. In this paper, we propose a novel single-shot procedure that performs CDE and DR simultaneously in an integrated way. Our key idea is to formulate DR as the problem of minimizing a squared-loss variant of conditional entropy, and this is solved via CDE. Thus, an additional CDE step is not needed after DR. We demonstrate the usefulness of the proposed method through extensive experiments on various datasets including humanoid robot transition and computer art.
Recommendations
- Direct conditional probability density estimation with sparse feature selection
- Dimension reduction and adaptation in conditional density estimation
- Converting high-dimensional regression to high-dimensional conditional density estimation
- Entropy-based sliced inverse regression
- scientific article; zbMATH DE number 4020220
Cites work
- A constructive approach to the estimation of dimension reduction directions
- Density-ratio matching under the Bregman divergence: a unified framework of density-ratio estimation
- Dual representation of \(\phi\)-divergences and applications.
- Estimating Divergence Functionals and the Likelihood Ratio by Convex Risk Minimization
- scientific article; zbMATH DE number 3252891 (Why is no real title available?)
- scientific article; zbMATH DE number 3322635 (Why is no real title available?)
- Kernel dimension reduction in regression
- On Information and Sufficiency
- Reinforcement learning. An introduction
- Robust and efficient estimation by minimising a density power divergence
- Robust Statistics
- Sliced Inverse Regression for Dimension Reduction
- Sliced Regression for Dimension Reduction
- Sufficient dimension reduction via Bayesian mixture modeling
- Sufficient Dimension Reduction via Inverse Regression
- Sufficient dimension reduction via squared-loss mutual information estimation
- The Geometry of Algorithms with Orthogonality Constraints
Cited in
(6)- Model-based reinforcement learning with dimension reduction
- Converting high-dimensional regression to high-dimensional conditional density estimation
- Sufficient dimension reduction via direct estimation of the gradients of logarithmic conditional densities
- Cross-domain metric and multiple kernel learning based on information theory
- Direct estimation of the derivative of quadratic mutual information with application in supervised dimension reduction
- Direct conditional probability density estimation with sparse feature selection
This page was built for publication: Conditional density estimation with dimensionality reduction via squared-loss conditional entropy minimization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5380192)
