\texttt{fastMI}: a fast and consistent copula-based nonparametric estimator of mutual information

DOI10.1016/J.JMVA.2023.105270MaRDI QIDQ6200945zbMATH OpenFDO

Authors Soumik Purkayastha, Peter X.-K. Song

Publication date 25 March 2024

Published in Journal of Multivariate Analysis (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2212.10268

zbMATH Keywords

copulafast Fourier transformationkernel density estimationpermutation teststatistical dependence

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Estimation in multivariate analysis (62H12)

Abstract: As a fundamental concept in information theory, mutual information ($MI$) has been commonly applied to quantify the association between random vectors. Most existing nonparametric estimators of $MI$ have unstable statistical performance since they involve parameter tuning. We develop a consistent and powerful estimator, called exttt{fastMI}, that does not incur any parameter tuning. Based on a copula formulation, exttt{fastMI} estimates $MI$ by leveraging Fast Fourier transform-based estimation of the underlying density. Extensive simulation studies reveal that exttt{fastMI} outperforms state-of-the-art estimators with improved estimation accuracy and reduced run time for large data sets. exttt{fastMI} provides a powerful test for independence that exhibits satisfactory type I error control. Anticipating that it will be a powerful tool in estimating mutual information in a broad range of data, we develop an R package exttt{fastMI} for broader dissemination.

Cites work

• A consistent multivariate test of association based on ranks of distances
• A fast and objective multidimensional kernel density estimation method: fastKDE
• A Mathematical Theory of Communication
• Analyzing dependent data with vine copulas. A practical guide with R
• Bandwidth selection for kernel density estimation: a review of fully automatic selectors
• Dependence modeling with copulas
• Detecting novel associations in large data sets
• Divergence Estimation for Multidimensional Densities Via k-Nearest-Neighbor Distances
• Equitability, mutual information, and the maximal information coefficient
• Estimation of Entropy and Mutual Information
• scientific article; zbMATH DE number 3820814 (Why is no real title available?)
• scientific article; zbMATH DE number 1222288 (Why is no real title available?)
• scientific article; zbMATH DE number 1255541 (Why is no real title available?)
• scientific article; zbMATH DE number 646830 (Why is no real title available?)
• scientific article; zbMATH DE number 765034 (Why is no real title available?)
• Information Theoretical Analysis of Multivariate Correlation
• Jackknife approach to the estimation of mutual information
• Measuring and testing dependence by correlation of distances
• On the Amount of Information
• On the Estimation of the Probability Density, I
• Probit transformation for nonparametric kernel estimation of the copula density
• Relationship between Kendall's tau correlation and mutual information
• Selected Topics in Characteristic Functions
• Self-consistent method for density estimation
• The Cambridge dictionary of statistics
• The Hellinger Correlation

This page was built for publication: \texttt{fastMI}: a fast and consistent copula-based nonparametric estimator of mutual information

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6200945)