Sequential estimation of Spearman rank correlation using Hermite series estimators

DOI10.1016/J.JMVA.2021.104783zbMATH Open1504.62079arXiv2012.06287OpenAlexW3179495697MaRDI QIDQ89438FDOQ89438

Authors: Michael Stephanou, Melvin Varughese, Michael Stephanou, Melvin M. Varughese 

Publication date: November 2021

Published in: Journal of Multivariate Analysis (Search for Journal in Brave)

Abstract: In this article we describe a new Hermite series based sequential estimator for the Spearman rank correlation coefficient and provide algorithms applicable in both the stationary and non-stationary settings. To treat the non-stationary setting, we introduce a novel, exponentially weighted estimator for the Spearman rank correlation, which allows the local nonparametric correlation of a bivariate data stream to be tracked. To the best of our knowledge this is the first algorithm to be proposed for estimating a time varying Spearman rank correlation that does not rely on a moving window approach. We explore the practical effectiveness of the Hermite series based estimators through real data and simulation studies demonstrating good practical performance. The simulation studies in particular reveal competitive performance compared to an existing algorithm. The potential applications of this work are manifold. The Hermite series based Spearman rank correlation estimator can be applied to fast and robust online calculation of correlation which may vary over time. Possible machine learning applications include, amongst others, fast feature selection and hierarchical clustering on massive data sets.

Full work available at URL: https://arxiv.org/abs/2012.06287

Recommendations

• Sequential quantiles via Hermite series density estimation
• On the properties of Hermite series based distribution function estimators
• Influence functions of the Spearman and Kendall correlation measures
• Nonparametric estimation of Spearman's rank correlation with bivariate survival data
• ESTIMATES OF STANDARD DEVIATION OF SPEARMAN'S RANK CORRELATION COEFFICIENTS WITH DEPENDENT OBSERVATIONS

zbMATH Keywords

\(O(1)\) update algorithmHermite series estimatorsincremental estimationnonparametric correlationonline estimationsequential estimationSpearman correlation coefficientstream data

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Measures of association (correlation, canonical correlation, etc.) (62H20) Sequential estimation (62L12)

Cites Work

• Sequential quantiles via Hermite series density estimation
• Pointwise consistency of the Hermite series density estimate
• Properties of Hermite series estimation of probability density
• Recursive estimation of fourth-order cumulants with application to identification.
• Recursive density estimators based on Robbins-Monro's scheme and using Bernstein polynomials
• On the properties of Hermite series based distribution function estimators
• The stochastic approximation method for estimation of a distribution function
• Influence functions of the Spearman and Kendall correlation measures
• Numerically stable, scalable formulas for parallel and online computation of higher-order multivariate central moments with arbitrary weights
• Title not available (Why is that?)
• Nonparametric System Identification
• Title not available (Why is that?)
• Updating mean and variance estimates
• Title not available (Why is that?)
• Title not available (Why is that?)
• Error bounds for the asymptotic expansion of the Hurwitz zeta function
• Recursive distribution estimator defined by stochastic approximation method using Bernstein polynomials
• High-frequency covariance estimates with noisy and asynchronous financial data
• Comparison of several algorithms for computation of means, standard deviations and correlation coefficients
• Estimation of Probability Density by an Orthogonal Series
• Adaptive estimation of the fourth-order cumulant of a white stochastic process
• Hermite series estimates of a probability density and its derivatives
• Hermite series estimators for probability densities

Cited In (3)

• Hermiter: \textbf{R} package for sequential nonparametric estimation
• On the properties of Hermite series based distribution function estimators
• hermiter

Uses Software

• R
• hermiter

This page was built for publication: Sequential estimation of Spearman rank correlation using Hermite series estimators

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q89438)