Nonparametric Heterogeneity Testing For Massive Data

arXiv1601.06212MaRDI QIDQ6269625FDOQ6269625

Authors: Junwei Lu, Guang Cheng, Han Liu 

Publication date: 22 January 2016

Abstract: A massive dataset often consists of a growing number of (potentially) heterogeneous sub-populations. This paper is concerned about testing various forms of heterogeneity arising from massive data. In a general nonparametric framework, a set of testing procedures are designed to accommodate a growing number of sub-populations, denoted as $s$, with computational feasibility. In theory, their null limit distributions are derived as being nearly Chi-square with diverging degrees of freedom as long as $s$ does not grow too fast. Interestingly, we find that a lower bound on $s$ needs to be set for obtaining a sufficiently powerful testing result, so-called "blessing of aggregation." As a by-produc, a type of homogeneity testing is also proposed with a test statistic being aggregated over all sub-populations. Numerical results are presented to support our theory.

This page was built for publication: Nonparametric Heterogeneity Testing For Massive Data

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6269625)