Data reduction via adaptive sampling.

DOI10.4310/CIS.2002.V2.N1.A3MaRDI QIDQ1421047zbMATH OpenOpenAlexFDO

Authors Xiaobai Li

Publication date 22 March 2004

Published in Communications in Information and Systems (Search for Journal in Brave)

Full work available at URL https://doi.org/10.4310/cis.2002.v2.n1.a3

Mathematics Subject Classification ID

Searching and sorting (68P10) Database theory (68P15) Problem solving in the context of artificial intelligence (heuristics, search strategies, etc.) (68T20) Quadratic programming (90C20)

Review text

Data reduction is an important issue in the field of data mining. This article describes a new method for selecting a subset of data from a large dataset. A simplified chi-square criterion is proposed for measuring the goodness-of-fit between the distributions of the reduced and full data sets. Under this criterion, the data reduction problem can be formulated as a binary quadratic program and a tabu search technique is used in the search/optimization process. The procedure is adaptive in that it involves not only random sampling but also deterministic search guided by the results of the previous search steps. The method is applicable primarily to discrete data, but can be extended to continuous data as well. An experimental study that compares the proposed method with simple random sampling on a number of simulated and real world datasets has been conducted. The results of the study indicate that the distributions of the samples produced by the proposed method are significantly closer to the true distribution than those of random samples.

Recommendations

• Adaptive data reduction for large-scale transaction data
• Enhancing Density-Based Data Reduction Using Entropy
• Application of agent-based simulated annealing and tabu search procedures to solving the data reduction problem
• An algorithm for reducing the dimension and size of a sample for data exploration procedures
• Optimization-based feature selection with adaptive instance sampling

Cited in

(8)

• Adaptive data reduction for large-scale transaction data
• Application of agent-based simulated annealing and tabu search procedures to solving the data reduction problem
• Sampling to maintain approximate probability distribution under chi-square test
• Comparing representative selection strategies for dissimilarity representations
• Cherry-picking for complex data: robust structure discovery
• Enhancing Density-Based Data Reduction Using Entropy
• An algorithm for reducing the dimension and size of a sample for data exploration procedures
• Eigen-entropy: a metric for multivariate sampling decisions

This page was built for publication: Data reduction via adaptive sampling.

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1421047)