Sequential category aggregation and partitioning approaches for multi-way contingency tables based on survey and census data

DOI10.1214/08-AOAS175zbMATH Open1149.62049arXiv0811.1686MaRDI QIDQ958326FDOQ958326

Authors: L. Fraser Jackson, Alistair G. Gray, Stephen E. Fienberg 

Publication date: 3 December 2008

Published in: The Annals of Applied Statistics (Search for Journal in Brave)

Abstract: Large contingency tables arise in many contexts but especially in the collection of survey and census data by government statistical agencies. Because the vast majority of the variables in this context have a large number of categories, agencies and users need a systematic way of constructing tables which are summaries of such contingency tables. We propose such an approach in this paper by finding members of a class of restricted log-linear models which maximize the likelihood of the data and use this to find a parsimonious means of representing the table. In contrast with more standard approaches for model search in hierarchical log-linear models (HLLM), our procedure systematically reduces the number of categories of the variables. Through a series of examples, we illustrate the extent to which it can preserve the interaction structure found with HLLMs and be used as a data simplification procedure prior to HLL modeling. A feature of the procedure is that it can easily be applied to many tables with millions of cells, providing a new way of summarizing large data sets in many disciplines. The focus is on information and description rather than statistical testing. The procedure may treat each variable in the table in different ways, preserving full detail, treating it as fully nominal, or preserving ordinality.

Full work available at URL: https://arxiv.org/abs/0811.1686

Recommendations

• Automatic aggregation of categories in multivariate contingency tables using information theory.
• Decomposition and model selection for large contingency tables
• Hierarchical subspace models for contingency tables
• Model Determination for Categorical Data With Factor Level Merging
• scientific article

zbMATH Keywords

Kullback-Leibler distancelog-linear modelingcollapsibilitySimpson's paradoxlevel mergingpartitioning informationreducing dimensionality

Mathematics Subject Classification ID

Statistical aspects of information-theoretic topics (62B10) Contingency tables (62H17) Sequential statistical methods (62L99)

Cites Work

• Elements of statistical disclosure control
• Three centuries of categorical data analysis: Log-linear models and maximum likelihood estima\-tion
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Model Determination for Categorical Data With Factor Level Merging
• Title not available (Why is that?)
• Title not available (Why is that?)
• The Analysis of Multidimensional Contingency Tables: Stepwise Procedures and Direct Estimation Methods for Building Models for Multiple Classifications
• Ignorability and coarse data
• Title not available (Why is that?)
• Title not available (Why is that?)
• Ignorability for categorical data
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Algebraic statistics and contingency table problems: log-linear models, likelihood estimatio, and disclosure limitation
• Collapsibility and response variables in contingency tables
• The Analysis of Cross-Classified Data: Independence, Quasi-Independence, and Interactions in Contingency Tables with or Without Missing Entries
• A Single General Method for the Analysis of Cross-Classified Data: Reconciliation and Synthesis of Some Methods of Pearson, Yule, and Fisher, and Also Some Methods of Correspondence Analysis and Association Analysis
• On the Application of Conditional Independence to Ordinal Data
• THE COMBINATION OF PROBABILITIES ARISING FROM DATA IN DISCRETE DISTRIBUTIONS

Cited In (1)

• Decomposition and model selection for large contingency tables

Uses Software

• DIGRAM

This page was built for publication: Sequential category aggregation and partitioning approaches for multi-way contingency tables based on survey and census data

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q958326)