Multivariate and functional robust fusion methods for structured big data

DOI10.1016/J.JMVA.2018.06.012MaRDI QIDQ1733276zbMATH OpenOpenAlexWikidataFDO

Authors C. Aaron, Alejandro Cholaquidis, Badih Ghattas, Ricardo Fraiman

Publication date 21 March 2019

Published in Journal of Multivariate Analysis (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1804.01858

zbMATH Keywords

big dataclusteringfunctional datarobustness

Mathematics Subject Classification ID

Nonparametric robustness (62G35) Measures of association (correlation, canonical correlation, etc.) (62H20) Classification and discrimination; cluster analysis (statistical aspects) (62H30)

Abstract: We address one of the important problems in Big Data, namely how to combine estimators from different subsamples by robust fusion procedures, when we are unable to deal with the whole sample. We propose a general framework based on the classic idea of `divide and conquer'. In particular we address in some detail the case of a multivariate location and scatter matrix, the covariance operator for functional data, and clustering problems.

Review text

The robust fusion method is proposed to implement the divide-and-combine technique for robust estimation of multivariate location and scatter matrix and clustering. The efficiency of the designed approach is demonstrated on three problems including the Kaggle MNIST images of handwritten digits clustering problem.

Recommendations

• Fast recursive and efficient algorithms for estimating the functional median and robust clustering in large dimension
• Computational connections between robust multivariate analysis and clustering
• Robust estimation of multivariate location and scatter in the presence of cellwise and casewise contamination
• \(i\)Fusion: individualized fusion learning
• Robust estimation of multivariate location and shape

Cites work

• A General Qualitative Definition of Robustness
• A partial overview of the theory of statistics with functional data
• An introduction to recent advances in high/infinite dimensional statistics
• Best approximations to random variables based on trimming procedures
• Big and complex data analysis. Methodologies and applications
• Functional statistics and related fields. Selected papers based on the presentations at the 4th international workshop on functional and operational statistics, IWFOS, Corunna, Spain, June 15--17, 2017
• scientific article; zbMATH DE number 3511563 (Why is no real title available?)
• scientific article; zbMATH DE number 3336465 (Why is no real title available?)
• Inference for functional data with applications
• Nonparametric functional data analysis. Theory and practice.
• On a Geometric Notion of Quantiles for Multivariate Data
• Robust Statistics
• Robust Statistics
• Statistical methods and computing for big data
• The multivariate L ₁ -median and associated data depth
• The spatial distribution in infinite dimensional spaces and related quantiles and depths
• Trimmed \(k\)-means: An attempt to robustify quantizers
• Uniformity in weak convergence

Cited in

(5)

• Recent advances in functional data analysis and high-dimensional statistics
• Model-assisted estimators with auxiliary functional data
• \(i\)Fusion: individualized fusion learning
• ibk-means: an iterative batch k-means algorithm for big data clustering.
• Reduced analytic dependency modeling: robust fusion for visual recognition

Describes a project that uses

Uses Software

• robustbase

This page was built for publication: Multivariate and functional robust fusion methods for structured big data

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1733276)