summarisebig (Q7370234)
From MaRDI portal
!
This is the item page for this Wikibase entity, intended for internal use and editing purposes. Please use the normal view instead:
Grouped Summaries for Large 'arrow' Datasets
| Language | Label | Description | Also known as |
|---|---|---|---|
| default for all languages | No label defined |
||
| English | summarisebig |
Grouped Summaries for Large 'arrow' Datasets |
Statements
Provides a 'dplyr'-like interface for grouped summaries on large 'arrow' datasets. Complete summaries are attempted in 'arrow' first. When a result can be reconstructed from 'arrow'-computable sufficient statistics, an explicit MapReduce-style reduction and 'R' finalization strategy is available. For arbitrary 'R' functions that require raw group observations, complete groups are materialized in bounded chunks, with optional parallel execution and shared-memory processing. The MapReduce strategy follows the programming model described by Dean and Ghemawat (2008) <doi:10.1145/1327452.1327492>.
0 references
21 September 2026
0 references
