Results of KROWN: Knowledge Graph Construction Benchmark
DOI10.5281/zenodo.10973892Zenodo10973892MaRDI QIDQ6720606FDOQ6720606
Dataset published at Zenodo repository.
David Chaves-Fraga, Dylan van Assche, Anastasia Dimou
Publication date: 17 April 2024
Copyright license: Creative Commons Attribution 4.0 International
In this Zenodo repository we present the results of using KROWN to benchmark popular RDF Graph Materialization systems such as RMLMapper, RMLStreamer, Morph-KGC, SDM-RDFizer, and Ontop (in materialization mode). What is KROWN 👑? KROWN 👑 is a benchmark for materialization systems to construct Knowledge Graphs from (semi-)heterogeneous data sources using declarative mappings such as RML. Many benchmarks already exist for virtualization systems e.g. GTFS-Madrid-Bench, NPD, BSBM which focus on complex queries with a single declarative mapping. However, materialization systems are unaffected by complex queries since their input is the dataset and the mappings to generate a Knowledge Graph. Some specialized datasets exist to benchmark specific limitations of materialization systems such as duplicated or empty values in datasets e.g. GENOMICS, but they do not cover all aspects of materialization systems. Therefore, it is hard to compare materialization systems among each other in general which is where KROWN 👑 comes in! Results The raw results are available as ZIP archives, the analysis of the results are available in the spreadsheet results.ods. Evaluation setup We generated several scenarios using KROWNs data generator and executed them 5 times with KROWNs execution framework. All experiments were performed on Ubuntu 22.04 LTS machines (Linux 5.15.0, x86_64) with each Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz, 48 GB RAM memory, and 2 GB swap memory. The output of each materialization system was set to N-Triples. Materialization systems We selected the most popular maintained materialization systems for constructing RDF graphs for performing our experiments with KROWN: RMLMapper RMLStreamer Morph-KGC SDM-RDFizer OntopM (Ontop in materialization mode) Note: KROWN is flexible and allows adding any other materialization system, see KROWNs execution framework documentation for more information. Scenarios We consider the following scenarios: Raw data: number of rows, columns and cell size Duplicates empty values: percentage of the data containing duplicates or empty values Mappings: Triples Maps (TM), Predicate Object Maps (POM), Named Graph Maps (NG). Joins: relations (1-N, N-1, N-M), conditions, and duplicates during joins Note: KROWN is flexible and allows adding any other scenario, see KROWNs data generator documentation for more information. In the table below we list all parameter values we used to configure our scenarios: Scenario Parameter values Raw data: rows 10K, 100K, 1M, 10M Raw data: columns 1, 10, 20, 30 Raw data: cell size 500, 1K, 5K, 10K Duplicates: percentage 0%, 25%, 50%, 75%, 100% Empty values: percentage 0%, 25%, 50%, 75%, 100% Mappings: TMs + 5POMs 1, 10, 20, 30 TMs Mappings: 20TMs + POMs 1, 3, 5, 10 POMs Mappings: NG in SM 1, 5, 10, 15 NGs Mappings: NG in POM 1, 5, 10, 15 NGs Mappings: NG in SM/POM 1/1, 5/5, 10/10, 15/15 NGs Joins: 1-N relations 1-1, 1-5, 1-10, 1-15 Joins: N-1 relations 1-1, 5-1, 10-1, 15-1 Joins: N-M relations 3-3, 3-5, 5-3, 10-5, 5-10 Joins: join conditions 1, 5, 10, 15 Joins: join duplicates 0, 5, 10, 15
This page was built for dataset: Results of KROWN: Knowledge Graph Construction Benchmark
