Billion Triple Challenge (BTC) 2019 Dataset
DOI10.5281/zenodo.2634588Zenodo2634588MaRDI QIDQ6724706FDOQ6724706
Dataset published at Zenodo repository.
Tobias Käfer, Jose Miguel Herrera, Aidan Hogan
Publication date: 10 April 2019
Copyright license: Creative Commons Attribution 4.0 International
The Billion Triple Challenge (BTC) 2019 Dataset is the result of a large-scale RDFcrawl (accepting RDF/XML, Turtle and N-Triples) conducted from 2018/12/12 until 2019/01/11 using LDspider. The data are stored as quads where the fourth element encodes the location of theWeb document from which the associated triple was parsed. The dataset contains2,155,856,033 quads, collected from 2,641,253 RDF documents on 394 pay-level domains.Merging the data into one RDF graph results in256,059,356 unique triples. These data (as quads or triples) contain38,156 unique predicates and instances of 120,037 unique classes. If you would like to use this dataset as part of a research work, we would ask you to please consider citing our paper: Jos-Miguel Herrera,Aidan Hoganand Tobias Kfer. BTC-2019: The 2019 Billion Triple Challenge Dataset. In the Proceedings of the18th International Semantic Web Conference (ISWC), Auckland, New Zealand, October 2630, 2019(Resources track). The dataset is published in three main parts: Quads:(*nq.gz): contains the quads retrieved during the crawl (N-Quads, GZipped). These data are divided into individual files for each of the top 100pay-level-domains by number of quads contributed(btc2019-[domain]_000XX.nq.gz).While most domains have one file, larger domains are further split into parts (000XX)with approximately 150,000,000 quads each. Finally, quads from the 294 domains not in the top 100 are merged into one file: btc2019-other_00001.nq.gz. Triples:(btc2019-triples.nt.gz): contains the unique triples resulting from taking all quads, dropping the fourth element (indicating the location of the source document) and computing the unique triples. VoID (void.nt): contains a VoID file offering statistics about the dataset. For parsing the files, we recommend a streaming parser, such as Raptor, RDF4j/Rio, or NxParser. The data are sourced from2,641,253 RDF documents. The top-10pay-level-domains in terms of documents contributed are: dbpedia.org 162,117 documents (6.14%) loc.gov 150,091 documents (5.68%) bnf.fr 146,186 documents (5.53%) sudoc.fr 144,877 documents (5.49%) theses.fr 141,228 documents (5.35%) wikidata.org 141,207 documents (5.35%) linkeddata.es 130,459 documents (4.94%) getty.edu 130,398 documents (4.94%) fao.org 92,838 documents (3.51%) ontobee.org 92,812 documents (3.51%) The data contain2,155,856,033 quads. The top-10pay-level-domains in terms of quadscontributed are: wikidata.org 2,006,338,975 quads (93.06%) dbpedia.org 36,686,161 quads (1.70%) idref.fr 22,013,225 quads (1.02%) bnf.fr 12,618,155 quads (0.59%) getty.edu 7,453,134 quads (0.35%) sudoc.fr 7,176,301 quads (0.33%) loc.gov 6,725,390 quads (0.31%) linkeddata.es 6,485,114 quads (0.30%) theses.fr 4,820,874 quads (0.22%) ontologycentral.com 4,633,947 quads (0.21%) The data contain256,059,356 unique triples. The top-10pay-level-domains in terms of unique triplescontributed are: wikidata.org 133,535,555 triples (52.15%) dbpedia.org 32,981,420 triples (12.88%) idref.fr 16,820,681 triples (6.57%) bnf.fr 11,769,268 triples (4.60%) getty.edu 6,571,525 triples (2.57%) linkeddata.es 5,898,762 triples (2.30%) loc.gov 5,362,064 triples (2.09%) sudoc.fr 4,972,647 triples (1.94%) ontologycentral.com 4,471,962 triples (1.75%) theses.fr 4,095,897 triples (1.60%) If you wish to download all N-Quads files, the following may be useful to copy and paste in Unix: wget https://zenodo.org/record/2634588/files/btc2019-acropolis.org.uk_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-aksw.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-babelnet.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-bbc.co.uk_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-berkeleybop.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-bibliotheken.nl_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-bl.uk_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-bne.es_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-bnf.fr_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-camera.it_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-cervantesvirtual.com_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-chemspider.com_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-cnr.it_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-comicmeta.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-crossref.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-cvut.cz_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-d-nb.info_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-datacite.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-dbpedia.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-dbtune.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-drugbank.ca_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-ebi.ac.uk_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-ebu.ch_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-ebusiness-unibw.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-edamontology.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-europa.eu_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-fao.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-gbv.de_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-geonames.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-geospecies.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-geovocab.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-gesis.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-getty.edu_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-github.io_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-githubusercontent.com_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-glottolog.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-iconclass.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-idref.fr_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-iflastandards.info_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-ign.fr_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-iptc.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-kanzaki.com_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-kasei.us_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-kit.edu_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-kjernsmo.net_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-korrekt.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-kulturarvsdata.se_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-kulturnav.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-l3s.de_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-lehigh.edu_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-lexvo.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-linkeddata.es_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-linkedopendata.gr_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-linkedresearch.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-loc.gov_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-lu.se_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-mcu.es_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-medra.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-myexperiment.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-ndl.go.jp_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-nih.gov_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-nobelprize.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-okfn.gr_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-ontobee.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-ontologycentral.com_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-openei.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-openlibrary.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-orcid.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-ordnancesurvey.co.uk_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-oszk.hu_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-other_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-persee.fr_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-pokepedia.fr_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-princeton.edu_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-productontology.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-rdaregistry.info_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-rdvocab.info_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-reegle.info_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-rhiaro.co.uk_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-schema.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-sf.net_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-simia.net_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-sti2.at_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-stoa.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-sudoc.fr_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-taxonconcept.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-theses.fr_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-timbl.com_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-uba.de_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-unesco.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-uni-mannheim.de_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-uniprot.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-unitn.it_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-verborgh.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-w3.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-wals.info_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-walsh.name_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-wikidata.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-wikidata.org_00002.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-wikidata.org_00003.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-wikidata.org_00004.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-wikidata.org_00005.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-wikidata.org_00006.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-wikidata.org_00007.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-wikidata.org_00008.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-wikidata.org_00009.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-wikidata.org_00010.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-wikidata.org_00011.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-wikidata.org_00012.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-wikidata.org_00013.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-wikidata.org_00014.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-worldcat.org_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-xmlns.com_00001.nq.gz wget https://zenodo.org/record/2634588/files/btc2019-zbw.eu_00001.nq.gz To merge these files into one (again on Unix): cat *.nq.gz btc2019-quads.nq.gz_bak mv btc2019-quads.nq.gz_bak btc2019-quads.nq.gz Links to previous BTC datasets: BTC 2014 BTC 2012 BTC 2011 BTC 2010 BTC 2009 BTC 2008 (no link available)
This page was built for dataset: Billion Triple Challenge (BTC) 2019 Dataset
