Project:QuickStatements: Difference between revisions
From MaRDI portal
Created page with "== Data Gift to Wikidata == This section describes how to export data from the MaRDI portal to Wikidata to make the data better findable for the general public. Example [https://www.wikidata.org/wiki/Property:P11166 EuDML] prposed by [https://www.wikidata.org/wiki/User:Vladimir_Alexiev Vladimir Alexiev] in 2022, which has currently [https://doi.org/10.5281/zenodo.13894124 17] results. # Prepare query for the [https://query.portal.mardi4nfdi.de MaRDI SPARQL endpoint] #..." |
|||
(27 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
== Data | == About == | ||
This page assists in managing queries that produce output in a format that can serve as input to QuickStatements, either on the [https://quickstatements.portal.mardi4nfdi.de/#/batch MaRDI instance] or on [https://quickstatements.toolforge.org/ the Wikidata one]. Other Wikibase instances also often have a QuickStatements instance that can be used in an analogous fashion. | |||
== Data transfer to Wikidata == | |||
This section describes how to export data from the MaRDI portal to Wikidata to make the data better findable for the general public. | This section describes how to export data from the MaRDI portal to Wikidata to make the data better findable for the general public. | ||
Example [https://www.wikidata.org/wiki/Property:P11166 EuDML] | Example [https://www.wikidata.org/wiki/Property:P11166 EuDML] proposed by [https://www.wikidata.org/wiki/User:Vladimir_Alexiev Vladimir Alexiev] in 2022, which had [https://doi.org/10.5281/zenodo.13894124 17] results as of 6 October 2024. | ||
# Prepare query for the [https://query.portal.mardi4nfdi.de MaRDI SPARQL endpoint] | # Prepare query for the [https://query.portal.mardi4nfdi.de MaRDI SPARQL endpoint] | ||
## Find the qIDs P12 = WikidataQId | ## Find the MaRDI qIDs with a statement using P12 = WikidataQId | ||
## The thing you want to export, here a part of the external url P205 | ## The thing you want to export, here a part of the external url P205 | ||
##* [https://query.portal.mardi4nfdi.de/#SELECT%20%28COUNT%28%3FMaRDI_item%29%20AS%20%3Fcount%29%20WHERE%20%7B%0A%20%20%3FMaRDI_item%20wdt%3AP205%20%3FURL%20.%0A%20%20FILTER%28STRSTARTS%28STR%28%3FURL%29%2C%20%22https%3A%2F%2Feudml%22%29%29%0A%20%20BIND%28SUBSTR%28STR%28%3FURL%29%2C%2023%20%29%20AS%20%3FEuDMLid%29%0A%7D%0A Check] for total number of MaRDI items with a P205 (External URL) statement. (170197 as of 7 October 2024) | |||
##* Filter the URLs for the pattern of interest — in our case, this would be [https://query.portal.mardi4nfdi.de/#SELECT%20%2a%20%0AWHERE%20%7B%0A%20%20SERVICE%20bd%3Asample%20%7B%20%3Fmardi_item%20wdt%3AP205%20%3FURL%20.%20bd%3AserviceParam%20bd%3Asample.limit%20123456%20%7D%0A%20%20FILTER%28STRSTARTS%28STR%28%3FURL%29%2C%20%22https%3A%2F%2Feudml%22%29%29%0A%20%20BIND%28SUBSTR%28STR%28%3FURL%29%2C%2023%20%29%20AS%20%3FEuDMLid%29%0A%23%20%20%20BIND%28STRAFTER%28STR%28%3FURL%29%2C%20%27https%3A%2F%2Feudml.org%2Fdoc%2F%27%29%20AS%20%3FEuDMLid1%29%0A%7D "starts with https://eudml.org/doc/"] (7115 as of 7 October 2024) | |||
## [https://query.portal.mardi4nfdi.de/#%23%20This%20query%20produces%20statements%20for%0A%23%20https%3A%2F%2Fquickstatements.portal.mardi4nfdi.de%2F%23%2Fbatch%2Fnew%0A%23%20Help%3A%20https%3A%2F%2Fwww.wikidata.org%2Fwiki%2FHelp%3AQuickStatements%20.%0A%0APREFIX%20wdt%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fprop%2Fdirect%2F%3E%0APREFIX%20wd%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fentity%2F%3E%0A%0APREFIX%20wikidata_wd%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fentity%2F%3E%0APREFIX%20wikidata_wdt%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fprop%2Fdirect%2F%3E%0A%0ASELECT%20DISTINCT%0A%0A%23%23%23%23%23%23%23%0A%23%20To%20get%20the%20QuickStatements%20commands%2C%20comment%20out%20the%20lines%20until%20the%20next%20hash%20line%0A%23%20%3Fmardi_item%0A%23%20%3Fmardi_item_label%0A%23%20%3Fwikidata%0A%23%20%3Fwikidata_label%0A%23%20%3FDOI%0A%23%23%23%23%23%23%23%0A%0A(REPLACE(STR(%3Fmardi_item)%2C%20%22.*Q%22%2C%20%22Q%22)%20AS%20%3Fmardi_qid)%20%0A(%22P12%22%20AS%20%3Fproperty)%0A(CONCAT(%22\%22%22%2C%20REPLACE(STR(%3Fwikidata)%2C%20%22.*Q%22%2C%20%22Q%22)%2C%20%22\%22%22)%20AS%20%3Fwikidata_qid)%20%0A%0AWHERE%20{%0A%20%20SERVICE%20bd%3Asample%20{%20%3Fmardi_item%20wdt%3AP205%20%3Ffulltext%20.%20bd%3AserviceParam%20bd%3Asample.limit%2020000%20}%0A%20%20%3Fmardi_item%20wdt%3AP27%20%3Fdoi%20.%0A%20%20BIND(UCASE(%3Fdoi)%20AS%20%3FDOI)%0A%20%20FILTER%20NOT%20EXISTS%20{%20%3Fmardi_item%20wdt%3AP12%20%3FWikidataQID%20}%20%0A%20%20service%20%3Chttps%3A%2F%2Fquery.wikidata.org%2Fsparql%3E%20{%0A%20%20%20%20%3Fwikidata%20wikidata_wdt%3AP356%20%3FDOI%20.%0A%20%20%20%20%3Fwikidata%20rdfs%3Alabel%20%3Fwikidata_label%20.%0A%20%20%20%20FILTER%20(LANG(%3Fwikidata_label)%20%3D%20%22en%22)%0A%20%20}%0A%20%20%3Fmardi_item%20rdfs%3Alabel%20%3Fmardi_item_label%20.%0A%20%20FILTER%20(LANG(%3Fmardi_item_label)%20%3D%20%22en%22)%0A}%0ALIMIT%201000 Check] whether there are MaRDI items with P205 (external URL) statements and without P12 (Wikidata QID) statements. If so, use MaRDI's QuickStatements tool to add that query's results to MaRDI ([https://portal.mardi4nfdi.de/w/index.php?title=Item%3AQ6197582&diff=37853390&oldid=37649161 sample edit]). | |||
# Write the query, example <syntaxhighlight lang=sparql> | # Write the query, example <syntaxhighlight lang=sparql> | ||
SELECT ? | SELECT ?MaRDI_qID ?MaRDI_item ?EuDMLid WHERE { | ||
? | ?MaRDI_item wdt:P205 ?URL; | ||
wdt:P12 ?Wikidata_qID. | |||
FILTER (STRSTARTS( | FILTER(STRSTARTS(STR(?URL), "https://eudml")) | ||
BIND ( | BIND(SUBSTR(STR(?URL), 23 ) AS ?EuDMLid) | ||
} | } | ||
LIMIT 10 | LIMIT 10 | ||
</syntaxhighlight> and | </syntaxhighlight> and investigate the result. | ||
# | #* Check for total number of MaRDI items with both (i) a Wikidata QID statement and (ii) a EuDML-based URL link ([https://query.portal.mardi4nfdi.de/#SELECT%20%28COUNT%28%3FMaRDI_item%29%20AS%20%3Fcount%29%20WHERE%20%7B%0A%20%20%3FMaRDI_item%20wdt%3AP205%20%3FURL%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20wdt%3AP12%20%3FWikidata_qID.%0A%20%20FILTER%28STRSTARTS%28STR%28%3FURL%29%2C%20%22https%3A%2F%2Feudml%22%29%29%0A%20%20BIND%28SUBSTR%28STR%28%3FURL%29%2C%2023%20%29%20AS%20%3FEuDMLid%29%0A%7D%0A 9296] as of 7 October 2024) | ||
# Find the properties in Wikidata, here only P11166 for eudml. | |||
# Check whether Wikidata already has a corresponding statement. | |||
##* This does [https://query.portal.mardi4nfdi.de/#%23%20Add%20help%20text%0A%0APREFIX%20wdt%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fprop%2Fdirect%2F%3E%0APREFIX%20wd%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fentity%2F%3E%0A%0APREFIX%20wikidata_wd%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fentity%2F%3E%0APREFIX%20wikidata_wdt%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fprop%2Fdirect%2F%3E%0A%0A%0ASELECT%20%3FMaRDI_qID%20%3FMaRDI_item%20%3FEuDMLid%20WHERE%20%7B%0A%20%20%3FMaRDI_item%20wdt%3AP205%20%3FURL%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20wdt%3AP12%20%3FWikidata_qID.%0A%20%20FILTER%28STRSTARTS%28STR%28%3FURL%29%2C%20%22https%3A%2F%2Feudml%22%29%29%0A%20%20BIND%28SUBSTR%28STR%28%3FURL%29%2C%2023%20%29%20AS%20%3FEuDMLid%29%0A%23%20%20%20FILTER%20NOT%20EXISTS%20%7B%20%3Fmardi_item%20wdt%3AP12%20%3FWikidataQID%20%7D%20%0A%20%20service%20%3Chttps%3A%2F%2Fquery.wikidata.org%2Fsparql%3E%20%7B%0A%20%20%20%20%3Fwikidata%20wikidata_wdt%3AP11166%20%3FEuDMLid%20.%0A%20%20%20%20%3Fwikidata%20rdfs%3Alabel%20%3Fwikidata_label%20.%0A%20%20%20%20FILTER%20%28LANG%28%3Fwikidata_label%29%20%3D%20%22en%22%29%0A%20%20%7D%0A%20%20%0A%7D%0A%23%20LIMIT%2010 not scale well]. Perhaps need to [https://query.portal.mardi4nfdi.de/#%23%20Add%20help%20text%0A%0APREFIX%20wdt%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fprop%2Fdirect%2F%3E%0APREFIX%20wd%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fentity%2F%3E%0A%0APREFIX%20wikidata_wd%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fentity%2F%3E%0APREFIX%20wikidata_wdt%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fprop%2Fdirect%2F%3E%0A%0A%0ASELECT%20%3FMaRDI_item%20%3FURL%20%3FEuDMLid%20%3FWikidata_item%20WHERE%20%7B%0A%20%20%3FMaRDI_item%20wdt%3AP205%20%3FURL%20%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20wdt%3AP27%20%3Fdoi%20%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20wdt%3AP12%20%20%3FWikidataQID%20.%0A%20%20%0A%20%20FILTER%28STRSTARTS%28STR%28%3FURL%29%2C%20%22https%3A%2F%2Feudml%22%29%29%0A%20%20BIND%28SUBSTR%28STR%28%3FURL%29%2C%2023%20%29%20AS%20%3FEuDMLid%29%0A%0A%20%20BIND%28UCASE%28%3Fdoi%29%20AS%20%3FDOI%29%0A%20%20%0A%20%20service%20%3Chttps%3A%2F%2Fquery.wikidata.org%2Fsparql%3E%20%7B%0A%20%20%20%20%3FWikidata_item%20wikidata_wdt%3AP356%20%3FDOI%20.%0A%20%20%20%20FILTER%20NOT%20EXISTS%20%7B%0A%20%20%20%20%20%20%3FWikidata_item%20wikidata_wdt%3AP11166%20%5B%5D%20.%0A%20%20%20%20%7D%20%20%0A%20%20%7D%0A%20%20%0A%7D%0ALIMIT%2010 use DOI] or zbMATH ID ([https://query.portal.mardi4nfdi.de/#%23%20Add%20help%20text%0A%0APREFIX%20wdt%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fprop%2Fdirect%2F%3E%0APREFIX%20wd%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fentity%2F%3E%0A%0APREFIX%20wikidata_wd%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fentity%2F%3E%0APREFIX%20wikidata_wdt%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fprop%2Fdirect%2F%3E%0A%0A%0ASELECT%20%3FMaRDI_item%20%3FURL%20%3FEuDMLid%20%3FWikidata_item%20WHERE%20%7B%0A%20%20%3FMaRDI_item%20wdt%3AP205%20%3FURL%20%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20wdt%3AP225%20%3FzbMATH_doc%20%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20wdt%3AP12%20%20%3FWikidataQID%20.%0A%20%20%0A%20%20FILTER%28STRSTARTS%28STR%28%3FURL%29%2C%20%22https%3A%2F%2Feudml%22%29%29%0A%20%20BIND%28SUBSTR%28STR%28%3FURL%29%2C%2023%20%29%20AS%20%3FEuDMLid%29%0A%0A%23%20%20%20BIND%28UCASE%28%3Fdoi%29%20AS%20%3FDOI%29%0A%20%20%0A%20%20service%20%3Chttps%3A%2F%2Fquery.wikidata.org%2Fsparql%3E%20%7B%0A%20%20%20%20%3FWikidata_item%20wikidata_wdt%3AP894%20%3FzbMATH_doc%20.%0A%20%20%20%20MINUS%20%7B%0A%20%20%20%20%20%20%3FWikidata_item%20wikidata_wdt%3AP11166%20%5B%5D%20.%0A%20%20%20%20%7D%20%20%0A%20%20%7D%0A%20%20%0A%7D%0ALIMIT%2010 doc]/ person) as a bridge | |||
## If yes, check consistency with MaRDI. | |||
### To do | |||
## If no, prepare upload to Wikidata. | |||
### To do | |||
# Rewrite the query to match the [https://www.wikidata.org/wiki/Help:QuickStatements Quickstatements CSV format] <syntaxhighlight lang=sparql> | # Rewrite the query to match the [https://www.wikidata.org/wiki/Help:QuickStatements Quickstatements CSV format] <syntaxhighlight lang=sparql> | ||
SELECT ? | SELECT ?qid ?P11166 WHERE { | ||
?qal854 wdt:P205 ?url ; | |||
wdt:P12 ?qid | |||
FILTER (STRSTARTS(str(?url) ,"https://eudml")) . | |||
BIND (Concat( | |||
"\"", ## Required for external identifiers by quickstatements | |||
SUBSTR(STR(?url),23), ## 23 = Length of the prefix | |||
"\"") as ?P11166) | |||
} | |||
LIMIT 3 | |||
</syntaxhighlight> | |||
# Download the CSV file and test with a few items on Wikidata | |||
# Optional add source linking back to the source of the information (MaRDI) <syntaxhighlight lang=sparql> | |||
SELECT ?qid ?P11166 ?S854 WHERE { | |||
?item wdt:P205 ?url ; | ?item wdt:P205 ?url ; | ||
wdt:P12 ? | wdt:P12 ?qid | ||
FILTER (STRSTARTS(str(?url) ,"https://eudml")) . | FILTER (STRSTARTS(str(?url) ,"https://eudml")) . | ||
BIND ( | BIND (Concat( | ||
"\"", ## Required for external identifiers by quickstatements | |||
SUBSTR(STR(?url),23), ## 23 = Length of the prefix | |||
"\"") as ?P11166). | |||
BIND (Concat( | |||
"\"", ## Required for external identifiers by quickstatements | |||
STR(?item), | |||
"\"") as ?S854) | |||
} | } | ||
LIMIT 3 | LIMIT 3 | ||
</syntaxhighlight> | </syntaxhighlight> | ||
# Remove the limit and do the full import | # Remove the limit and do the full import |
Latest revision as of 12:56, 10 October 2024
About
This page assists in managing queries that produce output in a format that can serve as input to QuickStatements, either on the MaRDI instance or on the Wikidata one. Other Wikibase instances also often have a QuickStatements instance that can be used in an analogous fashion.
Data transfer to Wikidata
This section describes how to export data from the MaRDI portal to Wikidata to make the data better findable for the general public.
Example EuDML proposed by Vladimir Alexiev in 2022, which had 17 results as of 6 October 2024.
- Prepare query for the MaRDI SPARQL endpoint
- Find the MaRDI qIDs with a statement using P12 = WikidataQId
- The thing you want to export, here a part of the external url P205
- Check for total number of MaRDI items with a P205 (External URL) statement. (170197 as of 7 October 2024)
- Filter the URLs for the pattern of interest — in our case, this would be "starts with https://eudml.org/doc/" (7115 as of 7 October 2024)
- Check whether there are MaRDI items with P205 (external URL) statements and without P12 (Wikidata QID) statements. If so, use MaRDI's QuickStatements tool to add that query's results to MaRDI (sample edit).
- Write the query, example and investigate the result.
SELECT ?MaRDI_qID ?MaRDI_item ?EuDMLid WHERE { ?MaRDI_item wdt:P205 ?URL; wdt:P12 ?Wikidata_qID. FILTER(STRSTARTS(STR(?URL), "https://eudml")) BIND(SUBSTR(STR(?URL), 23 ) AS ?EuDMLid) } LIMIT 10
- Check for total number of MaRDI items with both (i) a Wikidata QID statement and (ii) a EuDML-based URL link (9296 as of 7 October 2024)
- Find the properties in Wikidata, here only P11166 for eudml.
- Check whether Wikidata already has a corresponding statement.
- This does not scale well. Perhaps need to use DOI or zbMATH ID (doc/ person) as a bridge
- If yes, check consistency with MaRDI.
- To do
- If no, prepare upload to Wikidata.
- To do
- Rewrite the query to match the Quickstatements CSV format
SELECT ?qid ?P11166 WHERE { ?qal854 wdt:P205 ?url ; wdt:P12 ?qid FILTER (STRSTARTS(str(?url) ,"https://eudml")) . BIND (Concat( "\"", ## Required for external identifiers by quickstatements SUBSTR(STR(?url),23), ## 23 = Length of the prefix "\"") as ?P11166) } LIMIT 3
- Download the CSV file and test with a few items on Wikidata
- Optional add source linking back to the source of the information (MaRDI)
SELECT ?qid ?P11166 ?S854 WHERE { ?item wdt:P205 ?url ; wdt:P12 ?qid FILTER (STRSTARTS(str(?url) ,"https://eudml")) . BIND (Concat( "\"", ## Required for external identifiers by quickstatements SUBSTR(STR(?url),23), ## 23 = Length of the prefix "\"") as ?P11166). BIND (Concat( "\"", ## Required for external identifiers by quickstatements STR(?item), "\"") as ?S854) } LIMIT 3
- Remove the limit and do the full import