Project:QuickStatements: Difference between revisions

From MaRDI portal
 
(10 intermediate revisions by the same user not shown)
Line 1: Line 1:
== About ==
This page assists in managing queries that produce output in a format that can serve as input to QuickStatements, either on the [https://quickstatements.portal.mardi4nfdi.de/#/batch MaRDI instance] or on [https://quickstatements.toolforge.org/ the Wikidata one]. Other Wikibase instances also often have a QuickStatements instance that can be used in an analogous fashion.
== Data transfer to Wikidata ==
== Data transfer to Wikidata ==
This section describes how to export data from the MaRDI portal to Wikidata to make the data better findable for the general public.
This section describes how to export data from the MaRDI portal to Wikidata to make the data better findable for the general public.


Example [https://www.wikidata.org/wiki/Property:P11166 EuDML] prposed by [https://www.wikidata.org/wiki/User:Vladimir_Alexiev Vladimir Alexiev] in 2022, which has currently [https://doi.org/10.5281/zenodo.13894124 17] results.  
Example [https://www.wikidata.org/wiki/Property:P11166 EuDML] proposed by [https://www.wikidata.org/wiki/User:Vladimir_Alexiev Vladimir Alexiev] in 2022, which had [https://doi.org/10.5281/zenodo.13894124 17] results as of 6 October 2024.  


# Prepare query for the [https://query.portal.mardi4nfdi.de MaRDI SPARQL endpoint]
# Prepare query for the [https://query.portal.mardi4nfdi.de MaRDI SPARQL endpoint]
## Find the qIDs with a statement using P12 = WikidataQId
## Find the MaRDI qIDs with a statement using P12 = WikidataQId
## The thing you want to export, here a part of the external url P205
## The thing you want to export, here a part of the external url P205
## [https://query.portal.mardi4nfdi.de/#%23%20This%20query%20produces%20statements%20for%0A%23%20https%3A%2F%2Fquickstatements.portal.mardi4nfdi.de%2F%23%2Fbatch%2Fnew%0A%23%20Help%3A%20https%3A%2F%2Fwww.wikidata.org%2Fwiki%2FHelp%3AQuickStatements%20.%0A%0APREFIX%20wdt%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fprop%2Fdirect%2F%3E%0APREFIX%20wd%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fentity%2F%3E%0A%0APREFIX%20wikidata_wd%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fentity%2F%3E%0APREFIX%20wikidata_wdt%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fprop%2Fdirect%2F%3E%0A%0ASELECT%20DISTINCT%0A%0A%23%23%23%23%23%23%23%0A%23%20To%20get%20the%20QuickStatements%20commands%2C%20comment%20out%20the%20lines%20until%20the%20next%20hash%20line%0A%23%20%3Fmardi_item%0A%23%20%3Fmardi_item_label%0A%23%20%3Fwikidata%0A%23%20%3Fwikidata_label%0A%23%20%3FDOI%0A%23%23%23%23%23%23%23%0A%0A(REPLACE(STR(%3Fmardi_item)%2C%20%22.*Q%22%2C%20%22Q%22)%20AS%20%3Fmardi_qid)%20%0A(%22P12%22%20AS%20%3Fproperty)%0A(CONCAT(%22\%22%22%2C%20REPLACE(STR(%3Fwikidata)%2C%20%22.*Q%22%2C%20%22Q%22)%2C%20%22\%22%22)%20AS%20%3Fwikidata_qid)%20%0A%0AWHERE%20{%0A%20%20SERVICE%20bd%3Asample%20{%20%3Fmardi_item%20wdt%3AP205%20%3Ffulltext%20.%20bd%3AserviceParam%20bd%3Asample.limit%2020000%20}%0A%20%20%3Fmardi_item%20wdt%3AP27%20%3Fdoi%20.%0A%20%20BIND(UCASE(%3Fdoi)%20AS%20%3FDOI)%0A%20%20FILTER%20NOT%20EXISTS%20{%20%3Fmardi_item%20wdt%3AP12%20%3FWikidataQID%20}%20%0A%20%20service%20%3Chttps%3A%2F%2Fquery.wikidata.org%2Fsparql%3E%20{%0A%20%20%20%20%3Fwikidata%20wikidata_wdt%3AP356%20%3FDOI%20.%0A%20%20%20%20%3Fwikidata%20rdfs%3Alabel%20%3Fwikidata_label%20.%0A%20%20%20%20FILTER%20(LANG(%3Fwikidata_label)%20%3D%20%22en%22)%0A%20%20}%0A%20%20%3Fmardi_item%20rdfs%3Alabel%20%3Fmardi_item_label%20.%0A%20%20FILTER%20(LANG(%3Fmardi_item_label)%20%3D%20%22en%22)%0A}%0ALIMIT%201000 Check] whether there are MaRDI items with P205 (external URL) statements and without P12 (Wikidata QID) statements. If so, use MaRDI's QuickStatements tool to add that query's results to MaRDI.
##* [https://query.portal.mardi4nfdi.de/#SELECT%20%28COUNT%28%3FMaRDI_item%29%20AS%20%3Fcount%29%20WHERE%20%7B%0A%20%20%3FMaRDI_item%20wdt%3AP205%20%3FURL%20.%0A%20%20FILTER%28STRSTARTS%28STR%28%3FURL%29%2C%20%22https%3A%2F%2Feudml%22%29%29%0A%20%20BIND%28SUBSTR%28STR%28%3FURL%29%2C%2023%20%29%20AS%20%3FEuDMLid%29%0A%7D%0A Check] for total number of MaRDI items with a P205 (External URL) statement. (170197 as of 7 October 2024)
## [https://query.portal.mardi4nfdi.de/#%23%20This%20query%20produces%20statements%20for%0A%23%20https%3A%2F%2Fquickstatements.portal.mardi4nfdi.de%2F%23%2Fbatch%2Fnew%0A%23%20Help%3A%20https%3A%2F%2Fwww.wikidata.org%2Fwiki%2FHelp%3AQuickStatements%20.%0A%0APREFIX%20wdt%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fprop%2Fdirect%2F%3E%0APREFIX%20wd%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fentity%2F%3E%0A%0APREFIX%20wikidata_wd%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fentity%2F%3E%0APREFIX%20wikidata_wdt%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fprop%2Fdirect%2F%3E%0A%0ASELECT%20DISTINCT%0A%0A%23%23%23%23%23%23%23%0A%23%20To%20get%20the%20QuickStatements%20commands%2C%20comment%20out%20the%20lines%20until%20the%20next%20hash%20line%0A%23%20%3Fmardi_item%0A%23%20%3Fmardi_item_label%0A%23%20%3Fwikidata%0A%23%20%3Fwikidata_label%0A%23%20%3FDOI%0A%23%23%23%23%23%23%23%0A%0A(REPLACE(STR(%3Fmardi_item)%2C%20%22.*Q%22%2C%20%22Q%22)%20AS%20%3Fmardi_qid)%20%0A(%22P12%22%20AS%20%3Fproperty)%0A(CONCAT(%22\%22%22%2C%20REPLACE(STR(%3Fwikidata)%2C%20%22.*Q%22%2C%20%22Q%22)%2C%20%22\%22%22)%20AS%20%3Fwikidata_qid)%20%0A%0AWHERE%20{%0A%20%20SERVICE%20bd%3Asample%20{%20%3Fmardi_item%20wdt%3AP205%20%3Ffulltext%20.%20bd%3AserviceParam%20bd%3Asample.limit%2020000%20}%0A%20%20%3Fmardi_item%20wdt%3AP27%20%3Fdoi%20.%0A%20%20BIND(UCASE(%3Fdoi)%20AS%20%3FDOI)%0A%20%20FILTER%20NOT%20EXISTS%20{%20%3Fmardi_item%20wdt%3AP12%20%3FWikidataQID%20}%20%0A%20%20service%20%3Chttps%3A%2F%2Fquery.wikidata.org%2Fsparql%3E%20{%0A%20%20%20%20%3Fwikidata%20wikidata_wdt%3AP356%20%3FDOI%20.%0A%20%20%20%20%3Fwikidata%20rdfs%3Alabel%20%3Fwikidata_label%20.%0A%20%20%20%20FILTER%20(LANG(%3Fwikidata_label)%20%3D%20%22en%22)%0A%20%20}%0A%20%20%3Fmardi_item%20rdfs%3Alabel%20%3Fmardi_item_label%20.%0A%20%20FILTER%20(LANG(%3Fmardi_item_label)%20%3D%20%22en%22)%0A}%0ALIMIT%201000 Check] whether there are MaRDI items with P205 (external URL) statements and without P12 (Wikidata QID) statements. If so, use MaRDI's QuickStatements tool to add that query's results to MaRDI ([https://portal.mardi4nfdi.de/w/index.php?title=Item%3AQ6197582&diff=37853390&oldid=37649161 sample edit]).
# Write the query, example <syntaxhighlight lang=sparql>
# Write the query, example <syntaxhighlight lang=sparql>
SELECT ?MaRDI_qID ?MaRDI_item ?EuDMLid WHERE {
SELECT ?MaRDI_qID ?MaRDI_item ?EuDMLid WHERE {
Line 17: Line 22:
LIMIT 10
LIMIT 10
</syntaxhighlight> and investigate the result.  
</syntaxhighlight> and investigate the result.  
#* Check for total number of MaRDI items with EuDML links ([https://query.portal.mardi4nfdi.de/#SELECT%20%28COUNT%28%3FMaRDI_item%29%20AS%20%3Fcount%29%20WHERE%20%7B%0A%20%20%3FMaRDI_item%20wdt%3AP205%20%3FURL%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20wdt%3AP12%20%3FWikidata_qID.%0A%20%20FILTER%28STRSTARTS%28STR%28%3FURL%29%2C%20%22https%3A%2F%2Feudml%22%29%29%0A%20%20BIND%28SUBSTR%28STR%28%3FURL%29%2C%2023%20%29%20AS%20%3FEuDMLid%29%0A%7D%0A 9296] as of 7 October 2024)
#* Check for total number of MaRDI items with both (i) a Wikidata QID statement and (ii) a EuDML-based URL link ([https://query.portal.mardi4nfdi.de/#SELECT%20%28COUNT%28%3FMaRDI_item%29%20AS%20%3Fcount%29%20WHERE%20%7B%0A%20%20%3FMaRDI_item%20wdt%3AP205%20%3FURL%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20wdt%3AP12%20%3FWikidata_qID.%0A%20%20FILTER%28STRSTARTS%28STR%28%3FURL%29%2C%20%22https%3A%2F%2Feudml%22%29%29%0A%20%20BIND%28SUBSTR%28STR%28%3FURL%29%2C%2023%20%29%20AS%20%3FEuDMLid%29%0A%7D%0A 9296] as of 7 October 2024)
# Find the properties in Wikidata, here only P11166 for eudml.
# Find the properties in Wikidata, here only P11166 for eudml.
# Check whether Wikidata already has a corresponding statement.
# Check whether Wikidata already has a corresponding statement.
##* This does [https://query.portal.mardi4nfdi.de/#%23%20Add%20help%20text%0A%0APREFIX%20wdt%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fprop%2Fdirect%2F%3E%0APREFIX%20wd%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fentity%2F%3E%0A%0APREFIX%20wikidata_wd%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fentity%2F%3E%0APREFIX%20wikidata_wdt%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fprop%2Fdirect%2F%3E%0A%0A%0ASELECT%20%3FMaRDI_qID%20%3FMaRDI_item%20%3FEuDMLid%20WHERE%20%7B%0A%20%20%3FMaRDI_item%20wdt%3AP205%20%3FURL%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20wdt%3AP12%20%3FWikidata_qID.%0A%20%20FILTER%28STRSTARTS%28STR%28%3FURL%29%2C%20%22https%3A%2F%2Feudml%22%29%29%0A%20%20BIND%28SUBSTR%28STR%28%3FURL%29%2C%2023%20%29%20AS%20%3FEuDMLid%29%0A%23%20%20%20FILTER%20NOT%20EXISTS%20%7B%20%3Fmardi_item%20wdt%3AP12%20%3FWikidataQID%20%7D%20%0A%20%20service%20%3Chttps%3A%2F%2Fquery.wikidata.org%2Fsparql%3E%20%7B%0A%20%20%20%20%3Fwikidata%20wikidata_wdt%3AP11166%20%3FEuDMLid%20.%0A%20%20%20%20%3Fwikidata%20rdfs%3Alabel%20%3Fwikidata_label%20.%0A%20%20%20%20FILTER%20%28LANG%28%3Fwikidata_label%29%20%3D%20%22en%22%29%0A%20%20%7D%0A%20%20%0A%7D%0A%23%20LIMIT%2010 not scale well]. Perhaps need to [https://query.portal.mardi4nfdi.de/#%23%20Add%20help%20text%0A%0APREFIX%20wdt%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fprop%2Fdirect%2F%3E%0APREFIX%20wd%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fentity%2F%3E%0A%0APREFIX%20wikidata_wd%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fentity%2F%3E%0APREFIX%20wikidata_wdt%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fprop%2Fdirect%2F%3E%0A%0A%0ASELECT%20%3FMaRDI_item%20%3FURL%20%3FEuDMLid%20%3FWikidata_item%20WHERE%20%7B%0A%20%20%3FMaRDI_item%20wdt%3AP205%20%3FURL%20%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20wdt%3AP27%20%3Fdoi%20%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20wdt%3AP12%20%20%3FWikidataQID%20.%0A%20%20%0A%20%20FILTER%28STRSTARTS%28STR%28%3FURL%29%2C%20%22https%3A%2F%2Feudml%22%29%29%0A%20%20BIND%28SUBSTR%28STR%28%3FURL%29%2C%2023%20%29%20AS%20%3FEuDMLid%29%0A%0A%20%20BIND%28UCASE%28%3Fdoi%29%20AS%20%3FDOI%29%0A%20%20%0A%20%20service%20%3Chttps%3A%2F%2Fquery.wikidata.org%2Fsparql%3E%20%7B%0A%20%20%20%20%3FWikidata_item%20wikidata_wdt%3AP356%20%3FDOI%20.%0A%20%20%20%20FILTER%20NOT%20EXISTS%20%7B%0A%20%20%20%20%20%20%3FWikidata_item%20wikidata_wdt%3AP11166%20%5B%5D%20.%0A%20%20%20%20%7D%20%20%0A%20%20%7D%0A%20%20%0A%7D%0ALIMIT%2010 use DOI] or zbMATH ID ([https://query.portal.mardi4nfdi.de/#%23%20Add%20help%20text%0A%0APREFIX%20wdt%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fprop%2Fdirect%2F%3E%0APREFIX%20wd%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fentity%2F%3E%0A%0APREFIX%20wikidata_wd%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fentity%2F%3E%0APREFIX%20wikidata_wdt%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fprop%2Fdirect%2F%3E%0A%0A%0ASELECT%20%3FMaRDI_item%20%3FURL%20%3FEuDMLid%20%3FWikidata_item%20WHERE%20%7B%0A%20%20%3FMaRDI_item%20wdt%3AP205%20%3FURL%20%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20wdt%3AP225%20%3FzbMATH_doc%20%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20wdt%3AP12%20%20%3FWikidataQID%20.%0A%20%20%0A%20%20FILTER%28STRSTARTS%28STR%28%3FURL%29%2C%20%22https%3A%2F%2Feudml%22%29%29%0A%20%20BIND%28SUBSTR%28STR%28%3FURL%29%2C%2023%20%29%20AS%20%3FEuDMLid%29%0A%0A%23%20%20%20BIND%28UCASE%28%3Fdoi%29%20AS%20%3FDOI%29%0A%20%20%0A%20%20service%20%3Chttps%3A%2F%2Fquery.wikidata.org%2Fsparql%3E%20%7B%0A%20%20%20%20%3FWikidata_item%20wikidata_wdt%3AP894%20%3FzbMATH_doc%20.%0A%20%20%20%20MINUS%20%7B%0A%20%20%20%20%20%20%3FWikidata_item%20wikidata_wdt%3AP11166%20%5B%5D%20.%0A%20%20%20%20%7D%20%20%0A%20%20%7D%0A%20%20%0A%7D%0ALIMIT%2010 doc]/ person) as a bridge
## If yes, check consistency with MaRDI.
## If yes, check consistency with MaRDI.
### To do
## If no, prepare upload to Wikidata.
## If no, prepare upload to Wikidata.
### To do
# Rewrite the query to match the [https://www.wikidata.org/wiki/Help:QuickStatements Quickstatements CSV format] <syntaxhighlight lang=sparql>
# Rewrite the query to match the [https://www.wikidata.org/wiki/Help:QuickStatements Quickstatements CSV format] <syntaxhighlight lang=sparql>
SELECT ?qid ?P11166 WHERE {
SELECT ?qid ?P11166 WHERE {

Latest revision as of 22:11, 7 October 2024

About

This page assists in managing queries that produce output in a format that can serve as input to QuickStatements, either on the MaRDI instance or on the Wikidata one. Other Wikibase instances also often have a QuickStatements instance that can be used in an analogous fashion.

Data transfer to Wikidata

This section describes how to export data from the MaRDI portal to Wikidata to make the data better findable for the general public.

Example EuDML proposed by Vladimir Alexiev in 2022, which had 17 results as of 6 October 2024.

  1. Prepare query for the MaRDI SPARQL endpoint
    1. Find the MaRDI qIDs with a statement using P12 = WikidataQId
    2. The thing you want to export, here a part of the external url P205
      • Check for total number of MaRDI items with a P205 (External URL) statement. (170197 as of 7 October 2024)
    3. Check whether there are MaRDI items with P205 (external URL) statements and without P12 (Wikidata QID) statements. If so, use MaRDI's QuickStatements tool to add that query's results to MaRDI (sample edit).
  2. Write the query, example
    SELECT ?MaRDI_qID ?MaRDI_item ?EuDMLid WHERE {
      ?MaRDI_item wdt:P205 ?URL;
                  wdt:P12 ?Wikidata_qID.
      FILTER(STRSTARTS(STR(?URL), "https://eudml"))
      BIND(SUBSTR(STR(?URL), 23 ) AS ?EuDMLid)
    }
    LIMIT 10
    
    and investigate the result.
    • Check for total number of MaRDI items with both (i) a Wikidata QID statement and (ii) a EuDML-based URL link (9296 as of 7 October 2024)
  3. Find the properties in Wikidata, here only P11166 for eudml.
  4. Check whether Wikidata already has a corresponding statement.
    1. If yes, check consistency with MaRDI.
      1. To do
    2. If no, prepare upload to Wikidata.
      1. To do
  5. Rewrite the query to match the Quickstatements CSV format
    SELECT ?qid ?P11166 WHERE {
      ?qal854 wdt:P205 ?url ;
            wdt:P12 ?qid
      FILTER (STRSTARTS(str(?url) ,"https://eudml")) .
      BIND (Concat( 
        "\"", ## Required for external identifiers by quickstatements
        SUBSTR(STR(?url),23), ## 23 = Length of the prefix
        "\"") as ?P11166)
      }
    LIMIT 3
    
  6. Download the CSV file and test with a few items on Wikidata
  7. Optional add source linking back to the source of the information (MaRDI)
    SELECT ?qid ?P11166 ?S854 WHERE {
      ?item wdt:P205 ?url ;
            wdt:P12 ?qid
      FILTER (STRSTARTS(str(?url) ,"https://eudml")) .
      BIND (Concat( 
        "\"", ## Required for external identifiers by quickstatements
        SUBSTR(STR(?url),23), ## 23 = Length of the prefix
        "\"") as ?P11166).
        BIND (Concat( 
        "\"", ## Required for external identifiers by quickstatements
        STR(?item), 
        "\"") as ?S854)
      }
    LIMIT 3
    
  8. Remove the limit and do the full import