Project:QuickStatements: Difference between revisions

From MaRDI portal
 
(26 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Data Gift to Wikidata ==
== About ==
 
This page assists in managing queries that produce output in a format that can serve as input to QuickStatements, either on the [https://quickstatements.portal.mardi4nfdi.de/#/batch MaRDI instance] or on [https://quickstatements.toolforge.org/ the Wikidata one]. Other Wikibase instances also often have a QuickStatements instance that can be used in an analogous fashion.
 
== Data transfer to Wikidata ==
This section describes how to export data from the MaRDI portal to Wikidata to make the data better findable for the general public.
This section describes how to export data from the MaRDI portal to Wikidata to make the data better findable for the general public.


Example [https://www.wikidata.org/wiki/Property:P11166 EuDML] prposed by [https://www.wikidata.org/wiki/User:Vladimir_Alexiev Vladimir Alexiev] in 2022, which has currently [https://doi.org/10.5281/zenodo.13894124 17] results.  
Example [https://www.wikidata.org/wiki/Property:P11166 EuDML] proposed by [https://www.wikidata.org/wiki/User:Vladimir_Alexiev Vladimir Alexiev] in 2022, which had [https://doi.org/10.5281/zenodo.13894124 17] results as of 6 October 2024.  


# Prepare query for the [https://query.portal.mardi4nfdi.de MaRDI SPARQL endpoint]
# Prepare query for the [https://query.portal.mardi4nfdi.de MaRDI SPARQL endpoint]
## Find the qIDs P12 = WikidataQId
## Find the MaRDI qIDs with a statement using P12 = WikidataQId
## The thing you want to export, here a part of the external url P205
## The thing you want to export, here a part of the external url P205
##* [https://query.portal.mardi4nfdi.de/#SELECT%20%28COUNT%28%3FMaRDI_item%29%20AS%20%3Fcount%29%20WHERE%20%7B%0A%20%20%3FMaRDI_item%20wdt%3AP205%20%3FURL%20.%0A%20%20FILTER%28STRSTARTS%28STR%28%3FURL%29%2C%20%22https%3A%2F%2Feudml%22%29%29%0A%20%20BIND%28SUBSTR%28STR%28%3FURL%29%2C%2023%20%29%20AS%20%3FEuDMLid%29%0A%7D%0A Check] for total number of MaRDI items with a P205 (External URL) statement. (170197 as of 7 October 2024)
##* Filter the URLs for the pattern of interest — in our case, this would be [https://query.portal.mardi4nfdi.de/#SELECT%20%2a%20%0AWHERE%20%7B%0A%20%20SERVICE%20bd%3Asample%20%7B%20%3Fmardi_item%20wdt%3AP205%20%3FURL%20.%20bd%3AserviceParam%20bd%3Asample.limit%20123456%20%7D%0A%20%20FILTER%28STRSTARTS%28STR%28%3FURL%29%2C%20%22https%3A%2F%2Feudml%22%29%29%0A%20%20BIND%28SUBSTR%28STR%28%3FURL%29%2C%2023%20%29%20AS%20%3FEuDMLid%29%0A%23%20%20%20BIND%28STRAFTER%28STR%28%3FURL%29%2C%20%27https%3A%2F%2Feudml.org%2Fdoc%2F%27%29%20AS%20%3FEuDMLid1%29%0A%7D "starts with https://eudml.org/doc/"] (7115 as of 7 October 2024)
## [https://query.portal.mardi4nfdi.de/#%23%20This%20query%20produces%20statements%20for%0A%23%20https%3A%2F%2Fquickstatements.portal.mardi4nfdi.de%2F%23%2Fbatch%2Fnew%0A%23%20Help%3A%20https%3A%2F%2Fwww.wikidata.org%2Fwiki%2FHelp%3AQuickStatements%20.%0A%0APREFIX%20wdt%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fprop%2Fdirect%2F%3E%0APREFIX%20wd%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fentity%2F%3E%0A%0APREFIX%20wikidata_wd%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fentity%2F%3E%0APREFIX%20wikidata_wdt%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fprop%2Fdirect%2F%3E%0A%0ASELECT%20DISTINCT%0A%0A%23%23%23%23%23%23%23%0A%23%20To%20get%20the%20QuickStatements%20commands%2C%20comment%20out%20the%20lines%20until%20the%20next%20hash%20line%0A%23%20%3Fmardi_item%0A%23%20%3Fmardi_item_label%0A%23%20%3Fwikidata%0A%23%20%3Fwikidata_label%0A%23%20%3FDOI%0A%23%23%23%23%23%23%23%0A%0A(REPLACE(STR(%3Fmardi_item)%2C%20%22.*Q%22%2C%20%22Q%22)%20AS%20%3Fmardi_qid)%20%0A(%22P12%22%20AS%20%3Fproperty)%0A(CONCAT(%22\%22%22%2C%20REPLACE(STR(%3Fwikidata)%2C%20%22.*Q%22%2C%20%22Q%22)%2C%20%22\%22%22)%20AS%20%3Fwikidata_qid)%20%0A%0AWHERE%20{%0A%20%20SERVICE%20bd%3Asample%20{%20%3Fmardi_item%20wdt%3AP205%20%3Ffulltext%20.%20bd%3AserviceParam%20bd%3Asample.limit%2020000%20}%0A%20%20%3Fmardi_item%20wdt%3AP27%20%3Fdoi%20.%0A%20%20BIND(UCASE(%3Fdoi)%20AS%20%3FDOI)%0A%20%20FILTER%20NOT%20EXISTS%20{%20%3Fmardi_item%20wdt%3AP12%20%3FWikidataQID%20}%20%0A%20%20service%20%3Chttps%3A%2F%2Fquery.wikidata.org%2Fsparql%3E%20{%0A%20%20%20%20%3Fwikidata%20wikidata_wdt%3AP356%20%3FDOI%20.%0A%20%20%20%20%3Fwikidata%20rdfs%3Alabel%20%3Fwikidata_label%20.%0A%20%20%20%20FILTER%20(LANG(%3Fwikidata_label)%20%3D%20%22en%22)%0A%20%20}%0A%20%20%3Fmardi_item%20rdfs%3Alabel%20%3Fmardi_item_label%20.%0A%20%20FILTER%20(LANG(%3Fmardi_item_label)%20%3D%20%22en%22)%0A}%0ALIMIT%201000 Check] whether there are MaRDI items with P205 (external URL) statements and without P12 (Wikidata QID) statements. If so, use MaRDI's QuickStatements tool to add that query's results to MaRDI ([https://portal.mardi4nfdi.de/w/index.php?title=Item%3AQ6197582&diff=37853390&oldid=37649161 sample edit]).
# Write the query, example <syntaxhighlight lang=sparql>
# Write the query, example <syntaxhighlight lang=sparql>
SELECT ?qID ?item ?eudmlid WHERE {
SELECT ?MaRDI_qID ?MaRDI_item ?EuDMLid WHERE {
   ?item wdt:P205 ?url ;
   ?MaRDI_item wdt:P205 ?URL;
        wdt:P12 ?qID
              wdt:P12 ?Wikidata_qID.
   FILTER (STRSTARTS(str(?url) ,"https://eudml")) .
   FILTER(STRSTARTS(STR(?URL), "https://eudml"))
   BIND (substr(str(?url),23) as ?eudmlid)
   BIND(SUBSTR(STR(?URL), 23 ) AS ?EuDMLid)
}
}
LIMIT 10
LIMIT 10
</syntaxhighlight> and investiage the result.  
</syntaxhighlight> and investigate the result.  
# Fid the properties in Wikidata, here only P11166 for eudml.
#* Check for total number of MaRDI items with both (i) a Wikidata QID statement and (ii) a EuDML-based URL link ([https://query.portal.mardi4nfdi.de/#SELECT%20%28COUNT%28%3FMaRDI_item%29%20AS%20%3Fcount%29%20WHERE%20%7B%0A%20%20%3FMaRDI_item%20wdt%3AP205%20%3FURL%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20wdt%3AP12%20%3FWikidata_qID.%0A%20%20FILTER%28STRSTARTS%28STR%28%3FURL%29%2C%20%22https%3A%2F%2Feudml%22%29%29%0A%20%20BIND%28SUBSTR%28STR%28%3FURL%29%2C%2023%20%29%20AS%20%3FEuDMLid%29%0A%7D%0A 9296] as of 7 October 2024)
# Find the properties in Wikidata, here only P11166 for eudml.
# Check whether Wikidata already has a corresponding statement.
##* This does [https://query.portal.mardi4nfdi.de/#%23%20Add%20help%20text%0A%0APREFIX%20wdt%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fprop%2Fdirect%2F%3E%0APREFIX%20wd%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fentity%2F%3E%0A%0APREFIX%20wikidata_wd%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fentity%2F%3E%0APREFIX%20wikidata_wdt%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fprop%2Fdirect%2F%3E%0A%0A%0ASELECT%20%3FMaRDI_qID%20%3FMaRDI_item%20%3FEuDMLid%20WHERE%20%7B%0A%20%20%3FMaRDI_item%20wdt%3AP205%20%3FURL%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20wdt%3AP12%20%3FWikidata_qID.%0A%20%20FILTER%28STRSTARTS%28STR%28%3FURL%29%2C%20%22https%3A%2F%2Feudml%22%29%29%0A%20%20BIND%28SUBSTR%28STR%28%3FURL%29%2C%2023%20%29%20AS%20%3FEuDMLid%29%0A%23%20%20%20FILTER%20NOT%20EXISTS%20%7B%20%3Fmardi_item%20wdt%3AP12%20%3FWikidataQID%20%7D%20%0A%20%20service%20%3Chttps%3A%2F%2Fquery.wikidata.org%2Fsparql%3E%20%7B%0A%20%20%20%20%3Fwikidata%20wikidata_wdt%3AP11166%20%3FEuDMLid%20.%0A%20%20%20%20%3Fwikidata%20rdfs%3Alabel%20%3Fwikidata_label%20.%0A%20%20%20%20FILTER%20%28LANG%28%3Fwikidata_label%29%20%3D%20%22en%22%29%0A%20%20%7D%0A%20%20%0A%7D%0A%23%20LIMIT%2010 not scale well]. Perhaps need to [https://query.portal.mardi4nfdi.de/#%23%20Add%20help%20text%0A%0APREFIX%20wdt%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fprop%2Fdirect%2F%3E%0APREFIX%20wd%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fentity%2F%3E%0A%0APREFIX%20wikidata_wd%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fentity%2F%3E%0APREFIX%20wikidata_wdt%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fprop%2Fdirect%2F%3E%0A%0A%0ASELECT%20%3FMaRDI_item%20%3FURL%20%3FEuDMLid%20%3FWikidata_item%20WHERE%20%7B%0A%20%20%3FMaRDI_item%20wdt%3AP205%20%3FURL%20%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20wdt%3AP27%20%3Fdoi%20%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20wdt%3AP12%20%20%3FWikidataQID%20.%0A%20%20%0A%20%20FILTER%28STRSTARTS%28STR%28%3FURL%29%2C%20%22https%3A%2F%2Feudml%22%29%29%0A%20%20BIND%28SUBSTR%28STR%28%3FURL%29%2C%2023%20%29%20AS%20%3FEuDMLid%29%0A%0A%20%20BIND%28UCASE%28%3Fdoi%29%20AS%20%3FDOI%29%0A%20%20%0A%20%20service%20%3Chttps%3A%2F%2Fquery.wikidata.org%2Fsparql%3E%20%7B%0A%20%20%20%20%3FWikidata_item%20wikidata_wdt%3AP356%20%3FDOI%20.%0A%20%20%20%20FILTER%20NOT%20EXISTS%20%7B%0A%20%20%20%20%20%20%3FWikidata_item%20wikidata_wdt%3AP11166%20%5B%5D%20.%0A%20%20%20%20%7D%20%20%0A%20%20%7D%0A%20%20%0A%7D%0ALIMIT%2010 use DOI] or zbMATH ID ([https://query.portal.mardi4nfdi.de/#%23%20Add%20help%20text%0A%0APREFIX%20wdt%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fprop%2Fdirect%2F%3E%0APREFIX%20wd%3A%20%3Chttps%3A%2F%2Fportal.mardi4nfdi.de%2Fentity%2F%3E%0A%0APREFIX%20wikidata_wd%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fentity%2F%3E%0APREFIX%20wikidata_wdt%3A%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fprop%2Fdirect%2F%3E%0A%0A%0ASELECT%20%3FMaRDI_item%20%3FURL%20%3FEuDMLid%20%3FWikidata_item%20WHERE%20%7B%0A%20%20%3FMaRDI_item%20wdt%3AP205%20%3FURL%20%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20wdt%3AP225%20%3FzbMATH_doc%20%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20wdt%3AP12%20%20%3FWikidataQID%20.%0A%20%20%0A%20%20FILTER%28STRSTARTS%28STR%28%3FURL%29%2C%20%22https%3A%2F%2Feudml%22%29%29%0A%20%20BIND%28SUBSTR%28STR%28%3FURL%29%2C%2023%20%29%20AS%20%3FEuDMLid%29%0A%0A%23%20%20%20BIND%28UCASE%28%3Fdoi%29%20AS%20%3FDOI%29%0A%20%20%0A%20%20service%20%3Chttps%3A%2F%2Fquery.wikidata.org%2Fsparql%3E%20%7B%0A%20%20%20%20%3FWikidata_item%20wikidata_wdt%3AP894%20%3FzbMATH_doc%20.%0A%20%20%20%20MINUS%20%7B%0A%20%20%20%20%20%20%3FWikidata_item%20wikidata_wdt%3AP11166%20%5B%5D%20.%0A%20%20%20%20%7D%20%20%0A%20%20%7D%0A%20%20%0A%7D%0ALIMIT%2010 doc]/ person) as a bridge
## If yes, check consistency with MaRDI.
### To do
## If no, prepare upload to Wikidata.
### To do
# Rewrite the query to match the [https://www.wikidata.org/wiki/Help:QuickStatements Quickstatements CSV format] <syntaxhighlight lang=sparql>
# Rewrite the query to match the [https://www.wikidata.org/wiki/Help:QuickStatements Quickstatements CSV format] <syntaxhighlight lang=sparql>
SELECT ?qid ?P11166 WHERE {
SELECT ?qid ?P11166 WHERE {
   ?item wdt:P205 ?url ;
   ?qal854 wdt:P205 ?url ;
         wdt:P12 ?qid
         wdt:P12 ?qid
   FILTER (STRSTARTS(str(?url) ,"https://eudml")) .
   FILTER (STRSTARTS(str(?url) ,"https://eudml")) .
   BIND (substr(str(?url),23) as ?P11166)
   BIND (Concat(  
    "\"", ## Required for external identifiers by quickstatements
    SUBSTR(STR(?url),23), ## 23 = Length of the prefix
    "\"") as ?P11166)
   }
   }
LIMIT 3
LIMIT 3
</syntaxhighlight>
</syntaxhighlight>
# Download the CSV file and test with a few items on Wikidata
# Download the CSV file and test with a few items on Wikidata
<!--# Optional add qualifier linking back to the source of the information (MaRDI) <syntaxhighlight lang=sparql>
# Optional add source linking back to the source of the information (MaRDI) <syntaxhighlight lang=sparql>
SELECT ?qid ?P11166 (str(?item) as ?qal854) WHERE {
SELECT ?qid ?P11166 ?S854 WHERE {
   ?item wdt:P205 ?url ;
   ?item wdt:P205 ?url ;
         wdt:P12 ?qid
         wdt:P12 ?qid
   FILTER (STRSTARTS(str(?url) ,"https://eudml")) .
   FILTER (STRSTARTS(str(?url) ,"https://eudml")) .
   BIND (substr(str(?url),23) as ?P11166)
   BIND (Concat(  
    "\"", ## Required for external identifiers by quickstatements
    SUBSTR(STR(?url),23), ## 23 = Length of the prefix
    "\"") as ?P11166).
    BIND (Concat(
    "\"", ## Required for external identifiers by quickstatements
    STR(?item),
    "\"") as ?S854)
   }
   }
LIMIT 3
LIMIT 3
</syntaxhighlight> Note this step is not applicable as external identifiers should not have source information-->
</syntaxhighlight>
# Remove the limit and do the full import
# Remove the limit and do the full import

Latest revision as of 12:56, 10 October 2024

About

This page assists in managing queries that produce output in a format that can serve as input to QuickStatements, either on the MaRDI instance or on the Wikidata one. Other Wikibase instances also often have a QuickStatements instance that can be used in an analogous fashion.

Data transfer to Wikidata

This section describes how to export data from the MaRDI portal to Wikidata to make the data better findable for the general public.

Example EuDML proposed by Vladimir Alexiev in 2022, which had 17 results as of 6 October 2024.

  1. Prepare query for the MaRDI SPARQL endpoint
    1. Find the MaRDI qIDs with a statement using P12 = WikidataQId
    2. The thing you want to export, here a part of the external url P205
      • Check for total number of MaRDI items with a P205 (External URL) statement. (170197 as of 7 October 2024)
      • Filter the URLs for the pattern of interest — in our case, this would be "starts with https://eudml.org/doc/" (7115 as of 7 October 2024)
    3. Check whether there are MaRDI items with P205 (external URL) statements and without P12 (Wikidata QID) statements. If so, use MaRDI's QuickStatements tool to add that query's results to MaRDI (sample edit).
  2. Write the query, example
    SELECT ?MaRDI_qID ?MaRDI_item ?EuDMLid WHERE {
      ?MaRDI_item wdt:P205 ?URL;
                  wdt:P12 ?Wikidata_qID.
      FILTER(STRSTARTS(STR(?URL), "https://eudml"))
      BIND(SUBSTR(STR(?URL), 23 ) AS ?EuDMLid)
    }
    LIMIT 10
    
    and investigate the result.
    • Check for total number of MaRDI items with both (i) a Wikidata QID statement and (ii) a EuDML-based URL link (9296 as of 7 October 2024)
  3. Find the properties in Wikidata, here only P11166 for eudml.
  4. Check whether Wikidata already has a corresponding statement.
    1. If yes, check consistency with MaRDI.
      1. To do
    2. If no, prepare upload to Wikidata.
      1. To do
  5. Rewrite the query to match the Quickstatements CSV format
    SELECT ?qid ?P11166 WHERE {
      ?qal854 wdt:P205 ?url ;
            wdt:P12 ?qid
      FILTER (STRSTARTS(str(?url) ,"https://eudml")) .
      BIND (Concat( 
        "\"", ## Required for external identifiers by quickstatements
        SUBSTR(STR(?url),23), ## 23 = Length of the prefix
        "\"") as ?P11166)
      }
    LIMIT 3
    
  6. Download the CSV file and test with a few items on Wikidata
  7. Optional add source linking back to the source of the information (MaRDI)
    SELECT ?qid ?P11166 ?S854 WHERE {
      ?item wdt:P205 ?url ;
            wdt:P12 ?qid
      FILTER (STRSTARTS(str(?url) ,"https://eudml")) .
      BIND (Concat( 
        "\"", ## Required for external identifiers by quickstatements
        SUBSTR(STR(?url),23), ## 23 = Length of the prefix
        "\"") as ?P11166).
        BIND (Concat( 
        "\"", ## Required for external identifiers by quickstatements
        STR(?item), 
        "\"") as ?S854)
      }
    LIMIT 3
    
  8. Remove the limit and do the full import