Answers to specific technical questions

Sammelseite für offene Fragen (English / Deutsch).

Portal-Compose:

These are questions related to Readme.md in the project:

1. submodules init not necessary anymore? so it can be moved to Docker-Wikibase or completely removed from Readme

2. Add volume in dev-extensions for extension in readme.md in develop-locally section (Johannes)

3. readme.md for ci: why is it required to set default test-passwords in ci as the readme suggests, some tests seem to fail locally with default password ? Point out that the CI steps are defined in main.yml. Also point out that CI builds is triggered from GitHub itself and that the trigger usually on commit, and that this can be defined in the github environment.

4. Test locally: run_tests.sh. Add additional note in readme for this: Run compose-up locally then execute script

5. Admin password doesn't change locally (from default password) although all containers deleted before and new password defined in env variable, so the local test for this also fails.

6. Deploy on the MaRDI server: notes seem ok, but incomplete, how is deployment done?

7. Documentation for traefik missing.

8. Add hint in documentation of portal-compose on the linked repositories which create the custom portal containers

Portal-Examples:

- Check PR

- WB_wikidata_properties.ipynb recheck with creds. See in PR.

- Other scripts seem ok

- Reminder:, nochmal alle Scripte hier kurz durchgehen im Meeting. Question for discussion, should scripts for constant data updates running in cli be stored as ipynb format?

docker-importer:

- The docker-importer is a docker-container which has functionalities for data-import (e.g. from swMATH, zbMATH), and can trigger the import of data in a wikibase container in the same docker-composition cyclicly. The import scripts in the importer are written in python and examples for these scripts can be seen as Jupyter-Notebooks in the repository Portal-Examples. " Is this description correct ? If so, could you add it to the README.md ?

- If the data-import will go live, the data-importer will be located in portal-compose files. The current state is WIP and therefore a custom compose with wikibase is provided. Is it correct ?

- https://github.com/MaRDI4NFDI/docker-importer/blob/main/doc/activity.drawio.svg shows that data is read from wikibase database, but not stored somewhere If this is correct, what are the plans and the current state on updating the fetched data in wikibase?

- The shellscript import.sh is meant to trigger the import process as soon as the crontab is activated ? The script import.sh is meant to invoke the pythonscript src/import.py and this starts the import mechanisms. The exact functionality of import.py is defined by the cli-parameters on invoke? Why is import.sh not calling the python script ?

- Could the docker importer already be located in portal-compose and with a flag defining that the cron-pattern is deactivated and an external shellscript which triggers the import manually in a next-step ?

Testing-Concept:

- Tests in Selenium are defined like this: https://github.com/MaRDI4NFDI/portal-compose/blob/main/test/MathExtensionsTest.py Would it make sense for the 'System-Testing' just to have a selenium-ci container running which tests against the url deployed portal.mardi4nfdi.de. This could be triggered with a shellscript similar like the local unit-tests after the deployment (and not by github CI). Some cases of course cant be validated by an 'extrernally' running script. Have a flag which tags external testcases and an url switch should make it possible to reuse the already written testcases.

Future MaRDI Steps:

- Suggestions and Discussion on Future MaRDI Steps which not already have been explicitly mentioned in the other questions.

Overall:

- Could you re-check everybody has admin-access for all repos in MaRDI4NFDI ?

- Also for Trafik dashboard https://traefik.portal.mardi4nfdi.de if this relates to aot

See also: Technical_introduction

Backup

Backup and restore

How to configure automatically and manually backup the data, and restore a backup.

Testing

Testing concept

General guidelines about testing the MaRDI Portal.

Selenium, see also Deployment

Selenium container documentation

How to import data in the portal (in development)

Import process overview

UML action diagram showing import process from zwMath.

Import properties from Wikidata

How to to import items and properties from Wikidata into the Portal.

Read data from zbMath

There are millions of references to papers in the zbMath database. We just need (for now) those related to the list of mathematical software that has been imported into the MaRDI-Portal.

Populate the portal using the Wikibase API

Import the data read from zbMath into the data structure setup from properties imported from Wikidata.

Where to ask for help when new wikibase features are needed

WBSG projects and WBSG Rhizome Loomio (requires login)

The Wikibase Stakeholder Group coordinates a variety of projects for the broader Wikibase ecosystem.