Project:LLMQueryService POC: Difference between revisions
From MaRDI portal
No edit summary |
No edit summary |
||
| (One intermediate revision by the same user not shown) | |||
| Line 1: | Line 1: | ||
This page describes how to install the proof-of-concept LLM-based query service. | This page describes how to install the proof-of-concept LLM-based query service. | ||
Try it here: http://130.73.240.230/ | |||
(only from the ZIB network or VPN) | |||
=== Using a OpenStack VM === | === Using a OpenStack VM === | ||
# Create a new instance (if you use Debian: at least 12) | # Create a new instance (if you use Debian: at least 12) | ||
## See [[Project:Docker_OpenStackVM]] | |||
# Install necessary libraries | # Install necessary libraries | ||
## apt-get update | ## apt-get update | ||
Latest revision as of 21:34, 10 July 2024
This page describes how to install the proof-of-concept LLM-based query service.
Try it here: http://130.73.240.230/ (only from the ZIB network or VPN)
Using a OpenStack VM
- Create a new instance (if you use Debian: at least 12)
- Install necessary libraries
- apt-get update
- apt-get install git python3-pip python3-venv
- Clone the repository and follow the rest of the manual ( https://git.zib.de/bzfconra/mardi_llm_bottest )
- Install Ollama ( https://ollama.com/download/linux )
- Check the logs: journalctl -u ollama -f
- You might need ssh-port-forwarding
- Example: ssh -L 8000:127.0.0.1:8501 -i OPENSTACK_KEY_FILE.pem debian@130.73.240.230
