Project:LLMQueryService POC: Difference between revisions
From MaRDI portal
No edit summary |
No edit summary |
||
| Line 3: | Line 3: | ||
=== Using a OpenStack VM === | === Using a OpenStack VM === | ||
# Create a new instance (if you use Debian: at least 12) | # Create a new instance (if you use Debian: at least 12) | ||
## See [[Project:Docker_OpenStackVM]] | |||
# Install necessary libraries | # Install necessary libraries | ||
## apt-get update | ## apt-get update | ||
Revision as of 20:30, 10 July 2024
This page describes how to install the proof-of-concept LLM-based query service.
Using a OpenStack VM
- Create a new instance (if you use Debian: at least 12)
- Install necessary libraries
- apt-get update
- apt-get install git python3-pip python3-venv
- Clone the repository and follow the rest of the manual ( https://git.zib.de/bzfconra/mardi_llm_bottest )
- Install Ollama ( https://ollama.com/download/linux )
- Check the logs: journalctl -u ollama -f
- You might need ssh-port-forwarding
- Example: ssh -L 8000:127.0.0.1:8501 -i OPENSTACK_KEY_FILE.pem debian@130.73.240.230
