Self-Hosted AI Stack

Self-Hosted AI Stack

← All projects
Docker GPU Passthrough Ollama Proxmox

Designed and deployed a complete AI inference platform on a Proxmox VM with NVIDIA RTX 2000E Ada GPU passthrough. The stack runs via Docker Compose orchestrating Ollama for model inference, Open WebUI for the chat interface, Qdrant for vector search, and OpenClaw for multi-agent workflows. Models are managed through Docker bind mounts for persistence, with a curated lineup optimized for the available VRAM budget.

GPU Passthrough Setup Guide
Markdown
Model VRAM Budget Spreadsheet
Spreadsheet
docker-compose.yml
YAML · 3 KB