Training material
LLMs on Kubernetes hands-on
Running private AI yourself
From kind on your laptop to RKE2 with GPUs on Proxmox – run vLLM, LiteLLM, Open WebUI and code assistants yourself.
- vLLM
- LiteLLM
- Open WebUI
- Code-LLM
- GPU Operator
- DRA
Training plan
- Foreword
- Day 0: The Training Ground
- Day 1: kind Cluster for AI Workloads
- Day 2: First Model in a Pod
- Day 3: vLLM Basics – Engine, Sizing, Manifest
- Day 4: LiteLLM as a Gateway – Aliases, Keys, Budgets
- Day 5: Chat with Open WebUI in kind
- Day 6: Preparing Proxmox
- Day 7: Six VMs with OpenTofu
- Day 8: RKE2 HA with Ansible
- Day 9: Platform Foundation
- Day 10: GPUs in Kubernetes
- Day 11: vLLM on the GPU
- Day 12: LiteLLM in Production
- Day 13: LLM Chat for the Team – Open WebUI, Keycloak and RAG with Qdrant
- Day 14: Code LLM – Autocomplete and Chat in VS Code
- Day 15: Operations and Competition
- Appendix A: Cheat Sheet
- Appendix B: Final Exam