Day 5: Chat with Open WebUI in kind
Goal: Open WebUI runs at
http://chat.ki.localtest.me, talks exclusively through its own LiteLLM key, and has an admin, users, persistent data and document search viaembed. · Duration: approx. 75 min · Prerequisite: Day 4
Briefing
- Open WebUI (tested: 0.11.4, chart
open-webui/open-webui16.6.0) is the chat UI for the team: users, roles, histories, documents (RAG). - It gets one virtual key from LiteLLM. Budgets and limits apply to the whole UI; from Day 13 on, you separate individuals via SSO and groups.
- The chart ships with Ollama, Pipelines and Redis – in the lab you switch all of them off and use the slim image (167 MB instead of 1.4 GB compressed).
- Data (SQLite, uploads, vectors) lives under
/app/backend/dataon a PVC. - Many settings are PersistentConfig: after the first start, the value in the database applies, no longer the environment variable.
Browser ── http://chat.ki.localtest.me ──► ingress-nginx
│
ki-chat: open-webui (StatefulSet, PVC 2Gi) ──┘
│ OPENAI_API_BASE_URL + key "open-webui" │ RAG: embed (384 dim)
▼ ▼
ki-gateway: litellm:4000 ──► ki-models: llamacpp / llamacpp-embed
Drill 1: A dedicated key for the UI
mkdir -p ~/ki-lab/tag05 ~/ki-lab/code/open-webui
M="Authorization: Bearer sk-lab-master-1234"; J="Content-Type: application/json"
GW=http://llm.ki.localtest.me
curl -s $GW/key/generate -H "$M" -H "$J" -d '{
"key_alias": "open-webui",
"models": ["chat-small", "chat-large", "code", "embed"],
"max_budget": 5, "budget_duration": "30d", "rpm_limit": 60}' \
| tee ~/ki-lab/tag05/owui-key.json \
| jq '{key_alias, models, max_budget, rpm_limit, key: (.key[0:6] + "...")}'Expected (live against a local proxy, key will differ):
{
"key_alias": "open-webui",
"models": [
"chat-small",
"chat-large",
"code",
"embed"
],
"max_budget": 5.0,
"rpm_limit": 60,
"key": "sk-Fq_..."
}
Three values go into a Secret: the LiteLLM key, the key for login tokens, and the admin password.
kubectl -n ki-chat create secret generic owui-secrets \
--from-literal=litellm-key="$(jq -r .key ~/ki-lab/tag05/owui-key.json)" \
--from-literal=webui-secret="$(openssl rand -hex 32)" \
--from-literal=admin-password=lab-admin-1234
kubectl -n ki-chat get secret owui-secrets -o json | jq -r '.data | keys[]'
rm ~/ki-lab/tag05/owui-key.jsonExpected (not tested live in kind):
secret/owui-secrets created
admin-password
litellm-key
webui-secret
Security:
WEBUI_SECRET_KEYsigns the login tokens (JWT). If it’s missing, Open WebUI generates a random one – and then everyone is logged out after every restart.lab-admin-1234is a lab value.
Save your progress? Sign up for free
Drill 2: Read and render the values
helm repo add open-webui https://helm.openwebui.com/ 2>/dev/null
helm repo update open-webui >/dev/null
helm show chart open-webui/open-webui --version 16.6.0 \
| grep -E '^(name|version|appVersion):'Expected:
appVersion: 0.11.4
name: open-webui
version: 16.6.0
The lab values (also in the book’s code under
open-webui/values-kind.yaml):
cat > ~/ki-lab/code/open-webui/values-kind.yaml <<'EOF'
# Open WebUI in the kind lab (Day 5) – chart open-webui/open-webui 16.6.0
# Models exclusively via LiteLLM; create the Secrets beforehand (owui-secrets)
image:
useSlim: true # ghcr.io/open-webui/open-webui:0.11.4-slim
ollama:
enabled: false # no Ollama of its own in the chart
pipelines:
enabled: false
websocket:
enabled: false # one replica: no Redis needed
openaiBaseApiUrl: http://litellm.ki-gateway:4000/v1
openaiApiKeyExistingSecret: owui-secrets
openaiApiKeyExistingSecretKey: litellm-key
persistence:
enabled: true
size: 2Gi
storageClass: standard # SQLite + uploads + vectors under /app/backend/data
ingress:
enabled: true
class: nginx
host: chat.ki.localtest.me
annotations:
nginx.ingress.kubernetes.io/proxy-body-size: 20m
nginx.ingress.kubernetes.io/proxy-read-timeout: "300"
startupProbe: # first start: DB migrations
httpGet: {path: /health, port: http}
periodSeconds: 10
failureThreshold: 30
readinessProbe:
httpGet: {path: /health, port: http}
periodSeconds: 10
livenessProbe:
httpGet: {path: /health, port: http}
periodSeconds: 20
timeoutSeconds: 5
resources:
requests:
cpu: 250m
memory: 768Mi
limits:
memory: 1536Mi
extraEnvVars:
- name: WEBUI_URL
value: http://chat.ki.localtest.me
- name: WEBUI_SECRET_KEY # signs the login tokens (JWT)
valueFrom:
secretKeyRef: {name: owui-secrets, key: webui-secret}
- name: WEBUI_ADMIN_EMAIL # first admin without click registration
value: [email protected]
- name: WEBUI_ADMIN_PASSWORD
valueFrom:
secretKeyRef: {name: owui-secrets, key: admin-password}
- name: DEFAULT_USER_ROLE
value: pending
- name: BYPASS_MODEL_ACCESS_CONTROL # lab: LiteLLM handles permissions
value: "true"
- name: ENABLE_EVALUATION_ARENA_MODELS
value: "false"
- name: RAG_EMBEDDING_ENGINE # embeddings via LiteLLM instead of in the pod
value: openai
- name: RAG_EMBEDDING_MODEL
value: embed
- name: RAG_OPENAI_API_BASE_URL
value: http://litellm.ki-gateway:4000/v1
- name: RAG_OPENAI_API_KEY
valueFrom:
secretKeyRef: {name: owui-secrets, key: litellm-key}
- name: ENABLE_VERSION_UPDATE_CHECK
value: "false"
EOF| Key | Why |
|---|---|
image.useSlim |
without a built-in embedding model – LiteLLM provides the embeddings |
openaiApiKeyExistingSecret |
never put the key in values.yaml |
WEBUI_ADMIN_EMAIL/_PASSWORD |
first admin at startup; after that, registration is closed |
BYPASS_MODEL_ACCESS_CONTROL |
lab: all users see all models (Drill 5) |
RAG_OPENAI_API_BASE_URL |
must be set explicitly, otherwise RAG calls
api.openai.com |
cd ~/ki-lab/code/open-webui
helm template open-webui open-webui/open-webui --version 16.6.0 -n ki-chat \
-f values-kind.yaml > ~/ki-lab/tag05/owui.yaml
awk '/^kind:/{k=$2} /^ name:/{print k, $2}' ~/ki-lab/tag05/owui.yaml
grep -m2 'image:' ~/ki-lab/tag05/owui.yaml
kubeconform -strict -summary ~/ki-lab/tag05/owui.yamlExpected:
ServiceAccount open-webui-sa
PersistentVolumeClaim open-webui
Service open-webui
StatefulSet open-webui
Ingress open-webui
image: ghcr.io/open-webui/open-webui:0.11.4-slim
image: ghcr.io/open-webui/open-webui:0.11.4-slim
Summary: 5 resources found in 1 file - Valid: 5, Invalid: 0, Errors: 0, Skipped: 0
The first image belongs to the init container
copy-app-data: at startup it copies the default data from
the image onto the empty PVC (cp -R -n, overwrites
nothing).
Save your progress? Sign up for free
Drill 3: Install
helm upgrade --install open-webui open-webui/open-webui --version 16.6.0 \
-n ki-chat -f ~/ki-lab/code/open-webui/values-kind.yaml
kubectl -n ki-chat rollout status sts/open-webui --timeout=600s
kubectl -n ki-chat get pod,pvc,ingress
curl -s http://chat.ki.localtest.me/healthExpected (not tested live in kind, abbreviated):
Release "open-webui" does not exist. Installing it now.
STATUS: deployed
partitioned roll out complete: 1 new pods have been updated...
NAME READY STATUS RESTARTS AGE
pod/open-webui-0 1/1 Running 0 2m
NAME STATUS VOLUME CAPACITY ...
persistentvolumeclaim/open-webui Bound pvc-... 2Gi ...
NAME CLASS HOSTS ...
ingress.networking.k8s.io/open-webui nginx chat.ki.localtest.me ...
{"status":true}
On the first start, Open WebUI creates the database and the admin. In the log:
kubectl -n ki-chat logs open-webui-0 | grep -m2 -i 'admin account' | sed 's/.* - //'Expected (tested live with
pip install open-webui==0.11.4):
Creating admin account from environment variables: [email protected]
Admin account created successfully: [email protected]
Now open http://chat.ki.localtest.me in your browser and
log in with [email protected] / lab-admin-1234. The
model menu shows the four LiteLLM aliases.
Save your progress? Sign up for free
Drill 4: Smoke test via the API
Everything the UI can do also works via REST. The script logs in, lists models and sends a question – ideal for CI and the checkpoint.
cat > ~/ki-lab/code/open-webui/smoke.sh <<'EOF'
#!/usr/bin/env bash
# Smoke test for Open WebUI: login, models, one chat answer via LiteLLM.
# Usage: OWUI=http://chat.ki.localtest.me EMAIL=... PASS=... ./smoke.sh
set -euo pipefail
OWUI=${OWUI:-http://chat.ki.localtest.me}
EMAIL=${EMAIL:[email protected]}
PASS=${PASS:?PASS fehlt}
MODEL=${MODEL:-chat-small}
TOKEN=$(jq -n --arg e "$EMAIL" --arg p "$PASS" '{email: $e, password: $p}' \
| curl -sf "$OWUI/api/v1/auths/signin" -H 'Content-Type: application/json' \
-d @- | jq -r .token)
[ -n "$TOKEN" ] && [ "$TOKEN" != null ] || { echo "✘ Login"; exit 1; }
echo "✔ Login als $EMAIL"
MODELS=$(curl -sf "$OWUI/api/models" -H "Authorization: Bearer $TOKEN" \
| jq -r '[.data[].id] | join(" ")')
echo "✔ Modelle: $MODELS"
ANSWER=$(jq -n --arg m "$MODEL" \
'{model: $m, stream: false,
messages: [{role: "user", content: "Antworte nur mit: OK"}]}' \
| curl -sf "$OWUI/api/chat/completions" -H "Authorization: Bearer $TOKEN" \
-H 'Content-Type: application/json' -d @- \
| jq -r '.choices[0].message.content')
echo "✔ $MODEL antwortet: $ANSWER"
EOF
chmod +x ~/ki-lab/code/open-webui/smoke.sh
PASS=lab-admin-1234 ~/ki-lab/code/open-webui/smoke.shExpected (tested live against open-webui serve +
LiteLLM):
✔ Login als [email protected]
✔ Modelle: chat-small chat-large code embed
✔ chat-small antwortet: OK
Open WebUI queries LiteLLM with the key from
owui-secrets – in LiteLLM, all calls show up under the
alias open-webui, no matter which person asked.
Save your progress? Sign up for free
Drill 5: Users and roles
The admin creates a user. Self-registration has been closed ever since the admin was created via environment variable.
OWUI=http://chat.ki.localtest.me; J="Content-Type: application/json"
login() { jq -n --arg e "$1" --arg p "$2" '{email: $e, password: $p}' \
| curl -s $OWUI/api/v1/auths/signin -H "$J" -d @- | jq -r .token; }
ADMIN=$(login [email protected] lab-admin-1234)
curl -s $OWUI/api/v1/auths/add -H "Authorization: Bearer $ADMIN" -H "$J" \
-d '{"name":"Anna","email":"[email protected]","password":"lab-anna-1234",
"role":"user"}' | jq -c '{name, email, role}'
curl -s $OWUI/api/v1/auths/signup -H "$J" \
-d '{"name":"Eve","email":"[email protected]","password":"lab-eve-1234"}' \
| jq -r .detail | cut -c1-55Expected (tested live with open-webui serve):
{"name":"Anna","email":"[email protected]","role":"user"}
You do not have permission to access this resource. Ple
| Role | May |
|---|---|
pending |
nothing – waits for approval (default for self-registered users) |
user |
chat with approved models, own documents |
admin |
everything: users, connections, settings |
What does Anna see? Without BYPASS_MODEL_ACCESS_CONTROL,
/api/models returns an empty list for regular users, and
every chat ends with Model not found – models only become
visible once the admin makes them public in Admin Settings →
Models or assigns them to a group.
ANNA=$(login [email protected] lab-anna-1234)
curl -s $OWUI/api/models -H "Authorization: Bearer $ANNA" | jq -r '.data[].id'Expected (tested live, with
BYPASS_MODEL_ACCESS_CONTROL=true):
chat-small
chat-large
code
embed
That’s convenient in the lab, because LiteLLM only lets approved aliases through anyway. For the team (Day 13), you control visibility via groups from Keycloak.
Save your progress? Sign up for free
Drill 6: Persistence and the PersistentConfig trap
The pod is gone, the data stays – it lives on the PVC:
kubectl -n ki-chat exec open-webui-0 -- ls /app/backend/data
kubectl -n ki-chat delete pod open-webui-0
kubectl -n ki-chat wait pod/open-webui-0 --for=condition=Ready --timeout=300s
[ -n "$(login [email protected] lab-anna-1234)" ] && echo "Anna existiert noch"Expected (directory tested live, pod restart in kind not tested):
cache
uploads
vector_db
webui.db
pod "open-webui-0" deleted
pod/open-webui-0 condition met
Anna existiert noch
Now the trap: change a setting via environment variable.
kubectl -n ki-chat set env sts/open-webui DEFAULT_USER_ROLE=user
kubectl -n ki-chat rollout status sts/open-webui --timeout=300s
ADMIN=$(login [email protected] lab-admin-1234)
curl -s $OWUI/api/v1/auths/admin/config -H "Authorization: Bearer $ADMIN" \
| jq -c '{DEFAULT_USER_ROLE, ENABLE_SIGNUP}'Expected (tested live with open-webui serve):
{"DEFAULT_USER_ROLE":"pending","ENABLE_SIGNUP":false}
Why? DEFAULT_USER_ROLE is a PersistentConfig: on the
first start, the value moves into the config table, and
from then on the database wins. You can change such values in the admin
UI, via the API (POST /api/v1/auths/admin/config), or
globally with ENABLE_PERSISTENT_CONFIG=false – then only
the environment applies again. Back to the state from Git:
helm upgrade open-webui open-webui/open-webui --version 16.6.0 -n ki-chat \
-f ~/ki-lab/code/open-webui/values-kind.yamlSave your progress? Sign up for free
Drill 7: Ask your documents (RAG via LiteLLM)
On upload, Open WebUI splits the file, fetches embeddings via
embed and stores them in the built-in vector DB on the
PVC.
printf 'Bereitschaft im KI-Lab: Montag bis Freitag hat Team Plattform Dienst.%s\n' \
' Notfallnummer 4711.' > ~/ki-lab/tag05/notiz.txt
FILE=$(curl -s $OWUI/api/v1/files/ -H "Authorization: Bearer $ADMIN" \
-F file=@$HOME/ki-lab/tag05/notiz.txt | jq -r .id)
until curl -s $OWUI/api/v1/files/$FILE/process/status \
-H "Authorization: Bearer $ADMIN" | jq -e '.status == "completed"' >/dev/null
do sleep 2; done
jq -n --arg f "$FILE" '{model: "chat-small", stream: false,
files: [{type: "file", id: $f}],
messages: [{role: "user", content: "Wie lautet die Notfallnummer?"}]}' \
| curl -s $OWUI/api/chat/completions -H "Authorization: Bearer $ADMIN" \
-H "$J" -d @- | jq -r '.choices[0].message.content'Expected (tested live, answer text will differ):
Die Notfallnummer lautet **4711** [1].
[1] refers to the source. LiteLLM’s spend log shows
whether the embeddings really went through LiteLLM:
curl -s "$GW/spend/logs?summarize=false" -H "$M" \
| jq -r '.[] | select(.metadata.user_api_key_alias == "open-webui")
| .model_group' | sort | uniq -cExpected (tested live, numbers will differ):
2 chat-small
1 embed
Warning: Without
RAG_OPENAI_API_BASE_URL, Open WebUI 0.11.4 sends the embeddings tohttps://api.openai.com/v1– the upload fails with401 Unauthorized, and in the worst case documents leave the building. The variable is a PersistentConfig; a wrong first start stays stuck in the DB.
Save your progress? Sign up for free
Checkpoint
check() { if eval "$2" >/dev/null 2>&1; then echo "✔ $1"; else echo "✘ $1"; fi; }
OWUI=http://chat.ki.localtest.me
check "Pod ready" \
'kubectl -n ki-chat wait pod/open-webui-0 --for=condition=Ready --timeout=5s'
check "PVC gebunden" \
'[ "$(kubectl -n ki-chat get pvc open-webui -o jsonpath={.status.phase})" = Bound ]'
check "Health über Ingress" \
'curl -sf $OWUI/health | jq -e .status'
check "Registrierung zu" \
'[ "$(curl -s $OWUI/api/config | jq .features.enable_signup)" = false ]'
check "Rauchtest" \
'PASS=lab-admin-1234 ~/ki-lab/code/open-webui/smoke.sh'
check "Key nicht im Klartext in den Values" \
'! grep -q "sk-" ~/ki-lab/code/open-webui/values-kind.yaml'Challenge
The UI may use at most 30 requests per minute and 1 (lab currency) per month – without restarting Open WebUI or swapping the key.
Hint: Limits are attached to the virtual key in
LiteLLM, not to Open WebUI; /key/update changes them while
running.
Solution
OWUI_KEY=$(kubectl -n ki-chat get secret owui-secrets \
-o jsonpath='{.data.litellm-key}' | base64 -d)
curl -s $GW/key/update -H "$M" -H "$J" \
-d "{\"key\": \"$OWUI_KEY\", \"rpm_limit\": 30, \"max_budget\": 1}" \
| jq -c '{key_alias, rpm_limit, max_budget}'
curl -s "$GW/key/info?key=$OWUI_KEY" -H "$M" \
| jq -c '.info | {key_alias, spend, rpm_limit, max_budget}'Expected (live against a local proxy, spend will
differ):
{"key_alias":"open-webui","rpm_limit":30,"max_budget":1.0}
{"key_alias":"open-webui","spend":0.003651,"rpm_limit":30,"max_budget":1.0}
The key stays the same, and Open WebUI notices nothing – until the
limit kicks in. Then users see LiteLLM’s error message in the chat (429
or budget_exceeded).
Save your progress? Sign up for free
Enterprise Note
- SSO instead of local accounts: OIDC against Keycloak, roles and
model visibility via groups,
ENABLE_LOGIN_FORM=false(Day 13). - Multiple replicas need Postgres instead of SQLite
(
databaseUrl), Redis for websockets, and an RWX volume or S3 for uploads. - Put vectors in a dedicated database (Qdrant, pgvector) instead of the built-in Chroma on the PVC; never switch the embedding model without re-indexing.
- One LiteLLM key or one Open WebUI instance per department if budgets have to be billed separately.
- Backups: back up the PVC or Postgres with Velero/CNPG – chat histories can contain personal data (GDPR, deletion policy).