Skip to content
Hands-on IT IT Drills

Day 5: Chat with Open WebUI in kind

Goal: Open WebUI runs at http://chat.ki.localtest.me, talks exclusively through its own LiteLLM key, and has an admin, users, persistent data and document search via embed. · Duration: approx. 75 min · Prerequisite: Day 4

Briefing

  • Open WebUI (tested: 0.11.4, chart open-webui/open-webui 16.6.0) is the chat UI for the team: users, roles, histories, documents (RAG).
  • It gets one virtual key from LiteLLM. Budgets and limits apply to the whole UI; from Day 13 on, you separate individuals via SSO and groups.
  • The chart ships with Ollama, Pipelines and Redis – in the lab you switch all of them off and use the slim image (167 MB instead of 1.4 GB compressed).
  • Data (SQLite, uploads, vectors) lives under /app/backend/data on a PVC.
  • Many settings are PersistentConfig: after the first start, the value in the database applies, no longer the environment variable.
 Browser ── http://chat.ki.localtest.me ──► ingress-nginx
                                              │
 ki-chat: open-webui (StatefulSet, PVC 2Gi) ──┘
   │ OPENAI_API_BASE_URL + key "open-webui"   │ RAG: embed (384 dim)
   ▼                                          ▼
 ki-gateway: litellm:4000 ──► ki-models: llamacpp / llamacpp-embed

Drill 1: A dedicated key for the UI

mkdir -p ~/ki-lab/tag05 ~/ki-lab/code/open-webui
M="Authorization: Bearer sk-lab-master-1234"; J="Content-Type: application/json"
GW=http://llm.ki.localtest.me
curl -s $GW/key/generate -H "$M" -H "$J" -d '{
  "key_alias": "open-webui",
  "models": ["chat-small", "chat-large", "code", "embed"],
  "max_budget": 5, "budget_duration": "30d", "rpm_limit": 60}' \
  | tee ~/ki-lab/tag05/owui-key.json \
  | jq '{key_alias, models, max_budget, rpm_limit, key: (.key[0:6] + "...")}'

Expected (live against a local proxy, key will differ):

{
  "key_alias": "open-webui",
  "models": [
    "chat-small",
    "chat-large",
    "code",
    "embed"
  ],
  "max_budget": 5.0,
  "rpm_limit": 60,
  "key": "sk-Fq_..."
}

Three values go into a Secret: the LiteLLM key, the key for login tokens, and the admin password.

kubectl -n ki-chat create secret generic owui-secrets \
  --from-literal=litellm-key="$(jq -r .key ~/ki-lab/tag05/owui-key.json)" \
  --from-literal=webui-secret="$(openssl rand -hex 32)" \
  --from-literal=admin-password=lab-admin-1234
kubectl -n ki-chat get secret owui-secrets -o json | jq -r '.data | keys[]'
rm ~/ki-lab/tag05/owui-key.json

Expected (not tested live in kind):

secret/owui-secrets created
admin-password
litellm-key
webui-secret

Security: WEBUI_SECRET_KEY signs the login tokens (JWT). If it’s missing, Open WebUI generates a random one – and then everyone is logged out after every restart. lab-admin-1234 is a lab value.

Save your progress? Sign up for free

Drill 2: Read and render the values

helm repo add open-webui https://helm.openwebui.com/ 2>/dev/null
helm repo update open-webui >/dev/null
helm show chart open-webui/open-webui --version 16.6.0 \
  | grep -E '^(name|version|appVersion):'

Expected:

appVersion: 0.11.4
name: open-webui
version: 16.6.0

The lab values (also in the book’s code under open-webui/values-kind.yaml):

cat > ~/ki-lab/code/open-webui/values-kind.yaml <<'EOF'
# Open WebUI in the kind lab (Day 5) – chart open-webui/open-webui 16.6.0
# Models exclusively via LiteLLM; create the Secrets beforehand (owui-secrets)
image:
  useSlim: true                  # ghcr.io/open-webui/open-webui:0.11.4-slim
ollama:
  enabled: false                 # no Ollama of its own in the chart
pipelines:
  enabled: false
websocket:
  enabled: false                 # one replica: no Redis needed

openaiBaseApiUrl: http://litellm.ki-gateway:4000/v1
openaiApiKeyExistingSecret: owui-secrets
openaiApiKeyExistingSecretKey: litellm-key

persistence:
  enabled: true
  size: 2Gi
  storageClass: standard         # SQLite + uploads + vectors under /app/backend/data

ingress:
  enabled: true
  class: nginx
  host: chat.ki.localtest.me
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: 20m
    nginx.ingress.kubernetes.io/proxy-read-timeout: "300"

startupProbe:                    # first start: DB migrations
  httpGet: {path: /health, port: http}
  periodSeconds: 10
  failureThreshold: 30
readinessProbe:
  httpGet: {path: /health, port: http}
  periodSeconds: 10
livenessProbe:
  httpGet: {path: /health, port: http}
  periodSeconds: 20
  timeoutSeconds: 5

resources:
  requests:
    cpu: 250m
    memory: 768Mi
  limits:
    memory: 1536Mi

extraEnvVars:
  - name: WEBUI_URL
    value: http://chat.ki.localtest.me
  - name: WEBUI_SECRET_KEY       # signs the login tokens (JWT)
    valueFrom:
      secretKeyRef: {name: owui-secrets, key: webui-secret}
  - name: WEBUI_ADMIN_EMAIL      # first admin without click registration
    value: [email protected]
  - name: WEBUI_ADMIN_PASSWORD
    valueFrom:
      secretKeyRef: {name: owui-secrets, key: admin-password}
  - name: DEFAULT_USER_ROLE
    value: pending
  - name: BYPASS_MODEL_ACCESS_CONTROL   # lab: LiteLLM handles permissions
    value: "true"
  - name: ENABLE_EVALUATION_ARENA_MODELS
    value: "false"
  - name: RAG_EMBEDDING_ENGINE   # embeddings via LiteLLM instead of in the pod
    value: openai
  - name: RAG_EMBEDDING_MODEL
    value: embed
  - name: RAG_OPENAI_API_BASE_URL
    value: http://litellm.ki-gateway:4000/v1
  - name: RAG_OPENAI_API_KEY
    valueFrom:
      secretKeyRef: {name: owui-secrets, key: litellm-key}
  - name: ENABLE_VERSION_UPDATE_CHECK
    value: "false"
EOF
Key Why
image.useSlim without a built-in embedding model – LiteLLM provides the embeddings
openaiApiKeyExistingSecret never put the key in values.yaml
WEBUI_ADMIN_EMAIL/_PASSWORD first admin at startup; after that, registration is closed
BYPASS_MODEL_ACCESS_CONTROL lab: all users see all models (Drill 5)
RAG_OPENAI_API_BASE_URL must be set explicitly, otherwise RAG calls api.openai.com
cd ~/ki-lab/code/open-webui
helm template open-webui open-webui/open-webui --version 16.6.0 -n ki-chat \
  -f values-kind.yaml > ~/ki-lab/tag05/owui.yaml
awk '/^kind:/{k=$2} /^  name:/{print k, $2}' ~/ki-lab/tag05/owui.yaml
grep -m2 'image:' ~/ki-lab/tag05/owui.yaml
kubeconform -strict -summary ~/ki-lab/tag05/owui.yaml

Expected:

ServiceAccount open-webui-sa
PersistentVolumeClaim open-webui
Service open-webui
StatefulSet open-webui
Ingress open-webui
        image: ghcr.io/open-webui/open-webui:0.11.4-slim
        image: ghcr.io/open-webui/open-webui:0.11.4-slim
Summary: 5 resources found in 1 file - Valid: 5, Invalid: 0, Errors: 0, Skipped: 0

The first image belongs to the init container copy-app-data: at startup it copies the default data from the image onto the empty PVC (cp -R -n, overwrites nothing).

Save your progress? Sign up for free

Drill 3: Install

helm upgrade --install open-webui open-webui/open-webui --version 16.6.0 \
  -n ki-chat -f ~/ki-lab/code/open-webui/values-kind.yaml
kubectl -n ki-chat rollout status sts/open-webui --timeout=600s
kubectl -n ki-chat get pod,pvc,ingress
curl -s http://chat.ki.localtest.me/health

Expected (not tested live in kind, abbreviated):

Release "open-webui" does not exist. Installing it now.
STATUS: deployed
partitioned roll out complete: 1 new pods have been updated...
NAME               READY   STATUS    RESTARTS   AGE
pod/open-webui-0   1/1     Running   0          2m
NAME                                      STATUS   VOLUME   CAPACITY   ...
persistentvolumeclaim/open-webui          Bound    pvc-...  2Gi        ...
NAME                                   CLASS   HOSTS                  ...
ingress.networking.k8s.io/open-webui   nginx   chat.ki.localtest.me   ...
{"status":true}

On the first start, Open WebUI creates the database and the admin. In the log:

kubectl -n ki-chat logs open-webui-0 | grep -m2 -i 'admin account' | sed 's/.* - //'

Expected (tested live with pip install open-webui==0.11.4):

Creating admin account from environment variables: [email protected]
Admin account created successfully: [email protected]

Now open http://chat.ki.localtest.me in your browser and log in with [email protected] / lab-admin-1234. The model menu shows the four LiteLLM aliases.

Save your progress? Sign up for free

Drill 4: Smoke test via the API

Everything the UI can do also works via REST. The script logs in, lists models and sends a question – ideal for CI and the checkpoint.

cat > ~/ki-lab/code/open-webui/smoke.sh <<'EOF'
#!/usr/bin/env bash
# Smoke test for Open WebUI: login, models, one chat answer via LiteLLM.
# Usage: OWUI=http://chat.ki.localtest.me EMAIL=... PASS=... ./smoke.sh
set -euo pipefail
OWUI=${OWUI:-http://chat.ki.localtest.me}
EMAIL=${EMAIL:[email protected]}
PASS=${PASS:?PASS fehlt}
MODEL=${MODEL:-chat-small}

TOKEN=$(jq -n --arg e "$EMAIL" --arg p "$PASS" '{email: $e, password: $p}' \
  | curl -sf "$OWUI/api/v1/auths/signin" -H 'Content-Type: application/json' \
      -d @- | jq -r .token)
[ -n "$TOKEN" ] && [ "$TOKEN" != null ] || { echo "✘ Login"; exit 1; }
echo "✔ Login als $EMAIL"

MODELS=$(curl -sf "$OWUI/api/models" -H "Authorization: Bearer $TOKEN" \
  | jq -r '[.data[].id] | join(" ")')
echo "✔ Modelle: $MODELS"

ANSWER=$(jq -n --arg m "$MODEL" \
    '{model: $m, stream: false,
      messages: [{role: "user", content: "Antworte nur mit: OK"}]}' \
  | curl -sf "$OWUI/api/chat/completions" -H "Authorization: Bearer $TOKEN" \
      -H 'Content-Type: application/json' -d @- \
  | jq -r '.choices[0].message.content')
echo "✔ $MODEL antwortet: $ANSWER"
EOF
chmod +x ~/ki-lab/code/open-webui/smoke.sh
PASS=lab-admin-1234 ~/ki-lab/code/open-webui/smoke.sh

Expected (tested live against open-webui serve + LiteLLM):

✔ Login als [email protected]
✔ Modelle: chat-small chat-large code embed
✔ chat-small antwortet: OK

Open WebUI queries LiteLLM with the key from owui-secrets – in LiteLLM, all calls show up under the alias open-webui, no matter which person asked.

Save your progress? Sign up for free

Drill 5: Users and roles

The admin creates a user. Self-registration has been closed ever since the admin was created via environment variable.

OWUI=http://chat.ki.localtest.me; J="Content-Type: application/json"
login() { jq -n --arg e "$1" --arg p "$2" '{email: $e, password: $p}' \
  | curl -s $OWUI/api/v1/auths/signin -H "$J" -d @- | jq -r .token; }
ADMIN=$(login [email protected] lab-admin-1234)
curl -s $OWUI/api/v1/auths/add -H "Authorization: Bearer $ADMIN" -H "$J" \
  -d '{"name":"Anna","email":"[email protected]","password":"lab-anna-1234",
       "role":"user"}' | jq -c '{name, email, role}'
curl -s $OWUI/api/v1/auths/signup -H "$J" \
  -d '{"name":"Eve","email":"[email protected]","password":"lab-eve-1234"}' \
  | jq -r .detail | cut -c1-55

Expected (tested live with open-webui serve):

{"name":"Anna","email":"[email protected]","role":"user"}
You do not have permission to access this resource. Ple
Role May
pending nothing – waits for approval (default for self-registered users)
user chat with approved models, own documents
admin everything: users, connections, settings

What does Anna see? Without BYPASS_MODEL_ACCESS_CONTROL, /api/models returns an empty list for regular users, and every chat ends with Model not found – models only become visible once the admin makes them public in Admin Settings → Models or assigns them to a group.

ANNA=$(login [email protected] lab-anna-1234)
curl -s $OWUI/api/models -H "Authorization: Bearer $ANNA" | jq -r '.data[].id'

Expected (tested live, with BYPASS_MODEL_ACCESS_CONTROL=true):

chat-small
chat-large
code
embed

That’s convenient in the lab, because LiteLLM only lets approved aliases through anyway. For the team (Day 13), you control visibility via groups from Keycloak.

Save your progress? Sign up for free

Drill 6: Persistence and the PersistentConfig trap

The pod is gone, the data stays – it lives on the PVC:

kubectl -n ki-chat exec open-webui-0 -- ls /app/backend/data
kubectl -n ki-chat delete pod open-webui-0
kubectl -n ki-chat wait pod/open-webui-0 --for=condition=Ready --timeout=300s
[ -n "$(login [email protected] lab-anna-1234)" ] && echo "Anna existiert noch"

Expected (directory tested live, pod restart in kind not tested):

cache
uploads
vector_db
webui.db
pod "open-webui-0" deleted
pod/open-webui-0 condition met
Anna existiert noch

Now the trap: change a setting via environment variable.

kubectl -n ki-chat set env sts/open-webui DEFAULT_USER_ROLE=user
kubectl -n ki-chat rollout status sts/open-webui --timeout=300s
ADMIN=$(login [email protected] lab-admin-1234)
curl -s $OWUI/api/v1/auths/admin/config -H "Authorization: Bearer $ADMIN" \
  | jq -c '{DEFAULT_USER_ROLE, ENABLE_SIGNUP}'

Expected (tested live with open-webui serve):

{"DEFAULT_USER_ROLE":"pending","ENABLE_SIGNUP":false}

Why? DEFAULT_USER_ROLE is a PersistentConfig: on the first start, the value moves into the config table, and from then on the database wins. You can change such values in the admin UI, via the API (POST /api/v1/auths/admin/config), or globally with ENABLE_PERSISTENT_CONFIG=false – then only the environment applies again. Back to the state from Git:

helm upgrade open-webui open-webui/open-webui --version 16.6.0 -n ki-chat \
  -f ~/ki-lab/code/open-webui/values-kind.yaml

Save your progress? Sign up for free

Drill 7: Ask your documents (RAG via LiteLLM)

On upload, Open WebUI splits the file, fetches embeddings via embed and stores them in the built-in vector DB on the PVC.

printf 'Bereitschaft im KI-Lab: Montag bis Freitag hat Team Plattform Dienst.%s\n' \
  ' Notfallnummer 4711.' > ~/ki-lab/tag05/notiz.txt
FILE=$(curl -s $OWUI/api/v1/files/ -H "Authorization: Bearer $ADMIN" \
  -F file=@$HOME/ki-lab/tag05/notiz.txt | jq -r .id)
until curl -s $OWUI/api/v1/files/$FILE/process/status \
  -H "Authorization: Bearer $ADMIN" | jq -e '.status == "completed"' >/dev/null
do sleep 2; done
jq -n --arg f "$FILE" '{model: "chat-small", stream: false,
  files: [{type: "file", id: $f}],
  messages: [{role: "user", content: "Wie lautet die Notfallnummer?"}]}' \
  | curl -s $OWUI/api/chat/completions -H "Authorization: Bearer $ADMIN" \
      -H "$J" -d @- | jq -r '.choices[0].message.content'

Expected (tested live, answer text will differ):

Die Notfallnummer lautet **4711** [1].

[1] refers to the source. LiteLLM’s spend log shows whether the embeddings really went through LiteLLM:

curl -s "$GW/spend/logs?summarize=false" -H "$M" \
  | jq -r '.[] | select(.metadata.user_api_key_alias == "open-webui")
           | .model_group' | sort | uniq -c

Expected (tested live, numbers will differ):

   2 chat-small
   1 embed

Warning: Without RAG_OPENAI_API_BASE_URL, Open WebUI 0.11.4 sends the embeddings to https://api.openai.com/v1 – the upload fails with 401 Unauthorized, and in the worst case documents leave the building. The variable is a PersistentConfig; a wrong first start stays stuck in the DB.

Save your progress? Sign up for free

Checkpoint

check() { if eval "$2" >/dev/null 2>&1; then echo "✔ $1"; else echo "✘ $1"; fi; }
OWUI=http://chat.ki.localtest.me
check "Pod ready" \
  'kubectl -n ki-chat wait pod/open-webui-0 --for=condition=Ready --timeout=5s'
check "PVC gebunden" \
  '[ "$(kubectl -n ki-chat get pvc open-webui -o jsonpath={.status.phase})" = Bound ]'
check "Health über Ingress" \
  'curl -sf $OWUI/health | jq -e .status'
check "Registrierung zu" \
  '[ "$(curl -s $OWUI/api/config | jq .features.enable_signup)" = false ]'
check "Rauchtest" \
  'PASS=lab-admin-1234 ~/ki-lab/code/open-webui/smoke.sh'
check "Key nicht im Klartext in den Values" \
  '! grep -q "sk-" ~/ki-lab/code/open-webui/values-kind.yaml'

Challenge

The UI may use at most 30 requests per minute and 1 (lab currency) per month – without restarting Open WebUI or swapping the key.

Hint: Limits are attached to the virtual key in LiteLLM, not to Open WebUI; /key/update changes them while running.

Solution

OWUI_KEY=$(kubectl -n ki-chat get secret owui-secrets \
  -o jsonpath='{.data.litellm-key}' | base64 -d)
curl -s $GW/key/update -H "$M" -H "$J" \
  -d "{\"key\": \"$OWUI_KEY\", \"rpm_limit\": 30, \"max_budget\": 1}" \
  | jq -c '{key_alias, rpm_limit, max_budget}'
curl -s "$GW/key/info?key=$OWUI_KEY" -H "$M" \
  | jq -c '.info | {key_alias, spend, rpm_limit, max_budget}'

Expected (live against a local proxy, spend will differ):

{"key_alias":"open-webui","rpm_limit":30,"max_budget":1.0}
{"key_alias":"open-webui","spend":0.003651,"rpm_limit":30,"max_budget":1.0}

The key stays the same, and Open WebUI notices nothing – until the limit kicks in. Then users see LiteLLM’s error message in the chat (429 or budget_exceeded).

Save your progress? Sign up for free

Enterprise Note

  • SSO instead of local accounts: OIDC against Keycloak, roles and model visibility via groups, ENABLE_LOGIN_FORM=false (Day 13).
  • Multiple replicas need Postgres instead of SQLite (databaseUrl), Redis for websockets, and an RWX volume or S3 for uploads.
  • Put vectors in a dedicated database (Qdrant, pgvector) instead of the built-in Chroma on the PVC; never switch the embedding model without re-indexing.
  • One LiteLLM key or one Open WebUI instance per department if budgets have to be billed separately.
  • Backups: back up the PVC or Postgres with Velero/CNPG – chat histories can contain personal data (GDPR, deletion policy).