Skip to content
Hands-on IT IT Drills

Day 6: Multi-tenant Kubernetes

Goal: Two tenants share the kind cluster without seeing each other, interfering with each other or breaking out. · Duration: approx. 120 min · Prerequisite: Day 0

Briefing

  • One tenant = one namespace with a fixed set of guardrails: RBAC, ResourceQuota, LimitRange, NetworkPolicy, Pod Security Admission (PSA).
  • Tenants manage their workloads themselves, but never their own guardrails.
  • Network: default deny; only DNS, the tenant’s own namespace and the gateway in llm-system (Day 7) are allowed.
  • This is soft multi-tenancy: shared kernel, shared API, shared control plane.
 tenant-acme ──┐                         ┌── tenant-globex
  (team-acme)  │   ┌── llm-system ────┐  │   (team-globex)
               └──►│ llm-gateway:8080 │◄─┘
                   │      │           │
                   │      ▼           │
   ✘ acme ↔ globex │ ollama:11434     │
                   └──────────────────┘

Drill 1: Namespaces with labels

mkdir -p ~/sre-lab/tag06 && cd ~/sre-lab/tag06
cat > namespaces.yaml <<'EOF'
apiVersion: v1
kind: Namespace
metadata:
  name: tenant-acme
  labels:
    sre-lab.io/role: tenant
    sre-lab.io/tenant: acme
---
apiVersion: v1
kind: Namespace
metadata:
  name: tenant-globex
  labels:
    sre-lab.io/role: tenant
    sre-lab.io/tenant: globex
---
apiVersion: v1
kind: Namespace
metadata:
  name: llm-system
  labels:
    sre-lab.io/role: platform
EOF
kubectl apply -f namespaces.yaml
kubectl get ns -l sre-lab.io/role -L sre-lab.io/role,sre-lab.io/tenant

Expected:

NAME            STATUS   AGE   ROLE       TENANT
llm-system      Active   1s    platform
tenant-acme     Active   1s    tenant     acme
tenant-globex   Active   1s    tenant     globex

Why labels? NetworkPolicies, quotas, Kyverno rules (Day 13) and dashboards select on labels, not on names. Kubernetes sets kubernetes.io/metadata.name automatically – that is what you use to select individual namespaces.

Save your progress? Sign up for free

Drill 2: RBAC with ClusterRole aggregation

A tenant-admin ClusterRole is assembled from building blocks; it is bound via a RoleBinding only in the respective namespace.

cat > rbac-clusterroles.yaml <<'EOF'
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: tenant-admin
aggregationRule:
  clusterRoleSelectors:
    - matchLabels:
        sre-lab.io/aggregate-to-tenant-admin: "true"
rules: []        # filled by the controller from all matching ClusterRoles
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: tenant-admin-workloads
  labels:
    sre-lab.io/aggregate-to-tenant-admin: "true"
rules:
  - apiGroups: [""]
    resources: [pods, services, configmaps, secrets, persistentvolumeclaims]
    verbs: [get, list, watch, create, update, patch, delete]
  - apiGroups: [""]
    resources: [pods/log]
    verbs: [get]
  - apiGroups: [apps]
    resources: [deployments, statefulsets, replicasets]
    verbs: [get, list, watch, create, update, patch, delete]
  - apiGroups: [batch]
    resources: [jobs, cronjobs]
    verbs: [get, list, watch, create, update, patch, delete]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: tenant-admin-readonly-guardrails
  labels:
    sre-lab.io/aggregate-to-tenant-admin: "true"
rules:
  - apiGroups: [""]
    resources: [resourcequotas, limitranges, events]
    verbs: [get, list, watch]
  - apiGroups: [networking.k8s.io]
    resources: [networkpolicies]
    verbs: [get, list, watch]
EOF
for t in acme globex; do
  kubectl create rolebinding tenant-admin -n tenant-$t \
    --clusterrole=tenant-admin --group=team-$t \
    --dry-run=client -o yaml
  echo "---"
done > rbac-bindings.yaml
kubectl apply -f rbac-clusterroles.yaml -f rbac-bindings.yaml
kubectl get clusterrole tenant-admin \
  -o jsonpath='{range .rules[*]}{.resources}{"\n"}{end}'

Expected:

["resourcequotas","limitranges","events"]
["networkpolicies"]
["pods","services","configmaps","secrets","persistentvolumeclaims"]
["pods/log"]
["deployments","statefulsets","replicasets"]
["jobs","cronjobs"]

Now you check as Anna from team-acme (impersonation, no real user needed):

canI() { printf '%-40s %s\n' "$*" \
  "$(kubectl auth can-i "$@" --as=anna --as-group=team-acme)"; }
canI create deployments -n tenant-acme
canI create deployments -n tenant-globex
canI get secrets -n tenant-acme
canI delete networkpolicies -n tenant-acme
canI create rolebindings -n tenant-acme
canI patch resourcequotas -n tenant-acme

Expected:

create deployments -n tenant-acme        yes
create deployments -n tenant-globex      no
get secrets -n tenant-acme               yes
delete networkpolicies -n tenant-acme    no
create rolebindings -n tenant-acme       no
patch resourcequotas -n tenant-acme      no

Why no RoleBindings? Whoever may create RoleBindings can grant themselves any role they hold – and with escalate/bind even more. Guardrails belong to the platform.

Tip: kubectl auth can-i --list -n tenant-acme --as=anna --as-group=team-acme shows all effective permissions – ideal for audits at the customer.

Save your progress? Sign up for free

Drill 3: ResourceQuota and LimitRange

for t in acme:1 globex:0; do
  T=${t%%:*} GPU=${t##*:}
  cat > quota-$T.yaml <<EOF
apiVersion: v1
kind: ResourceQuota
metadata:
  name: tenant-quota
  namespace: tenant-$T
spec:
  hard:
    requests.cpu: "2"
    requests.memory: 4Gi
    limits.cpu: "4"
    limits.memory: 8Gi
    pods: "20"
    persistentvolumeclaims: "5"
    requests.storage: 20Gi
    services.loadbalancers: "0"
    services.nodeports: "0"
    requests.nvidia.com/gpu: "$GPU"   # extended resource: requests. prefix only
---
apiVersion: v1
kind: LimitRange
metadata:
  name: tenant-defaults
  namespace: tenant-$T
spec:
  limits:
    - type: Container
      defaultRequest: {cpu: 100m, memory: 128Mi}
      default: {cpu: 500m, memory: 512Mi}
      max: {cpu: "2", memory: 4Gi}
EOF
done
kubectl apply -f quota-acme.yaml -f quota-globex.yaml
kubectl -n tenant-globex describe quota tenant-quota

Expected:

Resource                 Used  Hard
--------                 ----  ----
limits.cpu               0     4
...
requests.nvidia.com/gpu  0     0
services.nodeports       0     0

Three cross-checks – defaults, maximum, GPU:

kubectl -n tenant-acme run plain --image=curlimages/curl:8.22.0 \
  --restart=Never -- sleep 3600
kubectl -n tenant-acme get pod plain -o jsonpath='{.spec.containers[0].resources}'
echo
kubectl -n tenant-acme run fat --image=curlimages/curl:8.22.0 --restart=Never \
  --overrides='{"spec":{"containers":[{"name":"fat",
  "image":"curlimages/curl:8.22.0","resources":{"limits":{"memory":"16Gi"}}}]}}'
kubectl -n tenant-globex run gpu --image=curlimages/curl:8.22.0 --restart=Never \
  --overrides='{"spec":{"containers":[{"name":"gpu",
  "image":"curlimages/curl:8.22.0","resources":{"limits":{"nvidia.com/gpu":"1"}}}]}}'
kubectl -n tenant-acme delete pod plain --now

Expected (abbreviated):

pod/plain created
{"limits":{"cpu":"500m","memory":"512Mi"},"requests":{"cpu":"100m","memory":"128Mi"}}
Error from server (Forbidden): pods "fat" is forbidden: maximum memory usage
  per Container is 4Gi, but limit is 16Gi
Error from server (Forbidden): pods "gpu" is forbidden: exceeded quota:
  tenant-quota, requested: requests.nvidia.com/gpu=1, used: ...=0, limited: ...=0

Why a LimitRange? As soon as a quota limits requests.cpu, the API rejects pods without requests. The LimitRange sets defaults and thus prevents support cases (“my pod won’t start”).

Warning: For GPUs (extended resources), the quota only supports requests.nvidia.com/gpu – limits. is not allowed there. In the pod, request and limit must be equal.

Save your progress? Sign up for free

Drill 4: NetworkPolicy – default deny and exceptions

First create the test pods (already restricted-compliant, you will need that in Drill 5):

for ns in tenant-acme tenant-globex; do cat <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: client
  namespace: $ns
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 100               # curl_user in the image
    seccompProfile: {type: RuntimeDefault}
  containers:
    - name: curl
      image: curlimages/curl:8.22.0
      command: [sleep, "86400"]
      securityContext:
        allowPrivilegeEscalation: false
        capabilities: {drop: [ALL]}
---
EOF
done > testpods.yaml
cat >> testpods.yaml <<'EOF'
apiVersion: v1
kind: Pod
metadata:
  name: web
  namespace: tenant-globex
  labels: {app: web}
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 101               # nginx in the unprivileged image
    seccompProfile: {type: RuntimeDefault}
  containers:
    - name: nginx
      image: nginxinc/nginx-unprivileged:1.31-alpine
      ports: [{containerPort: 8080}]
      securityContext:
        allowPrivilegeEscalation: false
        capabilities: {drop: [ALL]}
---
apiVersion: v1
kind: Service
metadata:
  name: web
  namespace: tenant-globex
spec:
  selector: {app: web}
  ports: [{port: 80, targetPort: 8080}]
EOF
kubectl apply -f testpods.yaml
kubectl wait --for=condition=Ready pod --all -n tenant-acme --timeout=120s
kubectl wait --for=condition=Ready pod --all -n tenant-globex --timeout=120s
sleep 5   # kube-proxy needs a moment for the Service rules

You will need the probe function several times:

probe() {
  if kubectl -n "$1" exec client -- curl -s -o /dev/null -m 3 "$2" 2>/dev/null
  then echo "✔ offen     $1 -> $2"; else echo "✘ blockiert $1 -> $2"; fi
}
probe tenant-acme   http://web.tenant-globex
probe tenant-globex http://web.tenant-globex
probe tenant-acme   https://example.com

Expected (without policies everything is open):

✔ offen     tenant-acme -> http://web.tenant-globex
✔ offen     tenant-globex -> http://web.tenant-globex
✔ offen     tenant-acme -> https://example.com

Now four policies per tenant:

for ns in tenant-acme tenant-globex; do
cat > netpol-$ns.yaml <<EOF
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: $ns
spec:
  podSelector: {}
  policyTypes: [Ingress, Egress]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns
  namespace: $ns
spec:
  podSelector: {}
  policyTypes: [Egress]
  egress:
    - to:
        - namespaceSelector:
            matchLabels: {kubernetes.io/metadata.name: kube-system}
          podSelector:
            matchLabels: {k8s-app: kube-dns}
      ports:
        - {protocol: UDP, port: 53}
        - {protocol: TCP, port: 53}
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-same-namespace
  namespace: $ns
spec:
  podSelector: {}
  policyTypes: [Ingress, Egress]
  ingress:
    - from: [{podSelector: {}}]
  egress:
    - to: [{podSelector: {}}]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-egress-llm-gateway
  namespace: $ns
spec:
  podSelector: {}
  policyTypes: [Egress]
  egress:
    - to:
        - namespaceSelector:
            matchLabels: {kubernetes.io/metadata.name: llm-system}
          podSelector:
            matchLabels: {app.kubernetes.io/name: llm-gateway}
      ports:
        - {protocol: TCP, port: 8080}
EOF
done
kubectl apply -f netpol-tenant-acme.yaml -f netpol-tenant-globex.yaml
probe tenant-acme   http://web.tenant-globex
probe tenant-globex http://web.tenant-globex
probe tenant-acme   https://example.com
kubectl -n tenant-acme exec client -- nslookup web.tenant-globex.svc.cluster.local \
  | grep -A1 '^Name'

Expected:

✘ blockiert tenant-acme -> http://web.tenant-globex
✔ offen     tenant-globex -> http://web.tenant-globex
✘ blockiert tenant-acme -> https://example.com
Name:   web.tenant-globex.svc.cluster.local
Address: 10.96.243.54

Why does this work in kind? Since kind 0.24, the default CNI kindnet enforces NetworkPolicies (verified here with kind 0.33) – Calico is not needed. On clusters without a policy-capable CNI, NetworkPolicies are silently ignored; that is why you always test with a cross-check like this one.

Warning: The selectors namespaceSelector and podSelector are in the same list entry (- namespaceSelector: … podSelector: …) and therefore count as AND. An additional - in front of podSelector turns this into OR – a classic mistake that opens up entire namespaces.

Save your progress? Sign up for free

Drill 5: Pod Security Admission (restricted)

First check whether running pods already meet the level – then enforce it:

kubectl label --dry-run=server --overwrite ns tenant-acme tenant-globex \
  pod-security.kubernetes.io/enforce=restricted
kubectl label --overwrite ns tenant-acme tenant-globex \
  pod-security.kubernetes.io/enforce=restricted \
  pod-security.kubernetes.io/enforce-version=latest \
  pod-security.kubernetes.io/warn=restricted \
  pod-security.kubernetes.io/audit=restricted

Expected:

namespace/tenant-acme labeled (server dry run)
namespace/tenant-globex labeled (server dry run)
namespace/tenant-acme labeled
namespace/tenant-globex labeled

No warning in the dry run = all running pods are compliant. Now a violation:

kubectl -n tenant-acme run root-nginx --image=nginx:1.31-alpine --restart=Never
kubectl -n tenant-acme create deployment bad --image=nginx:1.31-alpine
sleep 3
kubectl -n tenant-acme get deploy bad
kubectl -n tenant-acme get events --field-selector reason=FailedCreate \
  -o custom-columns=MSG:.message | head -2 | cut -c1-80

Expected (abbreviated):

Error from server (Forbidden): pods "root-nginx" is forbidden: violates PodSecurity
  "restricted:latest": allowPrivilegeEscalation != false (...), unrestricted
  capabilities (...), runAsNonRoot != true (...), seccompProfile (...)
Warning: would violate PodSecurity "restricted:latest": ...
deployment.apps/bad created
NAME   READY   UP-TO-DATE   AVAILABLE   AGE
bad    0/1     0            0           6s
MSG
Error creating: pods "bad-6dc9c6b47d-t8cvk" is forbidden: violates PodSecurity "

Why is the Deployment created? PSA only checks pods. The Deployment is valid; afterwards the ReplicaSet controller fails silently – only warn makes the error visible during apply. That is why you always set enforce and warn.

kubectl -n tenant-acme delete deploy bad

Save your progress? Sign up for free

Drill 6: ServiceAccount tokens

kubectl -n tenant-acme exec client -- ls /var/run/secrets/kubernetes.io/serviceaccount
for t in acme globex; do
  kubectl -n tenant-$t patch sa default -p '{"automountServiceAccountToken":false}'
done
kubectl -n tenant-acme delete pod client --now
kubectl apply -f testpods.yaml
kubectl -n tenant-acme wait --for=condition=Ready pod/client --timeout=60s
kubectl -n tenant-acme exec client -- ls /var/run/secrets/kubernetes.io/serviceaccount

Expected (abbreviated):

ca.crt
namespace
token
serviceaccount/default patched
...
ls: /var/run/secrets/kubernetes.io/serviceaccount: No such file or directory
command terminated with exit code 1

Pods that never talk to the API don’t need a token – otherwise a stolen token is the first step toward escalation. For CI there is a dedicated ServiceAccount with a short-lived token:

kubectl -n tenant-acme create sa ci-deployer
kubectl -n tenant-acme create rolebinding ci-deployer \
  --clusterrole=tenant-admin --serviceaccount=tenant-acme:ci-deployer
TOKEN=$(kubectl -n tenant-acme create token ci-deployer --duration=10m)
SERVER=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')
k() { kubectl --kubeconfig=/dev/null --server="$SERVER" \
  --insecure-skip-tls-verify --token="$TOKEN" "$@"; }   # lab only!
k auth whoami | grep Username
k -n tenant-acme get pods
k -n tenant-globex get pods

Expected:

Username   system:serviceaccount:tenant-acme:ci-deployer
NAME     READY   STATUS    RESTARTS   AGE
client   1/1     Running   0          13s
Error from server (Forbidden): pods is forbidden: User
  "system:serviceaccount:tenant-acme:ci-deployer" cannot list resource "pods" in
  API group "" in the namespace "tenant-globex"

Why --kubeconfig=/dev/null? With the normal kind kubeconfig, the admin client certificate wins, and you unknowingly test as kubernetes-admin.

Save your progress? Sign up for free

Drill 7: Tenant onboarding as one YAML bundle

Everything from Drills 1–6 as a template – a new tenant is a single command:

cat > tenant-bundle.yaml <<'EOF'
# envsubst '$TENANT $GPU' < tenant-bundle.yaml | kubectl apply -f -
apiVersion: v1
kind: Namespace
metadata:
  name: tenant-${TENANT}
  labels:
    sre-lab.io/role: tenant
    sre-lab.io/tenant: ${TENANT}
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/enforce-version: latest
    pod-security.kubernetes.io/warn: restricted
    pod-security.kubernetes.io/audit: restricted
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
  namespace: tenant-${TENANT}
automountServiceAccountToken: false
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: tenant-admin
  namespace: tenant-${TENANT}
subjects:
  - kind: Group
    name: team-${TENANT}
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: tenant-admin
  apiGroup: rbac.authorization.k8s.io
---
EOF
sed -e 's/tenant-acme/tenant-${TENANT}/' -e 's/"1"   #/"${GPU}"   #/' \
  quota-acme.yaml >> tenant-bundle.yaml
echo "---" >> tenant-bundle.yaml
sed 's/tenant-acme/tenant-${TENANT}/' netpol-tenant-acme.yaml >> tenant-bundle.yaml
grep -c 'TENANT}' tenant-bundle.yaml

Expected:

11

Test the onboarding of initech:

export TENANT=initech GPU=0
envsubst '$TENANT $GPU' < tenant-bundle.yaml | kubectl apply --dry-run=client -f - \
  | wc -l | tr -d ' '
envsubst '$TENANT $GPU' < tenant-bundle.yaml | kubectl apply --server-side -f -
kubectl -n tenant-initech get netpol,quota,limitrange,rolebinding -o name

Expected:

9
namespace/tenant-initech serverside-applied
serviceaccount/default serverside-applied
rolebinding.rbac.authorization.k8s.io/tenant-admin serverside-applied
...
networkpolicy.networking.k8s.io/allow-egress-llm-gateway serverside-applied
networkpolicy.networking.k8s.io/allow-dns
...
rolebinding.rbac.authorization.k8s.io/tenant-admin

Warning: --dry-run=server fails here with namespaces "tenant-initech" not found, because the namespace is never created in the dry run. For new tenants, use --dry-run=client or kubectl diff.

Why --server-side? The ServiceAccount default is created automatically as soon as the namespace exists. Client-side apply wants to create it and occasionally loses the race with AlreadyExists; server-side apply creates or merges.

Why envsubst '$TENANT $GPU'? Without a list of variables, envsubst replaces every $… – including those that are supposed to remain in ConfigMaps or scripts. On Day 12 the bundle becomes an OpenTofu module.

kubectl delete ns tenant-initech

Save your progress? Sign up for free

Checkpoint

check() { if eval "$2" >/dev/null 2>&1; then echo "✔ $1"; else echo "✘ $1"; fi; }
AS="--as=anna --as-group=team-acme"
check "anna darf in acme deployen" \
  "kubectl auth can-i create deployments -n tenant-acme $AS"
check "anna darf nicht in globex" \
  "! kubectl auth can-i create deployments -n tenant-globex $AS"
check "anna darf NetworkPolicies nicht löschen" \
  "! kubectl auth can-i delete networkpolicies -n tenant-acme $AS"
check "GPU-Quota globex = 0" \
  '[ "$(kubectl -n tenant-globex get quota tenant-quota \
     -o jsonpath="{.spec.hard.requests\.nvidia\.com/gpu}")" = 0 ]'
check "acme -> globex blockiert" \
  '! kubectl -n tenant-acme exec client -- curl -s -m 3 http://web.tenant-globex'
check "globex intern offen" \
  'kubectl -n tenant-globex exec client -- curl -s -m 3 http://web.tenant-globex'
check "PSA restricted in acme" \
  '[ "$(kubectl get ns tenant-acme -o \
     jsonpath="{.metadata.labels.pod-security\.kubernetes\.io/enforce}")" \
     = restricted ]'
check "kein SA-Token im acme-Client" \
  '! kubectl -n tenant-acme exec client -- \
     ls /var/run/secrets/kubernetes.io/serviceaccount'

Challenge

Prometheus (namespace monitoring, Day 8) should reach the web pod in tenant-globex on port 8080 – without tenant-acme gaining anything from it. Prove both with a test pod in monitoring.

Hint: NetworkPolicies are additive – you add an ingress policy in the target namespace, you don’t change an existing one.

Solution

cat > challenge-netpol.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-scrape-from-monitoring
  namespace: tenant-globex
spec:
  podSelector:
    matchLabels: {app: web}
  policyTypes: [Ingress]
  ingress:
    - from:
        - namespaceSelector:
            matchLabels: {kubernetes.io/metadata.name: monitoring}
      ports:
        - {protocol: TCP, port: 8080}
EOF
kubectl create ns monitoring
kubectl -n monitoring run client --image=curlimages/curl:8.22.0 -- sleep 3600
kubectl -n monitoring wait --for=condition=Ready pod/client --timeout=60s
probe monitoring http://web.tenant-globex
kubectl apply -f challenge-netpol.yaml
probe monitoring  http://web.tenant-globex
probe tenant-acme http://web.tenant-globex
kubectl -n monitoring delete pod client --now

Expected:

✘ blockiert monitoring -> http://web.tenant-globex
networkpolicy.networking.k8s.io/allow-scrape-from-monitoring created
✔ offen     monitoring -> http://web.tenant-globex
✘ blockiert tenant-acme -> http://web.tenant-globex

Why didn’t it work before, even though monitoring has no policies? The ingress side in tenant-globex was closed. A connection needs egress permission at the sender and ingress permission at the receiver.

Save your progress? Sign up for free

Interview Note

  • Typical question: “How do you isolate customers on a shared inference cluster?” – Answer in layers: identity (RBAC), resources (quota/LimitRange, GPU quota), network (default deny), runtime (PSA restricted), data (separate Secrets/PVCs).
  • Punchy: “Tenants manage workloads, the platform manages guardrails – that is why tenants get no RoleBindings, NetworkPolicies or quotas.”
  • Soft vs. hard: namespaces share the kernel and control plane. It gets harder with vCluster (a separate API per tenant), dedicated node pools (taints + nodeSelector, possibly gVisor/Kata) or separate clusters per customer.
  • Bridge from your own practice: in Vault/OpenBao you built the same pattern with namespaces, policies and auth mounts per tenant – least privilege and “policies as code” carry over 1:1.
  • Be able to show the pitfall: a NetworkPolicy without a policy-capable CNI has no effect – that is why cross-checks belong in the onboarding pipeline.

Enterprise Note

  • Identity comes from OIDC (Keycloak, Entra ID); groups like team-acme come from the IdP, not from certificates.
  • Regulated customers often demand hard separation: a dedicated GPU node pool per tenant, a separate cluster or vCluster, separate storage classes and KMS keys.
  • A policy engine (Kyverno/Gatekeeper, Day 13) enforces labels and image registries and forbids hostPath – PSA alone is not enough.
  • Egress to external APIs (e.g. api.mistral.ai) via an egress gateway or a CNI with FQDN policies (Cilium, Calico Enterprise) instead of IP lists.
  • No manual onboarding: GitOps (Argo CD/Flux) or OpenTofu (Day 12) with review, audit log and automatic cross-checks.