Appendix B: Final Exam
Goal: Apply the book without instructions: in the RKE2 cluster, build, operate and tear down a second tenant
pruefung, plus a short knowledge section. · Duration: approx. 120 min · Prerequisite: Day 15
Rules of Play
- Every exercise lists requirements and a check block of individual commands with expected output. You pass it if your output matches. Only read the solution afterwards.
- The exercises build on each other and run in
~/k8s-lab/anhb. The namespacenotesstays untouched and keeps running the whole time. - Allowed are Appendix A,
kubectl explain,--helpand the official docs, but not Days 1–15. The solution files are incode/anhb/loesung/. - You pass with eight matching check blocks and at least 8 of 10 points in the knowledge section.
source ~/k8s-lab/setup/env.sh && source ~/k8s-lab/setup/rke2.env
mkdir -p "$LAB/anhb" && cd "$LAB/anhb"
cp -R "$LAB/tag15/chart" "$LAB/tag15/values-rke2.yaml" .
lsExpected:
chart values-rke2.yaml
values-rke2.yaml already contains your server IP (Day
15). IDs, names, IPs and times in the output will differ on your
machine.
Exercise 1: A Second Tenant
Part 1 (Days 1, 6, 8). Write namespace.yaml:
- Namespace
pruefungwith Pod Securityrestricted(enforce and warn). - LimitRange
defaultsand ResourceQuotapruefungas innotes, but at most 10 Pods. - A ServiceAccount for the chart’s Pods, without an automatically mounted token.
Check block:
kubectl get ns pruefung --show-labels
kubectl -n pruefung get resourcequota,serviceaccountExpected (abridged):
NAME STATUS AGE LABELS
pruefung Active 15s kubernetes.io/metadata.name=pruefung,pod-security.kubernetes
.io/enforce=restricted,pod-security.kubernetes.io/warn=restricted
NAME REQUEST …
resourcequota/pruefung pods: 0/10, requests.cpu: 0/2, requests.memory: 0/2Gi …
NAME AGE
serviceaccount/default 15s
serviceaccount/notes 15s
Solution
The template is k8s/namespace.yaml from Day 10, with a
new name and a smaller quota. The ServiceAccount is still called
notes, because the chart hard-codes
serviceAccountName: notes:
# namespace.yaml – Appendix B, Exercise 1: second tenant "pruefung"
apiVersion: v1
kind: Namespace
metadata:
name: pruefung
labels:
pod-security.kubernetes.io/enforce: restricted
pod-security.kubernetes.io/warn: restricted
---
apiVersion: v1
kind: LimitRange
metadata:
name: defaults
namespace: pruefung
spec:
limits:
- type: Container
defaultRequest:
cpu: 50m
memory: 64Mi
default:
memory: 128Mi
---
apiVersion: v1
kind: ResourceQuota
metadata:
name: pruefung
namespace: pruefung
spec:
hard:
pods: "10"
requests.cpu: "2"
requests.memory: 2Gi
limits.memory: 4Gi
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: notes
namespace: pruefung
automountServiceAccountToken: falsekubectl apply -f namespace.yamlExpected:
namespace/pruefung created
limitrange/defaults created
resourcequota/pruefung created
serviceaccount/notes created
Warning:
kubectl apply --dry-run=serverfails here for all objects except the namespace itself:namespaces "pruefung" not found. The dry run doesn’t really create the namespace. Chained with&&, the realapplythen never runs.
Save your progress? Sign up for free
Exercise 2: Your Own Secrets
Part 1 (Days 4, 7). In pruefung, create the Secrets that
the chart expects, but with your own lab values: API key
lab-api-key-456, DB password lab-db-pw-456. No
file in the repository, no k8s/secrets.sh.
Check block:
kubectl -n pruefung get secrets
kubectl -n pruefung get secret notes-db -o jsonpath='{.data.db-url}' | base64 -d; echoExpected:
NAME TYPE DATA AGE
notes-api Opaque 1 1s
notes-db Opaque 2 1s
postgres://notes:lab-db-pw-456@db:5432/notes
Solution
kubectl -n pruefung create secret generic notes-api \
--from-literal=api-key=lab-api-key-456
kubectl -n pruefung create secret generic notes-db \
--from-literal=password=lab-db-pw-456 \
--from-literal=db-url='postgres://notes:lab-db-pw-456@db:5432/notes'Expected:
secret/notes-api created
secret/notes-db created
The names and keys are in values.yaml under
existingSecrets and in _helpers.tpl. The host
db resolves within the namespace itself, which is why the
same URL as in notes fits.
Save your progress? Sign up for free
Exercise 3: notes for the Tenant
Parts 1 and 2 (Days 10, 13). Install the same chart as release
notes in pruefung. For that, write only one
additional values file values-pruefung.yaml with the
deviations from values-rke2.yaml: two api replicas, 512Mi
for the database, no Gateway.
Check block:
helm list -A | grep -E '^(NAME|notes)'
kubectl -n pruefung get pods
kubectl -n pruefung get pvcExpected (abridged):
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
notes notes 1 2026-… deployed notes-0.3.0 0.2.0
notes pruefung 1 2026-… deployed notes-0.3.0 0.2.0
NAME READY STATUS RESTARTS AGE
api-76d88c5977-rplx7 1/1 Running 3 (50s ago) 51s
api-76d88c5977-zjz88 1/1 Running 3 (48s ago) 51s
db-0 1/1 Running 0 51s
migrate-mc2qt 0/1 Completed 0 10s
web-746c476955-cxg9s 1/1 Running 0 51s
web-746c476955-kqn5n 1/1 Running 0 51s
worker-69897c6c4c-tdx5l 1/1 Running 0 51s
NAME STATUS VOLUME CAPACITY …
data-db-0 Bound pvc-ecf8967f-dbde-4730-8362-2b97fad5cc25 512Mi …
Solution
# values-pruefung.yaml – Appendix B, Exercise 3: only the deviations from values-rke2.yaml
api:
replicas: 2
db:
storage: 512Mi
# A second Gateway would also get NodePort 30080 – notes already has it.
gateway:
enabled: falsehelm -n pruefung install notes chart/notes -f values-rke2.yaml \
-f values-pruefung.yaml --wait --timeout 5m | grep -E '^(NAMESPACE|STATUS):'
kubectl -n pruefung get resourcequotaExpected (abridged):
NAMESPACE: pruefung
STATUS: deployed
NAME REQUEST LIMIT …
pruefung pods: 6/10, requests.cpu: 320m/2, requests.memory: 448Mi/2Gi …
Multiple -f files are merged in order; later ones win.
The release name may be the same in two namespaces. The NetworkPolicies
come along; only the Gateway and HTTPRoute are missing.
Save your progress? Sign up for free
Exercise 4: Access Without a Gateway
Part 1 (Day 3). Reach web in pruefung from the Mac
without changing a Service or Gateway. Create the note “Prüfung
bestanden” (“exam passed”) and show that notes and
pruefung have separate data.
Check block:
curl -s localhost:8081/api/notes; echo
curl -s --resolve "notes.localhost:30080:$SERVER" http://notes.localhost:30080/api/notes
echoExpected (abridged):
[{"id":"14823b5d-…","text":"Prüfung bestanden"}]
[{"id":"342a2486-…","text":"umgezogen auf RKE2"}]
Solution
kubectl -n pruefung port-forward svc/web 8081:3000 &
curl -s localhost:8081/api/notes; echo
curl -s -X POST -H 'Content-Type: application/json' \
-d '{"text":"Prüfung bestanden"}' localhost:8081/api/notes; echoExpected (abridged):
Forwarding from 127.0.0.1:8081 -> 3000
…
[]
{"id":"14823b5d-…","text":"Prüfung bestanden"}
port-forward goes through the API server and the kubelet
directly into the Pod. The NetworkPolicy web-from-gateway
doesn’t apply to it. End the tunnel afterwards with
kill %1.
Save your progress? Sign up for free
Exercise 5: Tenants Are Separated
Part 1 (Days 3, 11). Prove with two commands from a web Pod in
pruefung: the tenant’s own api is reachable, the api in
notes is not.
Check block: the two commands of the solution.
Solution
kubectl -n pruefung exec deploy/web -- wget -qO- -T 3 http://api:8080/version
kubectl -n pruefung exec deploy/web -- wget -qO- -T 3 http://api.notes:8080/versionExpected:
0.2.0
wget: download timed out
command terminated with exit code 1
The short name api means: in your own namespace.
api.notes is a valid DNS name (Day 3), but
api-from-clients in notes only admits Pods
from notes (Day 11).
Save your progress? Sign up for free
Exercise 6: Maintenance Without Downtime
Part 2 (Days 14, 15). Find the agent on which
neither of the two databases lives, and drain it. If
the drain hangs, find the cause and fix it without deleting Pods or
using --disable-eviction. Release the node again
afterwards.
Check block:
kubectl get pdb -A
kubectl get nodesExpected: no budget with ALLOWED DISRUPTIONS 0, all
nodes without SchedulingDisabled.
NAMESPACE NAME MIN AVAILABLE MAX UNAVAILABLE ALLOWED DISRUPTIONS AGE
notes api 2 N/A 1 15m
notes web 1 N/A 1 15m
pruefung api 1 N/A 1 10m
pruefung web 1 N/A 1 10m
NAME STATUS ROLES AGE VERSION
rke2-agent1 Ready <none> 109m v1.37.1+rke2r1
rke2-agent2 Ready <none> 109m v1.37.1+rke2r1
rke2-server Ready control-plane,etcd 112m v1.37.1+rke2r1
Solution
c='NS:.spec.claimRef.namespace,NODE:.spec.nodeAffinity.required'
c+='.nodeSelectorTerms[0].matchExpressions[0].values[0]'
kubectl get pv -o custom-columns="$c"
kubectl drain rke2-agent1 --ignore-daemonsets --delete-emptydir-data --timeout=30s \
2>&1 | grep -m1 'disruption budget'
kubectl get pdb -AExpected (abridged):
NS NODE
pruefung rke2-server
notes rke2-agent2
error when evicting pods/"api-59b55c94fb-nh2w7" -n "pruefung" (will retry after 5s):
Cannot evict pod as it would violate the pod's disruption budget.
NAMESPACE NAME MIN AVAILABLE MAX UNAVAILABLE ALLOWED DISRUPTIONS AGE
notes api 2 N/A 1 15m
notes web 1 N/A 1 15m
pruefung api 2 N/A 0 10m
pruefung web 1 N/A 1 10m
The trap is in Exercise 3: two api replicas, but the chart demands
minAvailable: 2. That is Round 4 from Day 15 in a new
disguise. The budget has to match the number of replicas:
helm -n pruefung upgrade notes chart/notes --reuse-values \
--set pdb.api.minAvailable=1 --wait | grep -E '^(STATUS|REVISION):'
kubectl drain rke2-agent1 --ignore-daemonsets --delete-emptydir-data | tail -1
kubectl uncordon rke2-agent1Expected:
STATUS: deployed
REVISION: 2
node/rke2-agent1 drained
node/rke2-agent1 uncordoned
If on your machine there is a database on each agent, only the server remains. It can be drained too; the static Pods of the control plane stay in place.
Save your progress? Sign up for free
Exercise 7: A Version of Its Own for the Tenant
Part 2 (Day 13). Lift the api in pruefung to 0.2.1 from
your registry (Challenge Day 13). notes stays on 0.2.0, and
the change from Exercise 6 is preserved.
Check block:
kubectl -n pruefung exec deploy/web -- wget -qO- http://api:8080/version
kubectl -n notes exec deploy/web -- wget -qO- http://api:8080/version
kubectl -n pruefung get pdb apiExpected:
0.2.1
0.2.0
NAME MIN AVAILABLE MAX UNAVAILABLE ALLOWED DISRUPTIONS AGE
api 1 N/A 1 12m
Solution
helm -n pruefung upgrade notes chart/notes --reuse-values \
--set api.image.tag=0.2.1 --wait | grep -E '^(STATUS|REVISION):'
helm -n pruefung get values notes | grep -B1 -A1 'tag:'Expected (abridged):
STATUS: deployed
REVISION: 3
repository: …:5000/notes-api
tag: 0.2.1
replicas: 2
--reuse-values keeps everything from Exercises 3 and 6,
and --set only puts the tag on top. If image 0.2.1 is
missing from your registry, build and push it as in the challenge of Day
13.
Save your progress? Sign up for free
Exercise 8: Tear Down Cleanly
Parts 1 and 2 (Days 10, 12). Remove the tenant completely: release, namespace, volume. Prove that the data on the node is gone too.
Check block:
kubectl get ns pruefung
kubectl get pv
multipass exec rke2-server -- sudo ls /opt/local-path-provisionerExpected: namespace gone, only the volume of notes left,
and no directory left on the server (on your machine possibly a
different node, see Exercise 6).
Error from server (NotFound): namespaces "pruefung" not found
NAME CAPACITY … CLAIM …
pvc-f7f33bd8-7f9f-470d-92bd-39ac2b726af9 1Gi … notes/data-db-0 …
Solution
PV=$(kubectl -n pruefung get pvc data-db-0 -o jsonpath='{.spec.volumeName}')
helm -n pruefung uninstall notes --wait
kubectl -n pruefung get pvc
kubectl delete namespace pruefung
kubectl get pv "$PV"Expected (abridged):
release "notes" uninstalled
NAME STATUS VOLUME CAPACITY …
data-db-0 Bound pvc-ecf8967f-dbde-4730-8362-2b97fad5cc25 512Mi …
namespace "pruefung" deleted
Error from server (NotFound): persistentvolumes "pvc-ecf8967f-…" not found
helm uninstall leaves the PVC in place (Day 15, Round
10). Only deleting the namespace takes it along, and local-path’s
reclaim policy Delete removes the PV and the directory. No
etcd snapshot brings this data back (Day 14).
Save your progress? Sign up for free
Knowledge Section
One point each. Answers below.
- Why does a note survive
kubectl delete pod db-0, but notkubectl delete namespace notes? - How do
readinessProbeandlivenessProbediffer in their effect? - Why does a Pod with
runAsNonRoot: trueand the imageUSER nodehang? - What does
before-hook-creationdo for the chart’s migration Job? - A node is
NotReady. When does Kubernetes move its Pods, and why so late? - Why does RKE2’s API server need
tls-sanwith the server IP? - One pull fails with
x509, another with401. What is missing in each case? - Why doesn’t a PodDisruptionBudget help when draining the node with
db-0? - What does an etcd snapshot back up, and what not?
- Why must the server move to the new Kubernetes version before the agents?
Answers
- The PVC
data-db-0survives the Pod. With the namespace it is deleted, and local-path deletes the volume (Delete). - Readiness takes the Pod out of the Service, liveness restarts the container.
- With a user name, Kubernetes can’t check whether it is root. Without
a numeric
runAsUser, the kubelet refuses to start it. - Helm deletes the old Job before it creates the new one. Jobs are immutable.
- After a good 40 s of
NotReady, the taintunreachablefollows. By default, Pods tolerate it for 300 s so that short network glitches don’t trigger moves. kubectlchecks whether the address it connects to is in the certificate. Otherwise it rejects the connection.x509: the node doesn’t know the CA.401: the credentials are missing or wrong. Both are configured inregistries.yaml.- The volume binds
db-0to exactly this node. With one replica, no budget can prevent an outage; it can only block the drain. - All Kubernetes objects (Deployments, Secrets, PVC objects), but no data in the volumes.
- A kubelet may be older than the API server, but never newer.