Appendix B: Practice Exam
Goal: You solve 17 CKNE-style tasks in 120 minutes and grade yourself. · Duration: 120 min · Prerequisite: Days 0–21
Rules of Play
- Time: 120 minutes, set a timer. Each task has a
weight in percent, adding up to 100 %, distributed like the CKNE
curriculum domains (15/25/20/25/15 %). Here you pass with 66
% – modeled on the CKA. The passing score of the real CKNE is
stated in the Linux Foundation’s Candidate Handbook
(
https://training.linuxfoundation.org); check it before your exam date. - Allowed: the Kubernetes docs,
docs.cilium.io,k explainand the tools’-h/--help. Which pages the real exam permits is stated in the current exam instructions of the Linux Foundation. Not allowed: this book during the exam time. - Each task has its own namespace
pruef-NN(task 12 additionallypruef-12-db). Pass it with-nin every command. - Files go into
~/ckne/pruef. Task 6 changes CoreDNS, tasks 11 and 14 change Cilium’s Helm values: back up first. - Cluster-wide objects that you create start with
pruef-. - The solutions are at the end. Only read them after the time is up.
Preparation
All commands on cp1, contexts ost and
west as since Day 17. Namespaces and working directory:
mkdir -p ~/ckne/pruef/backup && cd ~/ckne/pruef
for i in $(seq -w 1 17); do k create ns pruef-$i; done
k create ns pruef-12-db
k --context west create ns pruef-07
A=registry.k8s.io/e2e-test-images/agnhost:2.66.1
E=registry.k8s.io/gateway-api/echo-basic:v1.5.1
C=curlimages/curl:8.17.0
N=nicolaka/netshoot:v0.16For task 2, a web server with steady customers:
k -n pruef-02 create deployment web --image=nginx:1.29
k -n pruef-02 expose deployment web --port=80
k -n pruef-02 run client --image=$C --command -- \
sh -c 'while true; do curl -s -o /dev/null web; sleep 2; done'For task 3, an agent in the host network and a configuration service:
k -n pruef-03 create deployment config --image=$A --port=8080 \
-- /agnhost netexec --http-port=8080
k -n pruef-03 expose deployment config --port=80 --target-port=8080
k -n pruef-03 create deployment node-agent --image=$C $do -- sleep infinity \
> 03-node-agent.yaml
sed -i 's/^ spec:$/ spec:\n hostNetwork: true/' 03-node-agent.yaml
k apply -f 03-node-agent.yamlFor tasks 4 and 5, a shop and a StatefulSet database without a Service:
k -n pruef-04 create deployment shop --image=$A --replicas=2 --port=8080 \
-- /agnhost netexec --http-port=8080
k apply -f - <<'EOF'
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: db
namespace: pruef-05
spec:
serviceName: db
replicas: 2
selector:
matchLabels:
app: db
template:
metadata:
labels:
app: db
spec:
containers:
- name: db
image: registry.k8s.io/e2e-test-images/agnhost:2.66.1
args: ["netexec", "--http-port=8080"]
EOF
k -n pruef-05 run client --image=$N -- sleep infinityFor tasks 6 and 7, a service for internal names and a catalog in both clusters:
k -n pruef-06 create deployment notes --image=$A --port=8080 \
-- /agnhost netexec --http-port=8080
k -n pruef-06 expose deployment notes --port=80 --target-port=8080
k -n pruef-06 run client --image=$N -- sleep infinity
for c in ost west; do
k --context $c -n pruef-07 create deployment catalog-$c --image=$A --port=8080 \
-- /agnhost netexec --http-port=8080
k --context $c -n pruef-07 expose deployment catalog-$c --name=catalog \
--port=80 --target-port=8080
k --context $c -n pruef-07 run client --image=$N -- sleep infinity
doneFor tasks 8 and 9, a GatewayClass with NodePort Services (as on Day 10) and the backends:
k apply -f - <<'EOF'
apiVersion: cilium.io/v2alpha1
kind: CiliumGatewayClassConfig
metadata:
name: nodeport
namespace: pruef-08
spec:
service:
type: NodePort
---
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: pruef-nodeport
spec:
controllerName: io.cilium/gateway-controller
parametersRef:
group: cilium.io
kind: CiliumGatewayClassConfig
name: nodeport
namespace: pruef-08
EOF
cat > echo-basic.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
name: shop-v1
spec:
selector:
matchLabels: {app: shop, version: v1}
template:
metadata:
labels: {app: shop, version: v1}
spec:
containers:
- name: echo
image: registry.k8s.io/gateway-api/echo-basic:v1.5.1
env:
- name: POD_NAME
valueFrom: {fieldRef: {fieldPath: metadata.name}}
EOF
sed -e 's/shop-v1/shop-v2/' -e 's/version: v1/version: v2/g' echo-basic.yaml \
> echo-basic-v2.yaml
k -n pruef-08 apply -f echo-basic.yaml -f echo-basic-v2.yaml
k -n pruef-08 expose deployment shop-v1 --port=8080 --target-port=3000
k -n pruef-08 expose deployment shop-v2 --port=8080 --target-port=3000
k -n pruef-09 create deployment app --image=$E
k -n pruef-09 expose deployment app --port=8080 --target-port=3000For task 9, a CA in cert-manager:
k apply -f - <<'EOF'
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: selfsigned
namespace: pruef-09
spec:
selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: pruef-ca
namespace: pruef-09
spec:
isCA: true
commonName: Pruef CA
secretName: pruef-ca
privateKey:
algorithm: ECDSA
issuerRef:
name: selfsigned
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: pruef-ca
namespace: pruef-09
spec:
ca:
secretName: pruef-ca
EOFFor task 10, a public service; for task 11, two Pods on
worker2 and a web server on router that logs
every client IP:
k -n pruef-10 create deployment public --image=$A --port=8080 \
-- /agnhost netexec --http-port=8080
k -n pruef-11 run legacy --image=$N -l app=legacy \
--overrides='{"spec":{"nodeName":"worker2"}}' -- sleep infinity
k -n pruef-11 run modern --image=$N \
--overrides='{"spec":{"nodeName":"worker2"}}' -- sleep infinity
ssh [email protected] 'nohup python3 -m http.server 8011 > pruef-11.log 2>&1 &'For tasks 12 to 15, three tiers, a fetcher, a client-server pair on two nodes and notes-api with two ServiceAccounts:
k -n pruef-12 run frontend --image=$C -l app=frontend --command -- sleep infinity
k -n pruef-12 create deployment backend --image=$A --port=8080 \
-- /agnhost netexec --http-port=8080
k -n pruef-12 expose deployment backend --port=8080
k -n pruef-12-db create deployment db --image=$A --port=5432 \
-- /agnhost netexec --http-port=5432
k -n pruef-12-db expose deployment db --port=5432
k -n pruef-13 run fetch --image=$C -l app=fetch --command -- sleep infinity
k -n pruef-14 run probe --image=$N \
--overrides='{"spec":{"nodeName":"worker2"}}' -- sleep infinity
k -n pruef-14 run web --image=$A --port=8080 \
--overrides='{"spec":{"nodeName":"worker1"}}' \
--command -- /agnhost netexec --http-port=8080
k -n pruef-15 create deployment notes --image=ghcr.io/thomaszachmann/notes-api:0.5.1 \
--port=8080 $do > 15-notes.yaml
k set env --local -f 15-notes.yaml NOTES_API_KEY=lab-key-123 -o yaml | k apply -f -
k -n pruef-15 expose deployment notes --port=80 --target-port=8080
for sa in reader writer; do
k -n pruef-15 create sa $sa
k -n pruef-15 run $sa --image=$C --command \
--overrides="{\"spec\":{\"serviceAccountName\":\"$sa\"}}" -- sleep infinity
done
k -n pruef-15 run other --image=$C --command -- sleep infinityFor task 16, four Pods that resolve names regularly:
for p in billing search inventory; do
k -n pruef-16 run $p --image=$N --command -- sh -c 'while true; do
dig +short kubernetes.default.svc.cluster.local >/dev/null; sleep 3; done'
done
k -n pruef-16 run mailer --image=$N --command -- sh -c 'while true; do
dig +short orders.pruef-16-old.svc.cluster.local >/dev/null; sleep 3; done'For task 17, an API with L7 visibility and three clients:
k -n pruef-17 create deployment api --image=$A --port=8080 \
-- /agnhost netexec --http-port=8080
k -n pruef-17 expose deployment api --port=80 --target-port=8080
k apply -f - <<'EOF'
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: api-l7
namespace: pruef-17
spec:
endpointSelector:
matchLabels:
app: api
ingress:
- fromEntities:
- cluster
toPorts:
- ports:
- port: "8080"
protocol: TCP
rules:
http:
- {}
EOF
k -n pruef-17 create deployment shop --image=$C -- sh -c \
'while true; do curl -s -o /dev/null api/hostname; sleep 1; done'
k -n pruef-17 create deployment crawler --image=$C -- sh -c \
'while true; do curl -s -o /dev/null "api/echo?msg=admin&code=404"; sleep 2; done'
k -n pruef-17 create deployment reporter --image=$C -- sh -c \
'while true; do curl -s -o /dev/null "api/echo?msg=backup&code=403"; sleep 4; done'Wait two minutes until everything is running
(k get pods -A | grep pruef | grep -v Running stays empty).
Start the clock.
Task 1: Inventory (5 %)
Write three lines to ~/ckne/pruef/01.txt: the Pod CIDR
that Cilium has assigned to the node worker2; Cilium’s
routing mode and tunnel protocol (separated by a space); the name of the
CNI configuration file on worker1.
Task 2: Capture inside the Pod (5 %)
Namespace pruef-02. In the network namespace of the
web Pod, capture the next five packets on TCP port 80 and
save them as pcap to ~/ckne/pruef/02.pcap. The image of
web stays unchanged, the Pod is not restarted.
Task 3: Agent in the host network (5 %)
Namespace pruef-03. The Deployment
node-agent runs in the host network and must reach the
Service config via the name config.pruef-03.
It stays in the host network.
Task 4: Real client IP at the NodePort (7 %)
Namespace pruef-04. Make shop (container
port 8080) reachable on NodePort 30804 (port 80). The Pod
is supposed to see the client’s real IP, and a client always ends up at
the same Pod. Check from router with
/clientip.
Task 5: Names for every replica (6 %)
Namespace pruef-05. The StatefulSet db is
running, but db-0.db.pruef-05.svc.cluster.local does not
resolve. Make sure that every replica has its own DNS name and that
db.pruef-05 returns the IPs of all replicas.
Task 6: Internal zone (6 %)
Every name <service>.intern.lab is supposed to
point cluster-wide to the Service of the same name in
pruef-06, for example notes.intern.lab. All
other names stay unchanged. Back up the CoreDNS configuration to
~/ckne/pruef/backup/coredns.yaml first.
Task 7: Catalog across two clusters (6 %)
Namespace pruef-07, Service catalog in
ost and west. Clients in ost use
the backends from west only when ost has none.
Clients in west must never end up at backends from
ost.
Task 8: Beta customers and canary (6 %)
Namespace pruef-08. Using the GatewayClass
pruef-nodeport, create the Gateway web (HTTP,
port 80) and the HTTPRoute shop for
shop.pruef.lab: requests with the header
x-beta: on always go to shop-v2, all others 80
% to shop-v1 and 20 % to shop-v2. Prove the
distribution with 50 requests.
Task 9: HTTPS only (5 %)
Namespace pruef-09. Publish app via a
Gateway secure (GatewayClass pruef-nodeport)
at https://secure.pruef.lab. The certificate (Secret
secure-tls) is issued by the Issuer pruef-ca.
HTTP on port 80 answers for this host only with a 301 to
HTTPS.
Task 10: Service IP for the router (5 %)
Namespace pruef-10. Make public reachable
from router as a LoadBalancer Service at the fixed IP
10.50.0.99 (port 80 → 8080). FRR on router is
ready (AS 65000, neighbors: all three nodes, cluster AS 65010). Only
Services from pruef-10 get IPs from your pool.
Task 11: Fixed source IP to the outside (4 %)
Namespace pruef-11. The Pod legacy runs on
worker2, but is supposed to always reach the web server on
router (172.16.0.123, port 8011) with the source IP of
worker1 (172.16.0.228); modern keeps its node
IP. Prepare Cilium for this and name the policy pruef-11.
Prove it with the log ~/pruef-11.log on
router. Afterwards you roll back the policy, the node label
and the Helm change: Egress Gateway does not get along with ClusterMesh
(task 7).
Task 12: Three tiers (7 %)
In pruef-12 and pruef-12-db, Default Deny
applies in both directions; DNS to CoreDNS stays allowed. After that,
only this applies: frontend → backend on TCP
8080, backend → db (in
pruef-12-db) on TCP 5432.
Task 13: Read-only at example.com (6 %)
Namespace pruef-13. The Pod fetch may only
send GET requests via HTTP (port 80) to
example.com to the outside – no HTTPS, no other
destinations, no other methods. Name resolution must work.
Task 14: Encrypted between the nodes (6 %)
Pod traffic between the nodes of ost is supposed to run
encrypted with WireGuard; traffic between the nodes themselves stays as
it is. Prove that every agent has the two other nodes as peers and that
a request from probe (worker2) to
web (worker1) in pruef-14 no
longer runs as VXLAN over eth0. Afterwards you roll back
the Helm change: west does not encrypt, and ClusterMesh
(task 7) is supposed to keep running.
Task 15: Permissions by ServiceAccount (6 %)
Namespace pruef-15. Pods with the ServiceAccount
reader may only do GET /notes and
GET /version at notes, Pods with
writer additionally POST /notes. All other
Pods do not reach notes at all.
Task 16: Who is asking into the void? (8 %)
Namespace pruef-16. One of the four Pods keeps querying
a name that does not exist. Find the Pod and the name with Hubble and
write both (pod name) to ~/ckne/pruef/16.txt.
You may create a policy, but it must not block any traffic.
Task 17: Who is causing the errors? (7 %)
Namespace pruef-17. Use Prometheus
(tag19-mon) and the Hubble metrics to determine which
workload gets the most 4xx responses from api. Write the
PromQL to ~/ckne/pruef/17-query.txt and the name of the
workload to ~/ckne/pruef/17.txt.
Stop the clock.
Solutions
All solutions on cp1 in the folder
~/ckne/pruef, except where ssh is shown.
Solution 1
k get ciliumnode worker2 -o jsonpath='{.spec.ipam.podCIDRs[0]}{"\n"}' > 01.txt
cilium config view | grep -E '^(routing-mode|tunnel-protocol) ' | awk '{print $2}' \
| paste -sd' ' >> 01.txt
ssh [email protected] sudo ls /etc/cni/net.d >> 01.txt
cat 01.txtExpected:
10.0.2.0/24
tunnel vxlan
05-cilium.conflist
With IPAM cluster-pool, the Pod CIDR is in the
CiliumNode, not in node.spec.podCIDR (Day 3).
You need sudo because only root may read
/etc/cni/net.d.
Solution 2
An ephemeral container shares the Pod’s network namespace, without an
image change or a restart. For that, tcpdump needs the
profile netadmin. First start the container with
sleep, wait a few seconds (pulling the image), then write
the raw data to stdout via exec:
P=$(k -n pruef-02 get pod -l app=web -o name | cut -d/ -f2)
k -n pruef-02 debug $P --profile=netadmin --image=nicolaka/netshoot:v0.16 -c sniff \
-- sleep 600
k -n pruef-02 exec $P -c sniff -- tcpdump -i eth0 -c 5 -w - tcp port 80 > 02.pcap
tcpdump -nr 02.pcap 2>/dev/null | cut -c1-72
k -n pruef-02 get pod $P -o jsonpath='{.status.containerStatuses[0].restartCount}'
echoExpected (abridged):
tcpdump: listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
5 packets captured
03:07:52.832174 IP 10.0.2.61.38152 > 10.0.1.247.80: Flags [S], seq 11322
03:07:52.832246 IP 10.0.1.247.80 > 10.0.2.61.38152: Flags [S.], seq 2888
03:07:52.833112 IP 10.0.2.61.38152 > 10.0.1.247.80: Flags [.], ack 1, wi
03:07:52.833239 IP 10.0.2.61.38152 > 10.0.1.247.80: Flags [P.], seq 1:68
03:07:52.833259 IP 10.0.1.247.80 > 10.0.2.61.38152: Flags [.], ack 68, w
0
The status lines of tcpdump go to stderr, the pcap data
to stdout – that is the only reason the file is clean.
k debug -i directly with tcpdump -w - produced
an empty file in testing, because attach only connects
after the short run. Equivalent: get the PID on the node via
crictl inspect and run
nsenter -t <PID> -n tcpdump … (Day 1).
Solution 3
k -n pruef-03 exec deploy/node-agent -- curl -sS -m3 config.pruef-03/hostname
k -n pruef-03 exec deploy/node-agent -- grep nameserver /etc/resolv.conf
k -n pruef-03 patch deploy node-agent \
-p '{"spec":{"template":{"spec":{"dnsPolicy":"ClusterFirstWithHostNet"}}}}'
k -n pruef-03 rollout status deploy node-agent
k -n pruef-03 exec deploy/node-agent -- curl -s -m3 config.pruef-03/hostname; echo
k -n pruef-03 exec deploy/node-agent -- grep nameserver /etc/resolv.confExpected (abridged):
curl: (6) Could not resolve host: config.pruef-03
command terminated with exit code 6
nameserver 172.16.0.15
nameserver 8.8.8.8
nameserver 1.1.1.1
deployment.apps/node-agent patched
deployment "node-agent" successfully rolled out
config-57f88f8956-gd2gv
nameserver 10.96.0.10
With hostNetwork: true, ClusterFirst
silently falls back to the node’s DNS. Only
ClusterFirstWithHostNet gives the Pod CoreDNS.
Solution 4
k -n pruef-04 expose deployment shop --name=shop-ext --type=NodePort --port=80 \
--target-port=8080
k -n pruef-04 patch svc shop-ext -p '{"spec":{"externalTrafficPolicy":"Local",
"sessionAffinity":"ClientIP","ports":[{"port":80,"nodePort":30804}]}}'
k -n pruef-04 get svc shop-ext
k -n pruef-04 get pods -o wide
ssh [email protected] curl -s -m3 http://172.16.0.228:30804/clientip; echo
for i in 1 2 3 4; do ssh [email protected] \
curl -s -m3 http://172.16.0.228:30804/hostname; echo; done | sort | uniq -cExpected (abridged):
service/shop-ext exposed
service/shop-ext patched
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
shop-ext NodePort 10.99.223.156 <none> 80:30804/TCP 1s
NAME READY STATUS ... NODE
shop-65c94b46cd-pbfsp 1/1 Running ... worker2
shop-65c94b46cd-tmb8q 1/1 Running ... worker1
172.16.0.123:36000
4 shop-65c94b46cd-tmb8q
externalTrafficPolicy: Local avoids the SNAT, the Pod
sees 172.16.0.123. In return, only use node IPs with a
local backend – there is none on cp1 (curl exit 7, Day 6).
With only one backend per node, the affinity only shows on a node with
two Pods; you check the setting with
k -n pruef-04 get svc shop-ext -o jsonpath='{.spec.sessionAffinity}'.
Solution 5
A StatefulSet only gets Pod names in DNS via a
headless Service with the name from
serviceName:
k -n pruef-05 create service clusterip db --clusterip=None --tcp=8080:8080
k -n pruef-05 get svc db -o jsonpath='{.spec.selector}'; echo
k -n pruef-05 get pods -o wide
k -n pruef-05 exec client -- dig +short db-0.db.pruef-05.svc.cluster.local
k -n pruef-05 exec client -- dig +short db.pruef-05.svc.cluster.local
k -n pruef-05 exec client -- curl -s -m3 db-1.db:8080/hostname; echoExpected (abridged):
service/db created
{"app":"db"}
NAME READY STATUS RESTARTS AGE IP ...
client 1/1 Running 0 3m 10.0.2.122 ...
db-0 1/1 Running 0 3m 10.0.1.148 ...
db-1 1/1 Running 0 3m 10.0.2.134 ...
10.0.1.148
10.0.1.148
10.0.2.134
db-1
k create service sets the selector
app=<name> – here it happens to match the Pods.
Otherwise use --dry-run and adjust the selector in the
YAML.
Solution 6
k -n kube-system get cm coredns -o yaml > backup/coredns.yaml
k -n kube-system get cm coredns -o jsonpath='{.data.Corefile}' > Corefile
sed -i 's/^ ready$/ ready\n rewrite name regex (.*)\\.intern\\.lab\\.$ '\
'{1}.pruef-06.svc.cluster.local. answer auto/' Corefile
sed -n 6,7p Corefile
coredns -conf Corefile 2>&1 | tail -1
k -n kube-system create configmap coredns --from-file=Corefile $do | k replace -f -Expected:
ready
rewrite name regex (.*)\.intern\.lab\.$ {1}.pruef-06.svc.cluster.local. answer auto
plugin/kubernetes: unable to load in-cluster configuration, KUBERNETES_SERVICE_HOST and
KUBERNETES_SERVICE_PORT must be defined
configmap/coredns replaced
The message from coredns -conf is the good result
(syntax OK, Day 8). answer auto writes the name in the
answer back; otherwise the client discards an answer for a name it did
not ask for. After about a minute:
k -n kube-system logs -l k8s-app=kube-dns --since=90s --prefix | grep 'Reloading c'
k -n pruef-06 exec client -- dig notes.intern.lab | grep -A1 'ANSWER SECTION'
k -n pruef-06 exec client -- curl -s -m3 notes.intern.lab/hostname; echo
k -n pruef-06 exec client -- dig +short kubernetes.default.svc.cluster.local \
example.comExpected:
[pod/coredns-7cd4959b5-87shd/coredns] [INFO] Reloading complete
[pod/coredns-7cd4959b5-nkb2d/coredns] [INFO] Reloading complete
;; ANSWER SECTION:
notes.intern.lab. 30 IN A 10.107.136.103
notes-67fbbb656f-2fjq4
10.96.0.1
172.66.147.243
104.20.23.154
Solution 7
ost uses foreign backends only in an emergency
(affinity=local) and does not hand out its own
(shared=false). west must be global so that
ost sees its backends:
k -n pruef-07 annotate svc catalog service.cilium.io/global=true \
service.cilium.io/affinity=local service.cilium.io/shared=false
k --context west -n pruef-07 annotate svc catalog service.cilium.io/global=true
for i in $(seq 8); do k -n pruef-07 exec client -- curl -s -m3 catalog/hostname
echo; done | sort | uniq -c
for i in $(seq 8); do k --context west -n pruef-07 exec client -- \
curl -s -m3 catalog/hostname; echo; done | sort | uniq -c
k -n pruef-07 scale deployment catalog-ost --replicas=0
for i in $(seq 4); do k -n pruef-07 exec client -- curl -s -m3 catalog/hostname
echo; done | sort | uniq -c
k -n pruef-07 scale deployment catalog-ost --replicas=1Expected:
service/catalog annotated
service/catalog annotated
8 catalog-ost-7f9cbb9fc8-5lwtt
8 catalog-west-5fd9689fc7-vmqrn
deployment.apps/catalog-ost scaled
4 catalog-west-5fd9689fc7-vmqrn
deployment.apps/catalog-ost scaled
Solution 8
The rule with the header match is more specific and wins; the rule
without matches catches the rest:
cat > 08-gw.yaml <<'EOF'
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: web
namespace: pruef-08
spec:
gatewayClassName: pruef-nodeport
listeners:
- name: http
protocol: HTTP
port: 80
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: shop
namespace: pruef-08
spec:
parentRefs:
- name: web
hostnames:
- shop.pruef.lab
rules:
- matches:
- headers:
- name: x-beta
value: "on"
backendRefs:
- name: shop-v2
port: 8080
- backendRefs:
- name: shop-v1
port: 8080
weight: 80
- name: shop-v2
port: 8080
weight: 20
EOF
k apply -f 08-gw.yaml
k -n pruef-08 wait --for=condition=Programmed gateway web
NP=$(k -n pruef-08 get svc cilium-gateway-web -o jsonpath='{.spec.ports[0].nodePort}')
for i in $(seq 50); do curl -s -H 'Host: shop.pruef.lab' http://172.16.0.228:$NP/ \
| jq -r .pod; done | cut -d- -f1,2 | sort | uniq -c
for i in $(seq 10); do curl -s -H 'Host: shop.pruef.lab' -H 'x-beta: on' \
http://172.16.0.228:$NP/ | jq -r .pod; done | cut -d- -f1,2 | sort | uniq -cExpected (distribution varies):
gateway.gateway.networking.k8s.io/web created
httproute.gateway.networking.k8s.io/shop created
gateway.gateway.networking.k8s.io/web condition met
39 shop-v1
11 shop-v2
10 shop-v2
"on" is in quotes – unquoted, YAML turns it into the
boolean true, and the API server rejects the field.
Solution 9
cat > 09.yaml <<'EOF'
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: secure
namespace: pruef-09
spec:
secretName: secure-tls
dnsNames:
- secure.pruef.lab
issuerRef:
name: pruef-ca
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: secure
namespace: pruef-09
spec:
gatewayClassName: pruef-nodeport
listeners:
- name: http
protocol: HTTP
port: 80
hostname: secure.pruef.lab
- name: https
protocol: HTTPS
port: 443
hostname: secure.pruef.lab
tls:
mode: Terminate
certificateRefs:
- name: secure-tls
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: redirect
namespace: pruef-09
spec:
parentRefs:
- name: secure
sectionName: http
rules:
- filters:
- type: RequestRedirect
requestRedirect:
scheme: https
statusCode: 301
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: app
namespace: pruef-09
spec:
parentRefs:
- name: secure
sectionName: https
rules:
- backendRefs:
- name: app
port: 8080
EOF
k apply -f 09.yaml
k -n pruef-09 wait --for=condition=Ready certificate secure
k -n pruef-09 wait --for=condition=Programmed gateway secure
S=cilium-gateway-secure
NPH=$(k -n pruef-09 get svc $S -o jsonpath='{.spec.ports[?(@.port==80)].nodePort}')
NPS=$(k -n pruef-09 get svc $S -o jsonpath='{.spec.ports[?(@.port==443)].nodePort}')
k -n pruef-09 get secret secure-tls -o jsonpath='{.data.ca\.crt}' | base64 -d \
> 09-ca.crt
curl -sI -H 'Host: secure.pruef.lab' http://172.16.0.228:$NPH/x \
| grep -iE '^HTTP|^location'
curl -s --cacert 09-ca.crt --resolve secure.pruef.lab:$NPS:172.16.0.228 \
https://secure.pruef.lab:$NPS/ -o /dev/null -w '%{http_code}\n'Expected (abridged):
certificate.cert-manager.io/secure condition met
gateway.gateway.networking.k8s.io/secure condition met
HTTP/1.1 301 Moved Permanently
location: https://secure.pruef.lab:443/x
200
The redirect points to port 443 – in the lab, HTTPS sits on a NodePort; with a load balancer in front of the Gateway, the address is correct.
Solution 10
k -n pruef-10 expose deployment public --type=LoadBalancer --port=80 \
--target-port=8080 --labels=announce=pruef-10
k -n pruef-10 annotate svc public lbipam.cilium.io/ips=10.50.0.99
cat > 10-bgp.yaml <<'EOF'
apiVersion: cilium.io/v2
kind: CiliumLoadBalancerIPPool
metadata:
name: pruef-10
spec:
blocks:
- cidr: 10.50.0.0/24
serviceSelector:
matchLabels:
io.kubernetes.service.namespace: pruef-10
---
apiVersion: cilium.io/v2
kind: CiliumBGPClusterConfig
metadata:
name: pruef-10
spec:
nodeSelector:
matchLabels:
kubernetes.io/os: linux
bgpInstances:
- name: ost
localASN: 65010
peers:
- name: router
peerASN: 65000
peerAddress: 172.16.0.123
peerConfigRef:
name: pruef-10
---
apiVersion: cilium.io/v2
kind: CiliumBGPPeerConfig
metadata:
name: pruef-10
spec:
families:
- afi: ipv4
safi: unicast
advertisements:
matchLabels:
advertise: pruef-10
---
apiVersion: cilium.io/v2
kind: CiliumBGPAdvertisement
metadata:
name: pruef-10
labels:
advertise: pruef-10
spec:
advertisements:
- advertisementType: Service
service:
addresses:
- LoadBalancerIP
selector:
matchLabels:
announce: pruef-10
EOF
k apply -f 10-bgp.yaml
k -n pruef-10 get svc public
cilium bgp routes advertised ipv4 unicast --node worker1
ssh [email protected] curl -s -m3 http://10.50.0.99/hostname; echoExpected (abridged, after about 20 s):
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
public LoadBalancer 10.101.75.48 10.50.0.99 80:30208/TCP 21s
Node VRouter Peer Prefix NextHop Age Attrs
worker1 65010 172.16.0.123 10.50.0.99/32 172.16.0.228 21s [{Origin: i} ...
public-5d8587dd9-4g27g
Four objects, one chain: ClusterConfig → PeerConfig by name
(peerConfigRef), PeerConfig → Advertisement by label
(advertise), Advertisement → Service by label
(announce); the pool selects by namespace. The fixed IP is
set by the annotation lbipam.cilium.io/ips (Day 15).
Solution 11
Egress Gateway needs two Helm keys (Day 16). Back up the values first:
helm get values cilium -n kube-system -o yaml > backup/cilium-values.yaml
helm upgrade cilium cilium/cilium -n kube-system --version 1.20.5 \
--reuse-values --set egressGateway.enabled=true --set bpf.masquerade=true
k -n kube-system rollout restart ds/cilium deploy/cilium-operator
k -n kube-system rollout status ds/cilium
cilium config view | grep -E 'enable-egress-gateway|enable-bpf-masquerade'
k label node worker1 pruef-egress=true
cat > 11-egw.yaml <<'EOF'
apiVersion: cilium.io/v2
kind: CiliumEgressGatewayPolicy
metadata:
name: pruef-11
spec:
selectors:
- podSelector:
matchLabels:
app: legacy
io.kubernetes.pod.namespace: pruef-11
destinationCIDRs:
- 172.16.0.123/32
egressGateway:
nodeSelector:
matchLabels:
pruef-egress: "true"
EOF
k apply -f 11-egw.yaml
for p in legacy modern; do k -n pruef-11 exec $p -- \
curl -s -m3 -o /dev/null -w '%{http_code}\n' http://172.16.0.123:8011/; done
ssh [email protected] tail -2 pruef-11.log
CP=$(k -n kube-system get pod -l k8s-app=cilium \
--field-selector spec.nodeName=worker1 -o name)
k -n kube-system exec $CP -c cilium-agent -- cilium-dbg bpf egress listExpected (abridged):
daemon set "cilium" successfully rolled out
enable-bpf-masquerade true
enable-egress-gateway true
node/worker1 labeled
ciliumegressgatewaypolicy.cilium.io/pruef-11 created
200
200
172.16.0.228 - - [19/Nov/2026 03:47:24] "GET / HTTP/1.1" 200 -
172.16.0.83 - - [19/Nov/2026 03:47:24] "GET / HTTP/1.1" 200 -
Source IP Destination CIDR Egress IP Gateway IP Egress Ifindex
10.0.2.58 172.16.0.123/32 172.16.0.228 172.16.0.228 2
legacy arrives as worker1, modern still as
worker2. The namespace is in the podSelector as the label
io.kubernetes.pod.namespace, because the CEGP applies
cluster-wide. Now roll back as on Day 16, exactly to the backup:
k delete cegp pruef-11
k label node worker1 pruef-egress-
helm upgrade cilium cilium/cilium -n kube-system --version 1.20.5 \
-f backup/cilium-values.yaml
diff backup/cilium-values.yaml <(helm get values cilium -n kube-system -o yaml) \
&& echo GLEICH
k -n kube-system rollout restart ds/cilium deploy/cilium-operator
k -n kube-system rollout status ds/cilium
k -n kube-system exec ds/cilium -c cilium-agent -- cilium-dbg status \
| grep MasqueradingExpected (abridged):
ciliumegressgatewaypolicy.cilium.io "pruef-11" deleted
node/worker1 unlabeled
GLEICH
daemon set "cilium" successfully rolled out
Masquerading: IPTables [IPv4: Enabled, IPv6: Disabled]
According to the docs, Egress Gateway and ClusterMesh are not compatible. In the lab, task 7 kept working with Egress Gateway alone, but the combination is not supported – that is why the rollback is part of the task.
Solution 12
Default Deny with a DNS exception in one policy per namespace, then every connection twice – egress at the source, ingress at the destination:
cat > 12-netpol.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
namespace: pruef-12
spec:
podSelector: {}
policyTypes: [Ingress, Egress]
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
k8s-app: kube-dns
ports:
- {protocol: UDP, port: 53}
- {protocol: TCP, port: 53}
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: frontend-to-backend
namespace: pruef-12
spec:
podSelector:
matchLabels:
app: frontend
policyTypes: [Egress]
egress:
- to:
- podSelector:
matchLabels:
app: backend
ports:
- port: 8080
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: backend-from-frontend
namespace: pruef-12
spec:
podSelector:
matchLabels:
app: backend
policyTypes: [Ingress]
ingress:
- from:
- podSelector:
matchLabels:
app: frontend
ports:
- port: 8080
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: backend-to-db
namespace: pruef-12
spec:
podSelector:
matchLabels:
app: backend
policyTypes: [Egress]
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: pruef-12-db
podSelector:
matchLabels:
app: db
ports:
- port: 5432
EOF
cat > 12-db.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
namespace: pruef-12-db
spec:
podSelector: {}
policyTypes: [Ingress, Egress]
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
k8s-app: kube-dns
ports:
- {protocol: UDP, port: 53}
- {protocol: TCP, port: 53}
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: db-from-backend
namespace: pruef-12-db
spec:
podSelector:
matchLabels:
app: db
policyTypes: [Ingress]
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: pruef-12
podSelector:
matchLabels:
app: backend
ports:
- port: 5432
EOF
k apply -f 12-netpol.yaml -f 12-db.yaml
k -n pruef-12 exec frontend -- curl -s -m3 backend:8080/hostname; echo
k -n pruef-12 exec deploy/backend -- /agnhost connect db.pruef-12-db:5432 \
--timeout=3s; echo "rc=$?"
k -n pruef-12 exec frontend -- curl -s -m3 db.pruef-12-db:5432/hostname; echo " rc=$?"
k -n pruef-12 run fremd --image=curlimages/curl:8.17.0 --rm -i --restart=Never \
--command -- curl -s -m3 backend:8080/hostname; echo " rc=$?"Expected (abridged):
networkpolicy.networking.k8s.io/default-deny created
networkpolicy.networking.k8s.io/frontend-to-backend created
networkpolicy.networking.k8s.io/backend-from-frontend created
networkpolicy.networking.k8s.io/backend-to-db created
networkpolicy.networking.k8s.io/default-deny created
networkpolicy.networking.k8s.io/db-from-backend created
backend-75c579f4-q7bn4
rc=0
command terminated with exit code 28
rc=28
pod pruef-12/fremd terminated (Error)
rc=28
agnhost connect only checks the TCP handshake – exit 0
means: backend reaches db. Across namespaces,
namespaceSelector and podSelector belong in
one list item (AND); otherwise it becomes an OR (Day
12).
Solution 13
cat > 13-cnp.yaml <<'EOF'
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: fetch-egress
namespace: pruef-13
spec:
endpointSelector:
matchLabels:
app: fetch
egress:
- toEndpoints:
- matchLabels:
k8s:io.kubernetes.pod.namespace: kube-system
k8s:k8s-app: kube-dns
toPorts:
- ports:
- port: "53"
protocol: ANY
rules:
dns:
- matchPattern: "*"
- toFQDNs:
- matchName: example.com
toPorts:
- ports:
- port: "80"
protocol: TCP
rules:
http:
- method: GET
EOF
k apply -f 13-cnp.yaml
F="kubectl -n pruef-13 exec fetch -- curl -s -m5 -o /dev/null -w %{http_code}\n"
$F http://example.com/
$F -X POST http://example.com/
$F https://example.com/
$F http://kubernetes.io/
hubble observe -P --port-forward-port 0 -n pruef-13 --protocol http --last 2Expected (abridged):
ciliumnetworkpolicy.cilium.io/fetch-egress created
200
403
000
command terminated with exit code 28
000
command terminated with exit code 28
pruef-13/fetch:57426 (ID:88003) -> 104.20.23.154:80 (ID:16777217) http-request
DROPPED (HTTP/1.1 POST http://example.com/)
pruef-13/fetch:57426 (ID:88003) <- 104.20.23.154:80 (ID:16777217) http-response
FORWARDED (HTTP/1.1 403 0ms (POST http://example.com/))
Trap: a DNS rule with only matchName: example.com makes
curl hang. The resolver first queries the search list
(example.com.pruef-13.svc.cluster.local …), the DNS proxy
answers that with REFUSED, and musl gives up. The names are
restricted by toFQDNs, not by the DNS rule.
Solution 14
WireGuard is a Helm value with an agent restart (Day 13);
nodeEncryption stays off, because only Pod traffic is to be
encrypted. You backed up the values in task 11; otherwise run
helm get values … > backup/cilium-values.yaml now:
helm upgrade cilium cilium/cilium -n kube-system --version 1.20.5 --reuse-values \
--set encryption.enabled=true --set encryption.type=wireguard
k -n kube-system rollout restart ds/cilium
k -n kube-system rollout status ds/cilium
cilium config view | grep enable-wireguard
for n in cp1 worker1 worker2; do
CP=$(k -n kube-system get pod -l k8s-app=cilium \
--field-selector spec.nodeName=$n -o name)
echo "$n $(k -n kube-system exec $CP -c cilium-agent -- cilium-dbg encrypt status \
| grep peers)"
doneExpected (abridged):
daemon set "cilium" successfully rolled out
enable-wireguard true
cp1 Number of peers: 2
worker1 Number of peers: 2
worker2 Number of peers: 2
Proof on the wire: capture on worker1 in a second
terminal, then call web from probe:
ssh [email protected] "sudo timeout 15 tcpdump -nn -i eth0 -c 4 \
'host 172.16.0.83 and (udp port 8472 or udp port 51871)'"WEB=$(k -n pruef-14 get pod web -o jsonpath='{.status.podIP}')
k -n pruef-14 exec probe -- curl -s -m3 $WEB:8080/hostname; echoExpected (abridged):
web
03:54:43.814553 IP 172.16.0.228.51871 > 172.16.0.83.51871: UDP, length 144
03:54:43.814843 IP 172.16.0.228.51871 > 172.16.0.83.51871: UDP, length 144
03:54:43.815974 IP 172.16.0.83.51871 > 172.16.0.228.51871: UDP, length 144
03:54:43.815975 IP 172.16.0.83.51871 > 172.16.0.228.51871: UDP, length 144
Between the workers only UDP 51871 (WireGuard) runs now, no VXLAN on
UDP 8472 – the VXLAN packet sits encrypted inside the tunnel. Without
rollout restart, it stays at
Encryption: Disabled.
When only ost encrypted, no Pod from ost
reached a Pod in west any more in the lab (task 7). In a
ClusterMesh, all clusters encrypt with the same method – so here, back
to the backed-up values:
helm upgrade cilium cilium/cilium -n kube-system --version 1.20.5 \
-f backup/cilium-values.yaml
k -n kube-system rollout restart ds/cilium
k -n kube-system rollout status ds/cilium
k -n kube-system exec ds/cilium -c cilium-agent -- cilium-dbg status | grep EncryptionExpected (abridged):
daemon set "cilium" successfully rolled out
Encryption: Disabled
Solution 15
Every Pod carries its ServiceAccount as the label
io.cilium.k8s.policy.serviceaccount in its identity (Day
14):
cat > 15-cnp.yaml <<'EOF'
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: notes-by-sa
namespace: pruef-15
spec:
endpointSelector:
matchLabels:
app: notes
ingress:
- fromEndpoints:
- matchLabels:
io.cilium.k8s.policy.serviceaccount: reader
toPorts:
- ports:
- port: "8080"
protocol: TCP
rules:
http:
- method: GET
path: "/(notes|version)"
- fromEndpoints:
- matchLabels:
io.cilium.k8s.policy.serviceaccount: writer
toPorts:
- ports:
- port: "8080"
protocol: TCP
rules:
http:
- method: GET
path: "/(notes|version)"
- method: POST
path: "/notes"
EOF
k apply -f 15-cnp.yaml
T='-s -m3 -o /dev/null -w %{http_code}\n'
W='-X POST -H X-API-Key:lab-key-123 -d {"text":"x"}'
for p in reader writer other; do
echo "$p: GET $(k -n pruef-15 exec $p -- curl $T notes/notes)" \
"POST $(k -n pruef-15 exec $p -- curl $T $W notes/notes)"
doneExpected (abridged):
ciliumnetworkpolicy.cilium.io/notes-by-sa created
reader: GET 200 POST 403
writer: GET 200 POST 201
command terminated with exit code 28
command terminated with exit code 28
other: GET 000 POST 000
Solution 16
Without a DNS rule, Hubble only sees UDP packets to port 53, no names:
hubble observe -P --port-forward-port 0 -n pruef-16 --protocol dns --last 2
hubble observe -P --port-forward-port 0 -n pruef-16 --to-port 53 --last 1Expected (abridged; the first command returns nothing):
pruef-16/mailer:39303 (ID:68289) -> kube-system/coredns-7cd4959b5-nkb2d:53
(ID:70078) to-endpoint FORWARDED (UDP)
A CNP with a DNS rule switches on the DNS proxy. Because an egress
rule means Default Deny for the selected Pods,
toEntities: all allows the rest:
cat > 16-dns-sicht.yaml <<'EOF'
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: dns-sicht
namespace: pruef-16
spec:
endpointSelector: {}
egress:
- toEndpoints:
- matchLabels:
k8s:io.kubernetes.pod.namespace: kube-system
k8s:k8s-app: kube-dns
toPorts:
- ports:
- port: "53"
protocol: ANY
rules:
dns:
- matchPattern: "*"
- toEntities:
- all
EOF
k apply -f 16-dns-sicht.yaml
hubble observe -P --port-forward-port 0 -n pruef-16 --protocol dns --last 20 -o json \
| jq -r 'select(.flow.l7.dns.rcode == 3)
| .flow.destination.pod_name + " " + .flow.l7.dns.query' | sort | uniq -c
echo "mailer orders.pruef-16-old.svc.cluster.local" > 16.txt
hubble observe -P --port-forward-port 0 --pod pruef-16/mailer --protocol dns --last 1Expected (abridged):
ciliumnetworkpolicy.cilium.io/dns-sicht created
8 mailer orders.pruef-16-old.svc.cluster.local.
pruef-16/mailer:46729 (ID:68289) <- kube-system/coredns-7cd4959b5-nkb2d:53
(ID:70078) dns-response proxy FORWARDED (DNS Answer RCode: Non-Existent Domain
TTL: 4294967295 (Proxy orders.pruef-16-old.svc.cluster.local. A))
Rcode 3 is NXDOMAIN. The answer comes from CoreDNS, which is why the
querying Pod shows up there as destination.
Solution 17
The function pq from Day 19 queries Prometheus inside
the Pod:
pq() {
k -n tag19-mon exec prometheus-kps-kube-prometheus-stack-prometheus-0 \
-c prometheus -- promtool query instant http://localhost:9090 "$1"
}
pq 'sum by (source_workload, status) (rate(hubble_http_requests_total{
destination_namespace="pruef-17"}[2m]))'
Q='topk(1, sum by (source_workload) (rate(hubble_http_requests_total{
destination_namespace="pruef-17", status=~"4.."}[2m])))'
echo "$Q" > 17-query.txt
pq "$Q"
echo crawler > 17.txtExpected (abridged, values vary):
{source_workload="shop", status="200"} => 0.9545454545454545 @[1795057428.741]
{source_workload="crawler", status="404"} => 0.45579386363636365 @[1795057428.741]
{source_workload="reporter", status="403"} => 0.22175625 @[1795057428.741]
{source_workload="crawler"} => 0.45847999999999994 @[1795057429.41]
The label source_workload only exists because the Hubble
metric httpV2 was switched on in Day 19 with
labelsContext=…source_workload…. Without the L7 policy
(api-l7), there would be no HTTP metrics at all.
Scoring
Add up the weights of the tasks that are completely and verifiably
solved – there are no partial points here. Whoever bypasses a
requirement (such as allowedRoutes: All, deleting policies,
method: ".*") gets zero for the task.
| Result | Assessment |
|---|---|
| 85–100 % | exam-ready |
| 66–84 % | passed, but narrowly: repeat your weak spots |
| below 66 % | repeat the days for the tasks you missed and start again |
| Tasks | Curriculum domain | Days |
|---|---|---|
| 1–3 | Core Infrastructure and CNI (15 %) | 1–5 |
| 4–7 | Service Networking and DNS (25 %) | 6–9, 17 |
| 8–11 | Advanced Traffic Management (20 %) | 10, 11, 15, 16 |
| 12–15 | Network Security and Policy (25 %) | 12–14 |
| 16, 17 | Observability (15 %) | 19, 20 |
Not covered: ReferenceGrant (Day 10, Day 21 round 6), LLM traffic (Day 18), mutual auth (Day 14) and Multus (Day 5) – go through these days once more before the exam.
Clean up – CoreDNS back, stop the web server on router,
cluster-wide objects and namespaces gone (tasks 11 and 14 have already
rolled back the Helm values):
grep -v 'resourceVersion:' backup/coredns.yaml | k replace -f -
ssh [email protected] 'pkill -f "http.server 8011"; rm -f pruef-11.log'
k delete -f 10-bgp.yaml
k delete gatewayclass pruef-nodeport
k --context west delete ns pruef-07 --wait=false
for i in $(seq -w 1 17); do k delete ns pruef-$i --wait=false; done
k delete ns pruef-12-db --wait=falseCheckpoint
k get ns | grep -c '^pruef-'; k --context west get ns | grep -c '^pruef-'Expected (after about a minute): 0 and
0.
diff <(k -n kube-system get cm coredns -o jsonpath='{.data}') \
<(k create -f ~/ckne/pruef/backup/coredns.yaml $do -o jsonpath='{.data}') \
&& echo IDENTISCHExpected: IDENTISCH (both CoreDNS Pods
reload after about a minute).
k get cegp,ippools,ciliumbgpclusterconfigs,ciliumbgppeerconfigs,ciliumbgpadvertisements
k get gatewayclassExpected: No resources found; as
GatewayClass only cilium.
diff backup/cilium-values.yaml <(helm get values cilium -n kube-system -o yaml) \
&& echo GLEICH
k -n kube-system exec ds/cilium -c cilium-agent -- cilium-dbg status \
| grep -E 'Masquerading|Encryption'Expected: GLEICH,
Masquerading: IPTables … and
Encryption: Disabled.
ssh [email protected] "ip route | grep -c proto.bgp"Expected: 0
cilium status | grep -E 'Cilium:|Operator:|Hubble Relay:|ClusterMesh:'
k get pods -A | grep -vE 'Running|Completed'Expected: OK four times, only the
header line.
Good luck in the exam.