Skip to content
Hands-on IT IT Drills

Day 0: Training Ground

Goal: A kubeadm cluster v1.37.1 with three nodes, Cilium, ingress, storage, metrics and your own registry is ready for all following days. · Duration: approx. 90 min · Prerequisite: three VMs with Ubuntu 24.04

Briefing

The CKS assumes CKA knowledge: you work on ready-made kubeadm clusters, access nodes via ssh and change static Pod manifests. This chapter builds the lab at full speed – deliberately on v1.37.1, so that Day 6 can practice an upgrade. Cilium serves as the CNI: it supports NetworkPolicies (Day 2) and Pod-to-Pod encryption (Day 12). Curriculum reference: foundation for “CKS: Cluster Setup”.

  cp1 .126                  worker1 .228             worker2 .83
  control-plane, kubectl,   worker                   worker
  helm, cilium, podman
  4 vCPU / 12 GB            2 vCPU / 4 GB            2 vCPU / 4 GB
        |_________________________|________________________|
           172.16.0.0/24 · Pods 10.244.0.0/16 (Cilium, VXLAN)
  NodePorts: 30080/30443 Traefik · 30500 Registry

Replace the IPs with your own. There is no name resolution between the VMs, which is why IPs appear everywhere. Drills 1 to 3 run on all three VMs, the rest on cp1.

Drill 1: Prepare the kernel

printf 'overlay\nbr_netfilter\n' | sudo tee /etc/modules-load.d/k8s.conf
sudo modprobe overlay
sudo modprobe br_netfilter
cat <<'EOF' | sudo tee /etc/sysctl.d/k8s.conf
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF
sudo sysctl --system >/dev/null
sysctl net.ipv4.ip_forward net.bridge.bridge-nf-call-iptables
swapon --show | wc -l

Expected:

net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-iptables = 1
0

If swapon shows anything: run sudo swapoff -a and comment out the swap line in /etc/fstab.

Save your progress? Sign up for free

Drill 2: containerd and runc

Both straight from the project, each with a checksum:

cd /tmp
V=2.4.2
U=https://github.com/containerd/containerd/releases/download/v$V
ARCH=$(dpkg --print-architecture)
curl -sSLO $U/containerd-$V-linux-${ARCH}.tar.gz
curl -sSLO $U/containerd-$V-linux-${ARCH}.tar.gz.sha256sum
sha256sum -c containerd-$V-linux-${ARCH}.tar.gz.sha256sum
sudo tar Cxzf /usr/local containerd-$V-linux-${ARCH}.tar.gz
sudo mkdir -p /usr/local/lib/systemd/system
sudo curl -sSL -o /usr/local/lib/systemd/system/containerd.service \
  https://raw.githubusercontent.com/containerd/containerd/v$V/containerd.service
R=1.5.3
U=https://github.com/opencontainers/runc/releases/download/v$R
curl -sSLO $U/runc.${ARCH}
curl -sSLO $U/runc.sha256sum
grep " runc.${ARCH}$" runc.sha256sum | sha256sum -c
sudo install -m 755 runc.${ARCH} /usr/local/sbin/runc
runc --version | head -1

Expected:

containerd-2.4.2-linux-amd64.tar.gz: OK
runc.amd64: OK
runc version 1.5.3

Why? A binary without a verified checksum is an open gate in the supply chain – Day 4 checks exactly that once more, on purpose.

Generate the configuration and switch to the systemd cgroup driver (like the kubelet):

sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml >/dev/null
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
grep -n SystemdCgroup /etc/containerd/config.toml
sudo systemctl daemon-reload
sudo systemctl enable --now containerd
containerd --version

Expected:

89:            SystemdCgroup = true
containerd github.com/containerd/containerd/v2 v2.4.2 b7a25261e029...

Save your progress? Sign up for free

Drill 3: kubeadm, kubelet, kubectl in v1.37.1

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl gpg
sudo mkdir -p -m 755 /etc/apt/keyrings
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.37/deb/Release.key \
  | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
K=/etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo "deb [signed-by=$K] https://pkgs.k8s.io/core:/stable:/v1.37/deb/ /" \
  | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
apt-cache madison kubeadm | head -3
sudo apt-get install -y kubelet=1.37.1-1.1 kubeadm=1.37.1-1.1 kubectl=1.37.1-1.1 \
  cri-tools
sudo apt-mark hold kubelet kubeadm kubectl
printf 'runtime-endpoint: unix:///run/containerd/containerd.sock\n' \
  | sudo tee /etc/crictl.yaml
printf 'image-endpoint: unix:///run/containerd/containerd.sock\n' \
  | sudo tee -a /etc/crictl.yaml
kubeadm version -o short
sudo crictl version

Expected (shortened):

   kubeadm | 1.37.2-1.1 | https://pkgs.k8s.io/core:/stable:/v1.37/deb  Packages
   kubeadm | 1.37.1-1.1 | https://pkgs.k8s.io/core:/stable:/v1.37/deb  Packages
...
kubelet set on hold.
kubeadm set on hold.
kubectl set on hold.
...
v1.37.1
RuntimeName:  containerd
RuntimeVersion:  v2.4.2

Why not the latest version? 1.37.2 is already in the repo – you practice the upgrade to that version on Day 6. hold prevents apt upgrade from doing it beforehand.

Save your progress? Sign up for free

Drill 4: Initialize the control plane

On cp1. The Pod network 10.244.0.0/16 overlaps neither with the VM network nor with the Service network 10.96.0.0/12:

sudo kubeadm init --kubernetes-version=v1.37.1 \
  --apiserver-advertise-address=172.16.0.126 --pod-network-cidr=10.244.0.0/16
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
mkdir -p ~/cks/tag00
kubectl get nodes

Expected (shortened):

Your Kubernetes control-plane has initialized successfully!
...
NAME   STATUS     ROLES           AGE   VERSION
cp1    NotReady   control-plane   4s    v1.37.1

NotReady is correct: without a CNI plugin, no Pod gets an IP.

Save your progress? Sign up for free

Drill 5: Install Cilium and join the workers

First the cilium CLI, again with a checksum:

cd /tmp
C=v0.20.3
U=https://github.com/cilium/cilium-cli/releases/download/$C
ARCH=$(dpkg --print-architecture)
curl -sSLO $U/cilium-linux-${ARCH}.tar.gz
curl -sSLO $U/cilium-linux-${ARCH}.tar.gz.sha256sum
sha256sum -c cilium-linux-${ARCH}.tar.gz.sha256sum
sudo tar xzf cilium-linux-${ARCH}.tar.gz -C /usr/local/bin
cilium version --client

Expected:

cilium-linux-amd64.tar.gz: OK
cilium-cli: v0.20.3 compiled with go1.27.3 on linux/amd64
cilium image (default): v1.20.3
cilium image (stable): v1.20.5

The CLI would install 1.20.3 – that is why you set the version explicitly. ipam.mode=kubernetes lets Cilium use the Pod networks that kubeadm assigns per node:

cilium install --version 1.20.5 --set ipam.mode=kubernetes
cilium status --wait
kubeadm token create --print-join-command

Expected (shortened):

Using Cilium version 1.20.5
Auto-detected kube-proxy has been installed
...
    Cilium:             OK
    Operator:           OK
    Envoy DaemonSet:    OK
    Hubble Relay:       disabled
...
kubeadm join 172.16.0.126:6443 --token ... --discovery-token-ca-cert-hash sha256:...

Run the printed kubeadm join line with sudo on worker1 and worker2. Then back on cp1:

kubectl get nodes
kubectl get pods -n kube-system -l k8s-app=cilium -o wide
kubectl get nodes -o custom-columns=NAME:.metadata.name,CIDR:.spec.podCIDR

Expected:

NAME      STATUS   ROLES           AGE     VERSION
cp1       Ready    control-plane   4m32s   v1.37.1
worker1   Ready    <none>          2m24s   v1.37.1
worker2   Ready    <none>          2m24s   v1.37.1
NAME           READY   STATUS    RESTARTS   AGE     IP             NODE
cilium-kf854   1/1     Running   0          2m25s   172.16.0.228   worker1
cilium-nm46l   1/1     Running   0          2m25s   172.16.0.83    worker2
cilium-sz85q   1/1     Running   0          4m      172.16.0.126   cp1
NAME      CIDR
cp1       10.244.0.0/24
worker1   10.244.3.0/24
worker2   10.244.1.0/24

Cilium ends up as the Helm release cilium in kube-system, not in a namespace of its own.

SSH from cp1 to the workers

In the exam you jump from the working host to the nodes via ssh. In the lab, cp1 is that host, so it needs a key of its own. On cp1, without a passphrase (a pure lab key):

ssh-keygen -t ed25519 -N '' -f ~/.ssh/id_ed25519 -C ubuntu@cp1

Expected (shortened):

Your identification has been saved in /home/ubuntu/.ssh/id_ed25519
Your public key has been saved in /home/ubuntu/.ssh/id_ed25519.pub

ssh-copy-id does not help here: the workers do not allow password logins. That is why your own machine, which reaches all three VMs by key, distributes the public key:

ssh [email protected] cat .ssh/id_ed25519.pub \
  | ssh [email protected] 'cat >> .ssh/authorized_keys'
ssh [email protected] cat .ssh/id_ed25519.pub \
  | ssh [email protected] 'cat >> .ssh/authorized_keys'

Back on cp1. accept-new stores the host key on first contact without asking:

ssh -o StrictHostKeyChecking=accept-new [email protected] hostname
ssh -o StrictHostKeyChecking=accept-new [email protected] hostname

Expected:

Warning: Permanently added '172.16.0.228' (ED25519) to the list of known hosts.
worker1
Warning: Permanently added '172.16.0.83' (ED25519) to the list of known hosts.
worker2

From now on, all ssh ubuntu@… commands in the book run on cp1.

Save your progress? Sign up for free

Drill 6: Inspect Cilium from the inside

Every Cilium agent has its own diagnostic tool cilium-dbg:

kubectl -n kube-system exec ds/cilium -c cilium-agent -- cilium-dbg status \
  | grep -E 'KubeProxyReplacement|IPAM|Routing|Encryption'
cilium config view | grep -E '^(ipam|kube-proxy-replacement|routing-mode) '

Expected:

KubeProxyReplacement:    False
IPAM:                    IPv4: 4/254 allocated from 10.244.0.0/24,
Routing:                 Network: Tunnel [vxlan]   Host: Legacy
Encryption:              Disabled
ipam                                              kubernetes
kube-proxy-replacement                            false
routing-mode                                      tunnel

Why? Encryption: Disabled means: Pod traffic between the nodes travels across the VM network in plain text. That stays this way until Day 12 – then you measure the difference.

Save your progress? Sign up for free

Drill 7: Helm, metrics-server, local-path, Traefik

cd /tmp
ARCH=$(dpkg --print-architecture)
curl -sSLO https://get.helm.sh/helm-v4.3.2-linux-${ARCH}.tar.gz
curl -sSLO https://get.helm.sh/helm-v4.3.2-linux-${ARCH}.tar.gz.sha256sum
sha256sum -c helm-v4.3.2-linux-${ARCH}.tar.gz.sha256sum
tar xzf helm-v4.3.2-linux-${ARCH}.tar.gz
sudo install -m 755 linux-${ARCH}/helm /usr/local/bin/helm
helm version --short

Expected:

helm-v4.3.2-linux-amd64.tar.gz: OK
v4.3.2+g8346aaa

In the lab, metrics-server needs --kubelet-insecure-tls because the kubelet certificates are self-signed:

M=https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.9.1
kubectl apply -f $M/components.yaml
kubectl -n kube-system patch deployment metrics-server --type=json -p \
  '[{"op":"add","path":"/spec/template/spec/containers/0/args/-",
     "value":"--kubelet-insecure-tls"}]'
L=https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.37
kubectl apply -f $L/deploy/local-path-storage.yaml
kubectl patch storageclass local-path -p \
  '{"metadata":{"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
kubectl get storageclass

Expected (shortened):

NAME                   PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE
local-path (default)   rancher.io/local-path   Delete          WaitForFirstConsumer

Warning: --kubelet-insecure-tls is a deliberate lab shortcut: metrics-server does not verify the identity of the kubelets. In production, use signed kubelet certificates.

Ingress controller Traefik, reachable via fixed NodePorts:

helm repo add traefik https://traefik.github.io/charts
helm install traefik traefik/traefik --version 42.4.1 -n traefik --create-namespace \
  --set service.spec.type=NodePort \
  --set ports.web.nodePort=30080 --set ports.websecure.nodePort=30443
kubectl get svc -n traefik
kubectl get ingressclass
curl -s -o /dev/null -w '%{http_code}\n' http://172.16.0.126:30080/

Expected:

NAME      TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
traefik   NodePort   10.106.194.128   <none>        80:30080/TCP,443:30443/TCP   25s
NAME                CONTROLLER                      PARAMETERS   AGE
traefik (default)   traefik.io/ingress-controller   <none>       25s
404

The 404 comes from Traefik itself: it is running but does not know any Ingress yet (TLS Ingress on Day 4).

Warning: --set service.type=NodePort fails against the chart’s schema with “additional properties ‘type’ not allowed”. The type lives under service.spec.

Save your progress? Sign up for free

Drill 8: Registry in the cluster and podman

Your own images (notes-api) need a destination. A simple registry as a Deployment with NodePort 30500:

cd ~/cks/tag00
kubectl create namespace registry
kubectl create deployment registry -n registry \
  --image=docker.io/library/registry:3.1.2 --port=5000
kubectl expose deployment registry -n registry --type=NodePort --port=5000 \
  --dry-run=client -o yaml > registry-svc.yaml

In registry-svc.yaml, add the line nodePort: 30500 under ports::

  ports:
  - nodePort: 30500
    port: 5000
    protocol: TCP
    targetPort: 5000
kubectl apply -f registry-svc.yaml
curl -s http://172.16.0.126:30500/v2/_catalog

Expected:

{"repositories":[]}

The registry only speaks HTTP. So that containerd pulls from it, on all three VMs:

sudo sed -i "s|config_path = ''|config_path = '/etc/containerd/certs.d'|" \
  /etc/containerd/config.toml
sudo mkdir -p /etc/containerd/certs.d/172.16.0.126:30500
cat <<'EOF' | sudo tee /etc/containerd/certs.d/172.16.0.126:30500/hosts.toml
server = "http://172.16.0.126:30500"

[host."http://172.16.0.126:30500"]
  capabilities = ["pull", "resolve"]
EOF
sudo systemctl restart containerd
grep -n "config_path" /etc/containerd/config.toml

Expected:

31:      config_path = '/etc/containerd/certs.d'
149:    plugin_config_path = '/etc/nri/conf.d'
255:    config_path = '/etc/containerd/certs.d'

podman on cp1 is used for building and pushing. A quick round trip with busybox proves that every node pulls from the registry:

sudo apt-get install -y podman
podman version | head -2
podman pull docker.io/library/busybox:1.37
podman tag docker.io/library/busybox:1.37 172.16.0.126:30500/busybox:1.37
podman push --tls-verify=false 172.16.0.126:30500/busybox:1.37
kubectl run regtest --image=172.16.0.126:30500/busybox:1.37 --restart=Never \
  -- sh -c 'echo pull-ok'
kubectl get pod regtest -o wide
kubectl logs regtest

Expected (shortened):

Client:       Podman Engine
Version:      4.9.3
...
Writing manifest to image destination
NAME      READY   STATUS      RESTARTS   AGE   IP           NODE
regtest   0/1     Completed   0          4s    10.244.1.9   worker2
pull-ok

Clean up – the restart empties the registry, because it stores data in an emptyDir:

kubectl delete pod regtest
kubectl -n registry rollout restart deployment registry
kubectl -n registry rollout status deployment registry
curl -s http://172.16.0.126:30500/v2/_catalog

Expected:

deployment "registry" successfully rolled out
{"repositories":[]}

Warning: If the registry Pod restarts, all images are gone – then simply push them again. You build the notes-api image from the companion repo https://github.com/thomaszachmann/cks-zum-anfassen on a later day.

Save your progress? Sign up for free

Exercise: Speed Drills

  1. Show which node the registry runs on. Target time: 20 s
  2. Check whether Cilium encrypts Pod traffic. Target time: 30 s
  3. Show which packages on cp1 are held back from updates. Target time: 10 s
  4. Show the CPU load of all nodes. Target time: 10 s

Solution

kubectl get pods -n registry -o wide
kubectl -n kube-system exec ds/cilium -c cilium-agent -- cilium-dbg status \
  | grep Encryption
apt-mark showhold
kubectl top nodes

Expected (shortened):

NAME                       READY   STATUS    RESTARTS   AGE     IP             NODE
registry-f69d7f897-4vhfc   1/1     Running   0          2m27s   10.244.1.168   worker2
Encryption:              Disabled
kubeadm
kubectl
kubelet
NAME      CPU(cores)   CPU(%)   MEMORY(bytes)   MEMORY(%)
cp1       1180m        29%      1185Mi          9%
worker1   199m         9%       554Mi           14%
worker2   408m         20%      562Mi           14%

Save your progress? Sign up for free

Checkpoint

kubectl get nodes -o wide

Expected: three nodes Ready, v1.37.1, containerd://2.4.2.

kubectl get pods -A --field-selector=status.phase!=Running

Expected: No resources found

cilium status

Expected: Cilium: OK, Operator: OK, Envoy DaemonSet: OK, DaemonSet cilium with Ready: 3/3.

helm list -A

Expected: cilium (chart cilium-1.20.5) and traefik (chart traefik-42.4.1), both deployed.

curl -s http://172.16.0.126:30500/v2/_catalog

Expected: {"repositories":[]}

ssh [email protected] hostname

Expected: worker2 – without prompting for a password or host key.

Challenge

How many Pods in the cluster run with hostNetwork: true – that is, directly in the node’s network instead of in a network namespace of their own?

Hint: A JSONPath filter ?(@.spec.hostNetwork==true) saves you from looking through them all.

Solution

kubectl get pods -A -o jsonpath=\
'{range .items[?(@.spec.hostNetwork==true)]}{.metadata.name}{"\n"}{end}' | wc -l

Expected:

14

Control plane, kube-proxy and all Cilium Pods: they need the host. Any additional Pod with hostNetwork would be a warning sign – Day 9 blocks that with Pod Security Standards.

Save your progress? Sign up for free

Exam Tip

  • In the CKS you do not build a cluster, but you work just like here: ssh to nodes, crictl, /etc/kubernetes/manifests, systemctl restart kubelet.
  • Which CNI is running decides whether NetworkPolicies take effect. Check it with kubectl get pods -n kube-system instead of assuming it.
  • Documentation pages: kubernetes.io “Installing kubeadm” and docs.cilium.io “Cilium Quick Installation”.

Enterprise Note

  • Production clusters come from managed offerings (EKS, AKS, GKE) or distributions (OpenShift, RKE2, Talos) with hardened defaults, not from manual work.
  • Binaries and packages only from verified sources with a checksum or signature, mirrored into an internal repository.
  • Registries run with TLS, authentication and persistent storage (Harbor, GitLab, Artifactory) – never as an emptyDir over HTTP.
  • metrics-server with --kubelet-insecure-tls is off-limits in production; kubelet serving certificates are signed by the cluster CA.
  • Cilium is operated via Helm with versioned values (GitOps), often with Hubble for network visibility.