Preface
A Pod hangs in Pending, a Deployment does not roll out,
a node is suddenly NotReady, and during on-call duty
someone is looking at you. In moments like these, what counts is not how
many commands you know but whether you narrow down the source of the
fault systematically: Which layer is it, which tool sees it, which
message is where? This book trains exactly that – not with slides, but
in the terminal. You break every fault in yourself, see the symptom,
narrow it down, find the cause, repair it and prove with a command that
it works again.
Over 20 days you work from the outside in: scheduling, images,
crashes, kubectl debug, Services, DNS, NetworkPolicies,
Ingress and TLS, storage, RBAC and Pod Security, Kyverno and Falco, then
nodes, control plane, etcd, webhooks and stuck namespaces, rollouts and
Helm. Two days on Talos Linux show the same questions on a node without
a shell. At the end come a competition and a final exam.
The book is for platform and DevOps engineers and admins with
Kubernetes basics: you should know Pods, Deployments, Services, PVCs,
Helm and kubectl from everyday work. It is not an exam
book.
Scope. “CKA hands-on” builds clusters with kubeadm at exam pace; here there is no kubeadm but RKE2 and Talos, and instead of speed, depth counts: deadlines, mechanics, admission, webhooks and the paths of the distributions. “Talos hands-on” covers installation via ISO, upgrades and operating Talos; here Talos is only the node without a shell on which you practise troubleshooting. What other books in the series go deeper into (CKAD, CKS, Kyverno, Linkerd, Gateway API) only appears here when it disrupts operations.
How the book works
- One day, one topic. Each day has a goal, a time estimate, a short briefing and 5–8 drills. You type every command yourself and compare with “Expected”.
- One pattern for every fault: Break it → Symptom → Investigate → Cause → Fix → Verify. The verification is always a command with “Expected”, before and after.
- No helper scripts. Every step is shown as a single command, manifests as heredocs, including the checkpoint at the end of each day.
- Every day cleans up. Faults are created in the
namespace
tagNNand disappear again at the end; the checkpoint shows the starting state for the next day. - Speed Drills, Checkpoint, Challenge, Interview Note, Enterprise Note close every day. The challenge is a task without instructions, with a hint and a solution. The interview note collects questions from job interviews and on-call duty; the enterprise note says what is different in production.
- Where the command runs is stated in the first line
of each code block:
# on cp1,# on worker1or# on argo.
At the end:
- Competition (Day 20): the app
notesbroken in eight rounds – you repair it against the clock. - Appendix A is the cheat sheet: from the message to the first command, RKE2 and Talos side by side, deadlines and exit codes. Appendix B is the final exam with 16 built-in faults.
The days at a glance
| Day | Topic | Duration |
|---|---|---|
| 0 | The training ground: RKE2 with three nodes, tools | approx. 90 min |
| 1 | The method: narrow down instead of guessing, exit codes | approx. 45 min |
| 2 | k9s | approx. 45 min |
| 3 | Pods stuck in Pending | approx. 60 min |
| 4 | Images and container start | approx. 60 min |
| 5 | Crashes and restarts | approx. 75 min |
| 6 | kubectl debug | approx. 60 min |
| 7 | Services and endpoints | approx. 60 min |
| 8 | DNS and NetworkPolicies | approx. 75 min |
| 9 | Ingress, Gateway and TLS | approx. 75 min |
| 10 | Storage | approx. 75 min |
| 11 | RBAC and Pod Security | approx. 60 min |
| 12 | Faults caused by Kyverno | approx. 75 min |
| 13 | Falco: alerts and faults | approx. 60 min |
| 14 | Nodes | approx. 75 min |
| 15 | Control plane and etcd | approx. 75 min |
| 16 | Webhooks, APIServices and stuck namespaces | approx. 60 min |
| 17 | Rollouts, autoscaling, Jobs and Helm | approx. 75 min |
| 18 | Talos: nodes without a shell | approx. 75 min |
| 19 | The same faults on Talos | approx. 75 min |
| 20 | Competition: the broken cluster | approx. 120 min |
Time: Days 0 to 19 add up to 22.5 hours (1350 minutes); the competition and the final exam add two hours each – a good 26 hours in total. With one training day per evening and five evenings per week, you are through in a little over four weeks.
The training ground
| VM | Role | IP in the book | In the lab |
|---|---|---|---|
cp1 |
RKE2 server (control plane, etcd), workstation | 172.16.0.126 | 4 vCPU, 11 GiB |
worker1 |
RKE2 agent | 172.16.0.228 | 2 vCPU, 4 GiB |
worker2 |
RKE2 agent | 172.16.0.83 | 2 vCPU, 4 GiB |
argo |
QEMU host for the Talos cluster (Days 18–19) | 172.16.0.182 | 4 vCPU, 8 GiB, KVM |
- All VMs run Ubuntu 24.04 with SSH access and
sudo. Day 0 builds the RKE2 clusterdebugon top of them, Day 18 a Talos cluster of two QEMU VMs onargo. argoneeds hardware virtualization (/dev/kvm; under Proxmox or VMware, nested virtualization). Withoutargo, all days except 18 and 19 work.- Internet access from the VMs for images (Docker Hub via the mirror
mirror.gcr.io, ghcr.io, registry.k8s.io, quay.io) and tool downloads from the official release pages. - A snapshot of all VMs after Day 0 saves you an evening if things go wrong: some days deliberately intervene in nodes and the control plane.
Proxmox, VMware, a cloud or Multipass – it does not matter. On a Mac
with Apple Silicon, Multipass VMs are arm64. All images in the book
exist for amd64 and arm64, and all downloads detect the architecture
themselves (ARCH=$(dpkg --print-architecture)). The outputs
in the book come from amd64; in file names, digests and checksums you
will see arm64 instead. Where the architecture itself is
the topic (Day 4, exec format error), the drill names both
variants.
Versions
| Component | Version | From day |
|---|---|---|
| Ubuntu (VMs) | 24.04 LTS | 0 |
| RKE2 (Kubernetes) | v1.37.2+rke2r1 | 0 |
| Helm / k9s | v4.3.2 / v0.51.0 | 0 |
| local-path-provisioner | v0.0.37 | 0 |
| Kyverno (CLI and chart) | v1.20.0 (chart 3.10.0) | 0, 12 |
| Falco (chart) | 0.45.2 (9.2.2) | 13 |
| Talos Linux / talosctl | v1.14.6 | 18 |
| Kubernetes on Talos | v1.37.2 | 18 |
| metrics-server on Talos (chart) | 0.9.0 (3.14.0) | 19 |
| netshoot / whoami / busybox | v0.15 / v1.11 / 1.37 | 2 |
| notes-api / postgres | 0.7.1 / 18.7-alpine | 20 |
What you can do afterwards
- Assign every fault to a layer and, with the right tool, get within minutes to the message that names the cause.
- Tell Pending, pull errors, CrashLoops, OOM and probe failures apart
by their messages and investigate containers without a shell using
kubectl debug. - Follow the path of a request through DNS, Service, EndpointSlice, NetworkPolicy and Ingress.
- Find storage, RBAC, Pod Security and admission faults – including those caused by Kyverno or an orphaned webhook.
- Investigate nodes and the control plane: kubelet, Lease, deadlines, etcd, static Pods, certificates – on RKE2 via SSH, on Talos via the Talos API.
- Resolve stuck rollouts, blind HPAs, stuck Helm releases and stuck namespaces cleanly, without force.
What was tested
All commands and outputs come from real runs on the described training ground in November 2026, shortened where it makes sense. Every day was played through completely once more after writing. Pod names, hashes, IPs and times will differ for you; where an output depends on timing, the text says which variants occur.
Note: This book is an independent practice book. Kubernetes is a trademark of the Linux Foundation. RKE2 is a project of SUSE/Rancher, Talos Linux a project of Sidero Labs, Inc.; Kyverno and Falco are projects of the Cloud Native Computing Foundation (CNCF). The book is not affiliated with these organizations and projects and is neither reviewed nor endorsed by them.
Values such as the database password lab-db-123, the API
key lab-key-123 and the self-signed certificates are pure
lab values; you generate the RKE2 token yourself on Day 0, in the book
it is shortened. In production, such values belong in a password vault
or a KMS, never in a book.
Enjoy the training camp.