Day 1: Building k0s with k0sctl
Goal: A k0s v1.36.6 cluster with one controller and three workers is running, the workers carry their zone as a label, and
kubectlonk0s-ctrltalks to it. · Duration: approx. 45 min · Prerequisite: Day 0
Briefing
- k0s is a single binary with API server, scheduler, etcd, containerd, kubelet and CNI. From the operating system, it only needs the kernel.
- k0sctl distributes k0s via SSH: download, configuration, join tokens, later updates. It runs on the workstation and needs access to all hosts from there.
- A pure controller is not a node: no kubelet, no
Pods, not in
kubectl get nodes. - The cluster description
k0sctl.yamlis the single source of truth.k0sctl applyreconciles the cluster with it, as often as you like. - You deliberately build v1.36.6. You practice the update to 1.37 on Day 15.
k0s-ctrl: k0sctl ──SSH──> k0s-ctrl k0scontroller (etcd, API, scheduler)
──SSH──> k0s-w1/2/3 k0sworker (containerd, kubelet)
kubectl ──6443──> API konnectivity: worker ──8132──> controller
Drill 1: Download k0sctl
In this book, you always download binaries matching the architecture.
dpkg --print-architecture returns amd64 or
arm64 – exactly what the release files are called.
# on k0s-ctrl
mkdir -p ~/k0s-lab/tag01 && cd ~/k0s-lab/tag01
ARCH=$(dpkg --print-architecture); echo $ARCH
V=v0.33.1
curl -sSfLO https://github.com/k0sproject/k0sctl/releases/download/$V/k0sctl-linux-$ARCH
curl -sSfLO https://github.com/k0sproject/k0sctl/releases/download/$V/checksums.txt
grep "k0sctl-linux-$ARCH\$" checksums.txt | sha256sum -c
sudo install -m 755 k0sctl-linux-$ARCH /usr/local/bin/k0sctl
k0sctl versionExpected:
amd64
k0sctl-linux-amd64: OK
version: v0.33.1
commit: c4bfa95
The \$ in the grep anchors the name at the
end of the line: sha256sum -c gets exactly the one line for
your file.
Save your progress? Sign up for free
Drill 2: kubectl matching the cluster
kubectl may differ from the API server by at most one minor version. You take exactly the cluster’s version.
# on k0s-ctrl
K=v1.36.6
curl -sSfLO https://dl.k8s.io/release/$K/bin/linux/$ARCH/kubectl
curl -sSfLO https://dl.k8s.io/release/$K/bin/linux/$ARCH/kubectl.sha256
echo "$(cat kubectl.sha256) kubectl" | sha256sum -c
sudo install -m 755 kubectl /usr/local/bin/kubectl
kubectl version --clientExpected:
kubectl: OK
Client Version: v1.36.6
Kustomize Version: v5.8.1
On the controller, there would also be sudo k0s kubectl.
But that runs as root with admin rights, and only there. For everyday
work, a normal kubectl with a kubeconfig is cleaner.
Save your progress? Sign up for free
Drill 3: The skeleton from k0sctl init
# on k0s-ctrl
k0sctl init --k0s -C 1 -i ~/.ssh/id_ed25519 [email protected] \
[email protected] [email protected] [email protected] > init.yaml
wc -l init.yaml
grep role: init.yaml
grep -A1 telemetry init.yaml
grep -c version: init.yamlExpected:
88 init.yaml
role: controller
role: worker
role: worker
role: worker
telemetry:
enabled: true
0
-C 1 makes the first address the controller and all
others workers. The skeleton writes out every default value – 88 lines
you do not want to maintain. Telemetry is on, and two things are missing
entirely: a fixed version (0 matches) and
the zones.
Save your progress? Sign up for free
Drill 4: Your own k0sctl.yaml
# on k0s-ctrl
cat > k0sctl.yaml <<'EOF'
apiVersion: k0sctl.k0sproject.io/v1beta1
kind: Cluster
metadata:
name: k0s-lab
spec:
hosts:
- role: controller
ssh: {address: 172.16.0.176, user: ubuntu, keyPath: ~/.ssh/id_ed25519}
- role: worker
ssh: {address: 172.16.0.186, user: ubuntu, keyPath: ~/.ssh/id_ed25519}
installFlags: [--labels=topology.kubernetes.io/zone=a]
- role: worker
ssh: {address: 172.16.0.36, user: ubuntu, keyPath: ~/.ssh/id_ed25519}
installFlags: [--labels=topology.kubernetes.io/zone=b]
- role: worker
ssh: {address: 172.16.0.72, user: ubuntu, keyPath: ~/.ssh/id_ed25519}
installFlags: [--labels=topology.kubernetes.io/zone=a]
k0s:
version: v1.36.6+k0s.0
config:
apiVersion: k0s.k0sproject.io/v1beta1
kind: ClusterConfig
metadata:
name: k0s-lab
spec:
api:
address: 172.16.0.176
sans: [172.16.0.176]
network:
provider: kuberouter
podCIDR: 10.244.0.0/16
serviceCIDR: 10.96.0.0/12
telemetry:
enabled: false
EOFrole: controllermeans control plane without workloads.controller+workerlets the controller carry Pods as well – meant for small clusters, not wanted here.versionpins k0s. Without that line, k0sctl takes the newest one, and a laterapplyupdates the cluster without asking. The suffix+k0s.0is part of it.installFlagsend up in the worker’s systemd unit. That way, the zone label is back on the node after every start and every re-registration. Akubectl labeldoes not survive that (Day 2).topology.kubernetes.io/zoneis the standard label that the scheduler and CSI drivers know. In LINSTOR, you additionally set the zone asAux/zoneon Day 4.
Save your progress? Sign up for free
Drill 5: The cluster comes to life
# on k0s-ctrl
k0sctl apply --config k0sctl.yamlExpected (abridged):
level=info msg="==> Running phase: Gather host facts"
level=info msg="172.16.0.72:22: detected amd64 architecture"
...
level=info msg="==> Running phase: Validate hosts"
level=info msg="validating clock skew"
...
level=info msg="172.16.0.72:22: downloading k0s v1.36.6+k0s.0"
...
level=info msg="==> Running phase: Initialize the k0s cluster"
level=info msg="172.16.0.176:22: installing k0s controller"
level=info msg="172.16.0.176:22: wait for kubernetes to reach ready state"
level=info msg="==> Running phase: Install workers"
level=info msg="172.16.0.176:22: generating a join token for worker 1"
...
level=info msg="172.16.0.186:22: waiting for node to become ready"
level=info msg="==> Finished in 2m38s"
level=info msg="k0s cluster version v1.36.6+k0s.0 is now installed"
k0sctl detects each host’s architecture and downloads the matching k0s binary directly there. Before that, it checks the clock on all hosts – that is why NTP mattered on Day 0.
Save your progress? Sign up for free
Drill 6: Connect kubectl
# on k0s-ctrl
mkdir -p ~/.kube
k0sctl kubeconfig --config k0sctl.yaml > ~/.kube/config
chmod 600 ~/.kube/config
grep server ~/.kube/config
kubectl get nodes -o wideExpected (abridged):
server: https://172.16.0.176:6443
NAME STATUS ROLES AGE VERSION INTERNAL-IP ... CONTAINER-RUNTIME
k0s-w1 Ready <none> 50s v1.36.6+k0s 172.16.0.186 ... containerd://2.3.6
k0s-w2 Ready <none> 50s v1.36.6+k0s 172.16.0.36 ... containerd://2.3.6
k0s-w3 Ready <none> 50s v1.36.6+k0s 172.16.0.72 ... containerd://2.3.6
Three nodes, not four: k0s-ctrl is missing, and that is
correct. Wait a minute, then the system Pods are ready:
# on k0s-ctrl
kubectl get pods -AExpected:
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-7c5b856895-4dxld 1/1 Running 0 3m51s
kube-system coredns-7c5b856895-l4zk8 1/1 Running 0 3m51s
kube-system konnectivity-agent-4249z 1/1 Running 0 3m57s
kube-system konnectivity-agent-9rhdh 1/1 Running 0 3m57s
kube-system konnectivity-agent-hnlbn 1/1 Running 0 3m56s
kube-system kube-proxy-5brnr 1/1 Running 0 3m57s
kube-system kube-proxy-kl9cr 1/1 Running 0 3m57s
kube-system kube-proxy-vz59j 1/1 Running 0 3m57s
kube-system kube-router-b9zjl 1/1 Running 0 3m57s
kube-system kube-router-t9dhd 1/1 Running 0 3m57s
kube-system kube-router-zh4sq 1/1 Running 0 3m57s
kube-system metrics-server-5749cd655b-dpx5r 1/1 Running 0 4m1s
API server, scheduler and etcd are missing from the list. They run on
k0s-ctrl as normal processes under systemd, not as Pods.
Right after the build, you briefly see three coredns Pods: k0s rescales
coredns as soon as more than one node exists.
Tip:
kubectl -n kube-system wait --for=condition=Ready pod --allhangs here until the timeout, because it also waits for the coredns Pod that was just deleted. Usekubectl get pods -Aorrollout statusfor individual Deployments.
Save your progress? Sign up for free
Drill 7: Look on the hosts
# on k0s-ctrl
kubectl get nodes -L topology.kubernetes.io/zone
sudo k0s status
sudo k0s etcd member-list
ssh k0s-w1 sudo k0s statusExpected (abridged):
NAME STATUS ROLES AGE VERSION ZONE
k0s-w1 Ready <none> 4m46s v1.36.6+k0s a
k0s-w2 Ready <none> 4m46s v1.36.6+k0s b
k0s-w3 Ready <none> 4m46s v1.36.6+k0s a
Version: v1.36.6+k0s.0
Process ID: 2604
Role: controller
Workloads: false
SingleNode: false
{"members":{"k0s-ctrl":"https://172.16.0.176:2380"}}
Version: v1.36.6+k0s.0
Process ID: 2551
Role: worker
Workloads: true
SingleNode: false
Kube-api probing successful: true
Workloads: false confirms: Pods never run on the
controller. But a single etcd member also means: if
k0s-ctrl fails, the Pods keep running, but nobody can
change anything anymore. How three controllers solve this is covered on
Day 19.
And one more apply, without changing anything:
# on k0s-ctrl
k0sctl apply --config k0sctl.yaml 2>&1 | grep -E 'phase|running k0s|Finished'Expected (abridged):
level=info msg="==> Running phase: Gather k0s facts"
level=info msg="172.16.0.176:22: is running k0s controller version v1.36.6+k0s.0"
level=info msg="172.16.0.186:22: is running k0s worker version v1.36.6+k0s.0"
...
level=info msg="==> Running phase: Run After Apply Hooks"
level=info msg="==> Running phase: Disconnect from hosts"
level=info msg="==> Finished in 9s"
No install phase, done after nine seconds. apply is
idempotent: you can run it safely at any time.
Save your progress? Sign up for free
Exercise: Speed Drills
- Show the nodes with their zone as a separate column. Target time: 15 s
- Show the k0s version on
k0s-w2. Target time: 15 s - Show all Pods running on
k0s-w2. Target time: 30 s - Show which API address kubectl talks to. Target time: 20 s
Solution
# on k0s-ctrl
kubectl get nodes -L topology.kubernetes.io/zone
ssh k0s-w2 sudo k0s version
kubectl get pods -A -o wide --field-selector spec.nodeName=k0s-w2
kubectl config view -o jsonpath='{.clusters[0].cluster.server}{"\n"}'Expected (abridged):
...
v1.36.6+k0s.0
NAMESPACE NAME READY STATUS RESTARTS AGE ...
kube-system konnectivity-agent-9rhdh 1/1 Running 0 4m49s
kube-system kube-proxy-vz59j 1/1 Running 0 4m49s
kube-system kube-router-zh4sq 1/1 Running 0 4m49s
kube-system metrics-server-5749cd655b-dpx5r 1/1 Running 0 4m53s
https://172.16.0.176:6443
Save your progress? Sign up for free
Checkpoint
# on k0s-ctrl
kubectl get nodes -L topology.kubernetes.io/zoneExpected: three nodes Ready,
v1.36.6+k0s, zones a, b, a.
# on k0s-ctrl
kubectl get pods -A --no-headers | grep -vc RunningExpected: 0
# on k0s-ctrl
sudo k0s status | grep -E 'Version|Workloads'Expected:
Version: v1.36.6+k0s.0
Workloads: false
Challenge
Start a Deployment spread with four pause Pods that
spread evenly across the two zones, and delete it afterwards.
Hint: topologySpreadConstraints with
the zone label as topologyKey.
Solution
# on k0s-ctrl
cat > spread.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata: {name: spread, namespace: default}
spec:
replicas: 4
selector: {matchLabels: {app: spread}}
template:
metadata: {labels: {app: spread}}
spec:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector: {matchLabels: {app: spread}}
containers:
- name: pause
image: registry.k8s.io/pause:3.10
EOF
kubectl apply -f spread.yaml
kubectl rollout status deploy/spread --timeout=120s
kubectl get pods -l app=spread -o custom-columns=NODE:.spec.nodeName --no-headers \
| sort | uniq -cExpected:
deployment.apps/spread created
...
deployment "spread" successfully rolled out
1 k0s-w1
2 k0s-w2
1 k0s-w3
Two Pods in zone a (spread across k0s-w1 and
k0s-w3), two in zone b – which only has
k0s-w2. You will meet the same imbalance again on Day 12:
zone b must carry one copy of every volume.
Save your progress? Sign up for free
Cleanup
# on k0s-ctrl
kubectl delete deploy spreadInterview Note
- What distinguishes k0s from kubeadm? A single binary without dependencies, the control plane as processes under systemd instead of static Pods, controllers without a kubelet by default.
- How do you distribute k0s to several hosts? With k0sctl and
a declarative
k0sctl.yaml.applyis idempotent and also serves for expansion and updates. - Why pin the version? Without a pin, every
applypulls the newest version – an unplanned update in the middle of operations.
Enterprise Note
- Three controllers (etcd quorum) behind a load balancer or with k0s
node-local load balancing. With an external load balancer,
api.externalAddresspoints to its address. k0sctl.yamlbelongs in Git, changes go through review. The SSH key of the automation host is a critical secret.- Air gap: k0sctl can upload binaries and images from the workstation
(
uploadBinary, airgap bundle) instead of downloading them on every host. - Do not distribute the admin’s kubeconfig. Use OIDC for humans, dedicated ServiceAccounts with RBAC for automation.