Day 11: Sandboxing with RuntimeClass and gVisor
Goal: You install gVisor on the workers, register
runscas a runtime handler in containerd and send Pods into the sandbox via RuntimeClass. · Duration: approx. 60 min · Prerequisite: Day 10
Briefing
- CKS: Minimize Microservice Vulnerabilities – “Understand and implement isolation techniques (multi-tenancy, sandboxed containers, etc.)”.
- Normal containers share the node’s kernel – one kernel vulnerability is enough for an escape. gVisor puts its own kernel written in Go (Sentry) in between.
- Chain: Pod
runtimeClassName→ RuntimeClasshandler→ containerd runtime of the same name → shimcontainerd-shim-runsc-v1→runsc. - Kubernetes installs nothing: the binary and the containerd entry belong on every node where such Pods are supposed to run.
Pod (runtimeClassName: gvisor)
-> RuntimeClass gvisor (handler: runsc)
-> containerd ...runtimes.runsc (runtime_type io.containerd.runsc.v1)
-> Sentry (its own "kernel") -> few syscalls -> node kernel
Preparation on cp1:
mkdir -p ~/cks/tag11 && cd ~/cks/tag11
k create ns tag11
k config set-context --current --namespace=tag11
k get runtimeclassExpected: No resources found
Drill 1: Installing gVisor
On worker1 (ssh [email protected]), then
the same on worker2. Since 2026 gVisor ships as an archive:
runsc, the containerd shim and a folder
gvisor-bin/ that has to sit next to runsc. A
fixed version instead of latest, with a checksum:
cd /tmp
V=20261116.0
ARCH=$(uname -m)
U=https://storage.googleapis.com/gvisor/releases/release/$V/${ARCH}
curl -sSfLO $U/gvisor.tar.zstd
curl -sSfLO $U/gvisor.tar.zstd.sha512
sha512sum -c gvisor.tar.zstd.sha512
tar --zstd -tf gvisor.tar.zstd
sudo tar --zstd -xf gvisor.tar.zstd -C /usr/local/bin
rm gvisor.tar.zstd gvisor.tar.zstd.sha512
runsc --versionExpected:
gvisor.tar.zstd: OK
gvisor-bin/
runsc
containerd-shim-runsc-v1
gvisor-bin/checkpointgofer
gvisor-bin/runsc-metric-server
gvisor-bin/runsc-netgofer
gvisor-bin/runsc-fd-parking
gvisor-bin/gvisor-sentry-prewarmer
gvisor-bin/gvisor-proc-umounter
gvisor-bin/gvisor_sentry
runsc version release-20261116.0
spec: 1.2.1
The .bz2 archive would need bzip2, which is
missing on Ubuntu servers – zstd is there. Alternative for
production: gVisor’s apt repository (package runsc).
Save your progress? Sign up for free
Drill 2: Registering the runtime handler in containerd
containerd 2.4 writes its configuration as version = 4;
the CRI runtimes live under the plugin
io.containerd.cri.v1.runtime (in containerd 1.x it was
called io.containerd.grpc.v1.cri). Look first, then append
with a backup – on both workers:
sudo head -1 /etc/containerd/config.toml
sudo grep -n 'runtimes\.' /etc/containerd/config.toml
sudo cp -p /etc/containerd/config.toml /etc/containerd/config.toml.tag11
sudo tee -a /etc/containerd/config.toml <<'EOF'
[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.runsc]
runtime_type = 'io.containerd.runsc.v1'
EOF
sudo systemctl restart containerd
systemctl is-active containerd
sudo crictl info | jq -r '.config.containerd.runtimes | keys[]'Expected:
version = 4
64: [plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.runc]
80: [plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.runc.options]
[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.runsc]
runtime_type = 'io.containerd.runsc.v1'
active
runc
runsc
The name runsc after runtimes. is the
handler name for Kubernetes. io.containerd.runsc.v1 tells
containerd to look for containerd-shim-runsc-v1 in the
PATH. Running containers survive the containerd restart;
check on cp1 anyway:
k get nodes
k get pods -A --no-headers | grep -v -E 'Running|Completed'Expected: all nodes Ready, no further
lines.
Save your progress? Sign up for free
Drill 3: RuntimeClass and the first sandbox Pod
On cp1:
cat > rc.yaml <<'EOF'
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
name: gvisor
handler: runsc
EOF
k apply -f rc.yaml
k get runtimeclass
cat > sandbox.yaml <<'EOF'
apiVersion: v1
kind: Pod
metadata:
name: sandbox
spec:
runtimeClassName: gvisor
containers:
- name: c
image: busybox:1.37
command: ["sleep", "1d"]
EOF
k apply -f sandbox.yaml
k wait --for=condition=Ready pod/sandbox
k get pod sandbox -o wide | awk '{print $1, $3, $7}'Expected:
runtimeclass.node.k8s.io/gvisor created
NAME HANDLER AGE
gvisor runsc 0s
pod/sandbox created
pod/sandbox condition met
NAME STATUS NODE
sandbox Running worker2
RuntimeClass is cluster-wide (no namespace) and has no
k create shortcut – the YAML is in the docs under “Runtime
Class”.
Save your progress? Sign up for free
Drill 4: Proof – a different kernel
k exec sandbox -- dmesg
k exec sandbox -- uname -r
k run normal --image=busybox:1.37 --overrides='{"spec":{"nodeName":"worker2"}}' \
-- sleep 1d
k wait --for=condition=Ready pod/normal
k exec normal -- uname -r
k exec normal -- dmesgExpected (abridged, the quips vary):
[ 0.000000] Starting gVisor...
[ 0.534806] Granting licence to kill(2)...
[ 0.553271] Accelerating teletypewriter to 9600 baud...
...
[ 2.151784] Ready!
4.19.0-gvisor
pod/normal condition met
6.8.0-149-generic
dmesg: klogctl: Operation not permitted
command terminated with exit code 1
In the sandbox Pod the gVisor kernel answers
(4.19.0-gvisor), in the normal Pod the node’s real Ubuntu
kernel. From the outside, on the sandbox Pod’s node (here
worker2):
ps -C sleep -o pid,args
ps -eo comm | grep -E 'runsc|gvisor' | sort | uniq -c
sudo crictl pods --name sandbox -o json | grep -i runtimeHandlerExpected (abridged):
PID COMMAND
27743 sleep 1d
8 gvisor_sentry
2 runsc
1 runsc-fd-parkin
"runtimeHandler": "runsc"
Only one sleep 1d – the normal Pod’s. The process in the
sandbox Pod doesn’t exist for the host kernel; it lives in
gvisor_sentry.
Save your progress? Sign up for free
Drill 5: Failure patterns
k create -f - <<'EOF'
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
name: tag11-kaputt
handler: kata
EOF
k run kaputt --image=busybox:1.37 \
--overrides='{"spec":{"runtimeClassName":"tag11-kaputt"}}' -- sleep 1d
k run fehlt --image=busybox:1.37 \
--overrides='{"spec":{"runtimeClassName":"gibtsnicht"}}' -- sleep 1d
k get pod kaputt -w # Ctrl+C after about 15 s
k get events --field-selector involvedObject.name=kaputt,reason=FailedCreatePodSandBox \
-o jsonpath='{.items[0].message}{"\n"}'Expected (abridged):
runtimeclass.node.k8s.io/tag11-kaputt created
pod/kaputt created
Error from server (Forbidden): pods "fehlt" is forbidden: pod rejected: RuntimeClass
"gibtsnicht" not found
NAME READY STATUS RESTARTS AGE
kaputt 0/1 ContainerCreating 0 15s
Failed to create pod sandbox: rpc error: code = Unknown desc = unable to get OCI runtime
for sandbox "8946f2c7…": no runtime for "kata" is configured
If the RuntimeClass is missing, the API server rejects the Pod. If
the handler is missing on the node, the Pod hangs in
ContainerCreating – the cause shows up only in the events.
Clean up:
k delete pod kaputt --now
k delete runtimeclass tag11-kaputtSave your progress? Sign up for free
Drill 6: A real app in the sandbox
k create secret generic notes-key --from-literal=NOTES_API_KEY=lab-key-123
k create deploy notes --image=172.16.0.126:30500/notes-api:1.0 --replicas=2 \
$do > notes.yaml
sed -i 's/^ spec:/&\n runtimeClassName: gvisor/' notes.yaml
k apply -f notes.yaml
k set env deploy/notes --from=secret/notes-key
k rollout status deploy/notes
k expose deploy notes --port=8080
k get pods -l app=notes -o wide | awk '{print $1, $3, $7}'
k exec normal -- wget -qO- -T5 http://notes:8080/healthz; echo
k exec deploy/notes -- uname -rExpected (abridged):
deployment "notes" successfully rolled out
service/notes exposed
NAME STATUS NODE
notes-5ddf68d9ff-4p62s Running worker2
notes-5ddf68d9ff-cjzsh Running worker1
ok
4.19.0-gvisor
runtimeClassName belongs in the Pod template
(spec.template.spec). Network, DNS, Service and Cilium work
unchanged; gVisor brings its own network stack, and the packets leave
the sandbox through the normal veth interface.
Save your progress? Sign up for free
Drill 7: Scheduling in the RuntimeClass
cp1 has no gVisor. So that sandbox Pods never land on a
node without the handler, the RuntimeClass itself carries a
nodeSelector (and tolerations if needed):
k label node worker1 sandbox=gvisor
k patch runtimeclass gvisor --type=merge \
-p '{"scheduling":{"nodeSelector":{"sandbox":"gvisor"}}}'
k rollout restart deploy/notes
k rollout status deploy/notes
k get pods -l app=notes \
-o custom-columns=NAME:.metadata.name,NODE:.spec.nodeName,SEL:.spec.nodeSelectorExpected (abridged):
node/worker1 labeled
runtimeclass.node.k8s.io/gvisor patched
deployment "notes" successfully rolled out
NAME NODE SEL
notes-679466cd77-dzxpx worker1 map[sandbox:gvisor]
notes-679466cd77-w9hvq worker1 map[sandbox:gvisor]
The RuntimeClass admission controller merges the
selector into every Pod at creation. Both Pods now land on
worker1 because only it has the label. worker2
has gVisor too:
k label node worker2 sandbox=gvisor
k get nodes -L sandboxExpected:
node/worker2 labeled
NAME STATUS ROLES AGE VERSION SANDBOX
cp1 Ready control-plane 97m ...
worker1 Ready <none> 95m ... gvisor
worker2 Ready <none> 95m ... gvisor
Save your progress? Sign up for free
Drill 8: Limits
k run hp --image=busybox:1.37 \
--overrides='{"spec":{"runtimeClassName":"gvisor","hostPID":true}}' -- sleep 1d
k wait --for=condition=Ready pod/hp
k exec hp -- ps -o pid,args
k patch runtimeclass gvisor --type=merge -p '{"handler":"runc"}'
k delete pod hp --nowExpected (abridged):
pod/hp condition met
PID COMMAND
1 /pause
2 sleep 1d
3 ps -o pid,args
The RuntimeClass "gvisor" is invalid: handler: ... field is immutable
hostPID is accepted, but the Pod still sees only its
sandbox. What gVisor can’t do, or does poorly:
- Host access (host namespaces, devices, kernel modules) – CNI, CSI
and monitoring Pods stay on
runc. - Not every syscall is implemented; I/O- and syscall-heavy apps get
slower, and every Pod needs more memory (
overhead.podFixedin the RuntimeClass factors that into scheduling). - Host tools that observe syscalls via eBPF (Days 16, 17) see only the Sentry, not the processes in the Pod.
- A RuntimeClass’s handler is immutable – changing it means deleting and recreating.
Save your progress? Sign up for free
Exercise: Speed Drills
- Which runtime handlers does containerd know on
worker1? Target time: 1 min - Start Pod
sb(busybox:1.37,sleep 1d) in gVisor and prove it with one command in the Pod. Target time: 1 min - List all Pods in the cluster with their RuntimeClass. Target time: 1 min
Solution
ssh [email protected] \
"sudo crictl info | jq -r '.config.containerd.runtimes | keys[]'"
k run sb --image=busybox:1.37 \
--overrides='{"spec":{"runtimeClassName":"gvisor"}}' -- sleep 1d
k wait --for=condition=Ready pod/sb
k exec sb -- dmesg | head -1
k get pods -A -o custom-columns=\
NS:.metadata.namespace,NAME:.metadata.name,RC:.spec.runtimeClassName | grep -v '<none>'Expected (abridged):
runc
runsc
pod/sb condition met
[ 0.000000] Starting gVisor...
NS NAME RC
tag11 notes-679466cd77-dzxpx gvisor
tag11 notes-679466cd77-w9hvq gvisor
tag11 sandbox gvisor
tag11 sb gvisor
Save your progress? Sign up for free
Checkpoint
ssh [email protected] 'runsc --version | head -1'Expected:
runsc version release-20261116.0 (the same on
172.16.0.228).
k get runtimeclass gvisor -o jsonpath='{.handler} {.scheduling.nodeSelector}{"\n"}'Expected:
runsc {"sandbox":"gvisor"}
k get nodes -l sandbox=gvisorExpected: worker1 and
worker2.
k get ns tag11 tag11-chExpected (after cleanup): NotFound
twice.
In the lab, these stay: gVisor
(/usr/local/bin/runsc, shim, gvisor-bin/) and
the containerd entry runsc on both workers, the
RuntimeClass gvisor with nodeSelector, and the
label sandbox=gvisor on both workers. Day 19 uses them.
Challenge
A web server is running in the namespace tag11-ch
(below). From now on it is to run in gVisor, via a new RuntimeClass
tag11-untrusted. Prove for every Pod that it runs in the
sandbox.
k create ns tag11-ch
k -n tag11-ch create deploy web --image=busybox:1.37 --replicas=2 \
-- httpd -f -p 8080Hint: A patch to the Pod template triggers the rollout by itself.
Solution
k create -f - <<'EOF'
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
name: tag11-untrusted
handler: runsc
EOF
k -n tag11-ch patch deploy web \
-p '{"spec":{"template":{"spec":{"runtimeClassName":"tag11-untrusted"}}}}'
k -n tag11-ch rollout status deploy/web
k -n tag11-ch get pods -o custom-columns=\
NAME:.metadata.name,RC:.spec.runtimeClassName,NODE:.spec.nodeName
for p in $(k -n tag11-ch get pods -l app=web -o name); do
k -n tag11-ch exec $p -- dmesg | head -1
doneExpected (abridged, once the old Pods are gone):
deployment "web" successfully rolled out
NAME RC NODE
web-5745ffc6b-rpcmf tag11-untrusted worker2
web-5745ffc6b-rrg8t tag11-untrusted worker1
[ 0.000000] Starting gVisor...
[ 0.000000] Starting gVisor...
Clean up (the RuntimeClass gvisor stays):
k config set-context --current --namespace=default
k delete ns tag11 tag11-ch
k delete runtimeclass tag11-untrustedSave your progress? Sign up for free
Exam Tip
- In the exam gVisor is usually already installed: read the
handlerfrom the containerd config (runtimes.<name>), create the RuntimeClass, putruntimeClassNameinto the Pod template. Don’t forget the proof viadmesgoruname -rin the Pod. - If a Pod is stuck in
ContainerCreating, the events tell you whether the handler is missing on that node. After a containerd change, alwayssystemctl restart containerd. - Docs: kubernetes.io → Concepts → Containers → “Runtime Class”; gvisor.dev → “Containerd Quick Start”.
Enterprise Note
- Managed offerings: GKE Sandbox (gVisor), AKS Pod Sandboxing and Kata Containers (lightweight VMs instead of a Sentry) – same RuntimeClass mechanics, different handler.
- Sandboxing pays off for foreign or untrusted code: build jobs, customer plugins, multi-tenant platforms. For your own, vetted services the extra cost is usually too high.
- Separate sandbox node pools by label and taint; the RuntimeClass
brings the matching
nodeSelectorandtolerations. - Admission rules (Kyverno, ValidatingAdmissionPolicy) can enforce
runtimeClassNamefor certain namespaces. - Update gVisor regularly like any runtime – pin the release channel and checksum.