Day 8: SecurityContext and ServiceAccounts
Goal: You run containers with as few privileges as possible and give pods their own identity towards the Kubernetes API. · Duration: approx. 60 min · Prerequisite: Day 6
Briefing
CKAD: Application Environment, Configuration and Security (25 %) – “Understand ServiceAccounts”, “Understand Application Security (SecurityContexts, Capabilities, etc.)”.
spec.securityContext pod level: runAsUser, runAsGroup, fsGroup,
runAsNonRoot, seccompProfile
spec.containers[].securityContext container level: the same runAs* fields
(take precedence) plus capabilities,
readOnlyRootFilesystem, allowPrivilegeEscalation
RBAC (who may do what) follows on Day 9; today is about the identity itself.
mkdir -p ~/ckad/tag08 && cd ~/ckad/tag08
k create ns tag08
k config set-context --current --namespace=tag08Drill 1: runAsNonRoot
cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
name: web
spec:
securityContext:
runAsNonRoot: true
containers:
- name: nginx
image: nginx:1.29
EOF
k get pod web
k describe pod web | grep -m1 "Error:"Expected (shortened):
NAME READY STATUS RESTARTS AGE
web 0/1 CreateContainerConfigError 0 10s
Warning Failed 7s (x3 over 9s) kubelet spec.containers{nginx}: Error: container
has runAsNonRoot and image will run as root (pod: "web_tag08(ef75…)",
container: nginx)
Before starting, the kubelet checks which user the image runs as. nginx starts as root – so it does not start. notes-api runs as 65532 and passes the check (Drill 5).
k delete pod web --nowSave your progress? Sign up for free
Drill 2: User, group, fsGroup
cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
name: ids
spec:
securityContext:
runAsUser: 1000
runAsGroup: 3000
fsGroup: 2000
containers:
- name: c
image: busybox:1.37
command: ["sh", "-c", "sleep 3600"]
volumeMounts:
- name: data
mountPath: /data
volumes:
- name: data
emptyDir: {}
EOF
k wait --for=condition=Ready pod/ids
k exec ids -- id
k exec ids -- sh -c 'touch /data/f; ls -ln /data; ls -ldn /data'
k exec ids -- touch /fExpected:
uid=1000 gid=3000 groups=2000,3000
total 0
-rw-r--r-- 1 1000 2000 0 Nov 18 15:47 f
drwxrwsrwx 2 0 2000 4096 Nov 18 15:47 /data
touch: /f: Permission denied
fsGroup applies to the volumes: the directory gets group
2000 and the setgid bit (s), and new files inherit the
group. This lets a non-root process write to volumes.
Save your progress? Sign up for free
Drill 3: Container level and a read-only filesystem
cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
name: hard
spec:
securityContext:
runAsUser: 1000
containers:
- name: c
image: busybox:1.37
command: ["sh", "-c", "sleep 3600"]
securityContext:
runAsUser: 2000
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
volumeMounts:
- name: tmp
mountPath: /tmp
volumes:
- name: tmp
emptyDir: {}
EOF
k wait --for=condition=Ready pod/hard
k exec hard -- id -u
k exec hard -- touch /etc/x
k exec hard -- touch /tmp/x && echo tmp ok
k exec hard -- grep NoNewPrivs /proc/1/statusExpected:
2000
touch: /etc/x: Read-only file system
tmp ok
NoNewPrivs: 1
Container level beats pod level (2000 instead of 1000). With a
read-only root filesystem, every directory the app really writes to gets
an emptyDir. allowPrivilegeEscalation: false sets
no_new_privs: setuid programs like sudo no
longer grant additional privileges.
Save your progress? Sign up for free
Drill 4: Capabilities and seccomp
cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
name: caps
spec:
containers:
- name: c
image: busybox:1.37
command: ["sh", "-c", "sleep 3600"]
securityContext:
capabilities:
drop: ["ALL"]
add: ["NET_BIND_SERVICE"]
seccompProfile:
type: RuntimeDefault
EOF
k run plain --image=busybox:1.37 -- sleep 3600
k wait --for=condition=Ready pod/caps pod/plain
k exec plain -- grep -E 'CapEff|Seccomp:' /proc/1/status
k exec caps -- grep -E 'CapEff|Seccomp:' /proc/1/status
k exec plain -- chown 1000 /tmp && echo chown ok
k exec caps -- chown 1000 /tmpExpected:
CapEff: 00000000a80425fb
Seccomp: 0
CapEff: 0000000000000400
Seccomp: 2
chown ok
chown: /tmp: Operation not permitted
Both run as root, but caps only has
NET_BIND_SERVICE left (bit 10 = 0x400, ports
below 1024). Without CHOWN, not even root may change
owners. You write capabilities without the CAP_ prefix.
Seccomp: 0 shows: without a setting, a pod runs
unfiltered; RuntimeDefault (2 = filter
active) blocks dangerous system calls.
Save your progress? Sign up for free
Drill 5: Hardened notes-api
All of this together is what the Pod Security Standard
restricted (Day 9) requires:
k create secret generic notes-key --from-literal=api-key=lab-key-123
cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
name: notes
spec:
securityContext:
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
containers:
- name: notes-api
image: 172.16.0.126:30500/notes-api:1.0
env:
- name: NOTES_API_KEY
valueFrom:
secretKeyRef:
name: notes-key
key: api-key
securityContext:
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
EOF
k wait --for=condition=Ready pod/notes
k exec notes -- id
k exec notes -- wget -qO- localhost:8080/healthzExpected:
pod/notes condition met
uid=65532 gid=0(root) groups=0(root)
ok
notes-api listens on 8080 and writes nothing to disk – so it needs neither capabilities nor an emptyDir.
Save your progress? Sign up for free
Drill 6: Your own ServiceAccount
Every namespace has a ServiceAccount default. Separate
accounts split privileges per app.
k create serviceaccount notes-sa
cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
name: api-client
spec:
serviceAccountName: notes-sa
containers:
- name: c
image: nginx:1.29
command: ["sleep", "3600"]
EOF
k wait --for=condition=Ready pod/api-client
k exec api-client -- ls /var/run/secrets/kubernetes.io/serviceaccount
k get pod api-client -o jsonpath='{.spec.volumes[0].projected.sources[0]}{"\n"}'Expected:
ca.crt
namespace
token
{"serviceAccountToken":{"expirationSeconds":3607,"path":"token"}}
The token is no longer a Secret object but a projected, time-limited
JWT (volume kube-api-access-…, as in Day 5 Drill 6). nginx
is used here only for curl.
Save your progress? Sign up for free
Drill 7: Query the API with the token
k exec api-client -- sh -c 'D=/var/run/secrets/kubernetes.io/serviceaccount;
curl -s --cacert $D/ca.crt -H "Authorization: Bearer $(cat $D/token)" \
https://kubernetes.default.svc/api/v1/namespaces/tag08/pods' | grep -E 'message|code'Expected:
"message": "pods is forbidden: User \"system:serviceaccount:tag08:notes-sa\" cannot
list resource \"pods\" in API group \"\" in the namespace \"tag08\"",
"code": 403
403 means: the API server knows you (authentication ok), but you are
not allowed to do anything (no RBAC rule). Without a token you would be
system:anonymous; without --cacert,
curl aborts with exit code 60 (unknown certificate). The
same from outside, with a short-lived token:
TOKEN=$(k create token notes-sa --duration=10m)
k config view --raw \
-o jsonpath='{.clusters[0].cluster.certificate-authority-data}' | base64 -d > ca.crt
curl -s --cacert ca.crt -H "Authorization: Bearer $TOKEN" \
https://172.16.0.126:6443/api/v1/namespaces/tag08/pods | grep code
k auth can-i list pods --as=system:serviceaccount:tag08:notes-saExpected:
"code": 403
no
k create token creates the token via the TokenRequest
API; it is not stored anywhere. --audience defines who it
is valid for.
Save your progress? Sign up for free
Drill 8: No token where none is needed
Most apps never talk to the Kubernetes API. In that case the token does not belong in the pod:
cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
name: no-token
spec:
serviceAccountName: notes-sa
automountServiceAccountToken: false
containers:
- name: c
image: busybox:1.37
command: ["sleep", "3600"]
EOF
k wait --for=condition=Ready pod/no-token
k exec no-token -- ls /var/run/secrets/kubernetes.io/serviceaccountExpected:
ls: /var/run/secrets/kubernetes.io/serviceaccount: No such file or directory
The same field exists on the ServiceAccount
(k patch sa notes-sa -p '{"automountServiceAccountToken":false}');
the value on the pod wins.
Save your progress? Sign up for free
Exercise: Speed Drills
- Pod
secure(busybox:1.37,sleep 3600) runs as user 1001, group 1001 and without privilege escalation. Target time: 2 min - ServiceAccount
builder; Deploymentci(nginx:1.29) uses it. Target time: 45 s - Create a token for
builder, valid for one hour. Target time: 20 s
Solution
k run secure --image=busybox:1.37 $do --command -- sleep 3600 > secure.yamlspec:
securityContext:
runAsUser: 1001
runAsGroup: 1001
containers:
- name: secure
image: busybox:1.37
command: ["sleep", "3600"]
securityContext:
allowPrivilegeEscalation: falsek apply -f secure.yaml
k create sa builder
k create deploy ci --image=nginx:1.29
k set serviceaccount deploy/ci builder
k create token builder --duration=1hExpected (shortened):
pod/secure created
serviceaccount/builder created
deployment.apps/ci created
deployment.apps/ci serviceaccount updated
eyJhbGciOiJSUzI1NiIs...
Save your progress? Sign up for free
Checkpoint
k exec secure -- idExpected:
uid=1001 gid=1001 groups=1001
k get deploy ci -o jsonpath='{.spec.template.spec.serviceAccountName}{"\n"}'Expected: builder
k get pod notes -o jsonpath='{.spec.containers[0].securityContext.capabilities}{"\n"}'Expected: {"drop":["ALL"]}
Challenge
Pod peek (busybox:1.37) runs with notes-sa
but should have no API token in the default location.
Instead it needs a token with audience vault under
/var/run/vault/token that expires after 10 minutes.
Hint: A projected volume can supply a
serviceAccountToken even without automount.
Solution
cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
name: peek
spec:
serviceAccountName: notes-sa
automountServiceAccountToken: false
containers:
- name: c
image: busybox:1.37
command: ["sleep", "3600"]
volumeMounts:
- name: vault-token
mountPath: /var/run/vault
readOnly: true
volumes:
- name: vault-token
projected:
sources:
- serviceAccountToken:
path: token
audience: vault
expirationSeconds: 600
EOF
k wait --for=condition=Ready pod/peek
k exec peek -- ls /var/run/vault
k exec peek -- cat /var/run/vault/token | cut -d. -f2 | base64 -d 2>/dev/null \
| cut -c1-40Expected:
token
{"aud":["vault"],"exp":1795017538,"iat":
600 seconds is the minimum for expirationSeconds. Vault
(or any other service) verifies signature and audience via the
TokenReview API.
Clean up:
k config set-context --current --namespace=default
k delete ns tag08Save your progress? Sign up for free
Exam Tip
- Fields like
capabilitiesandreadOnlyRootFilesystemexist only on the container,fsGrouponly on the pod. When in doubt,k explain pod.spec.securityContextandk explain pod.spec.containers.securityContext. serviceAccountNameis immutable on a running pod – for a Deployment usek set serviceaccount, for a podk replace --force.- Docs: kubernetes.io → Tasks → Configure Pods → “Configure a Security Context” and “Configure Service Accounts for Pods”.
Enterprise Note
- Pod Security Admission with
restrictedon all app namespaces enforces the fields from Drill 5 (Day 9). - Build images with a fixed, numeric non-root
USER(Day 2); otherwise the kubelet cannot verifyrunAsNonRoot. - ServiceAccount tokens are workload identity: cloud IAM (IRSA, Workload Identity) and Vault trust them via the audience.
automountServiceAccountToken: falseas the default for all ServiceAccounts that do not need the API.