Skip to content
Hands-on IT IT Drills

Day 8: SecurityContext and ServiceAccounts

Goal: You run containers with as few privileges as possible and give pods their own identity towards the Kubernetes API. · Duration: approx. 60 min · Prerequisite: Day 6

Briefing

CKAD: Application Environment, Configuration and Security (25 %) – “Understand ServiceAccounts”, “Understand Application Security (SecurityContexts, Capabilities, etc.)”.

spec.securityContext                 pod level: runAsUser, runAsGroup, fsGroup,
                                     runAsNonRoot, seccompProfile
spec.containers[].securityContext    container level: the same runAs* fields
                                     (take precedence) plus capabilities,
                                     readOnlyRootFilesystem, allowPrivilegeEscalation

RBAC (who may do what) follows on Day 9; today is about the identity itself.

mkdir -p ~/ckad/tag08 && cd ~/ckad/tag08
k create ns tag08
k config set-context --current --namespace=tag08

Drill 1: runAsNonRoot

cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: web
spec:
  securityContext:
    runAsNonRoot: true
  containers:
  - name: nginx
    image: nginx:1.29
EOF
k get pod web
k describe pod web | grep -m1 "Error:"

Expected (shortened):

NAME   READY   STATUS                       RESTARTS   AGE
web    0/1     CreateContainerConfigError   0          10s
  Warning  Failed  7s (x3 over 9s)  kubelet  spec.containers{nginx}: Error: container
  has runAsNonRoot and image will run as root (pod: "web_tag08(ef75…)",
  container: nginx)

Before starting, the kubelet checks which user the image runs as. nginx starts as root – so it does not start. notes-api runs as 65532 and passes the check (Drill 5).

k delete pod web --now

Save your progress? Sign up for free

Drill 2: User, group, fsGroup

cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: ids
spec:
  securityContext:
    runAsUser: 1000
    runAsGroup: 3000
    fsGroup: 2000
  containers:
  - name: c
    image: busybox:1.37
    command: ["sh", "-c", "sleep 3600"]
    volumeMounts:
    - name: data
      mountPath: /data
  volumes:
  - name: data
    emptyDir: {}
EOF
k wait --for=condition=Ready pod/ids
k exec ids -- id
k exec ids -- sh -c 'touch /data/f; ls -ln /data; ls -ldn /data'
k exec ids -- touch /f

Expected:

uid=1000 gid=3000 groups=2000,3000
total 0
-rw-r--r--    1 1000     2000             0 Nov 18 15:47 f
drwxrwsrwx    2 0        2000          4096 Nov 18 15:47 /data
touch: /f: Permission denied

fsGroup applies to the volumes: the directory gets group 2000 and the setgid bit (s), and new files inherit the group. This lets a non-root process write to volumes.

Save your progress? Sign up for free

Drill 3: Container level and a read-only filesystem

cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: hard
spec:
  securityContext:
    runAsUser: 1000
  containers:
  - name: c
    image: busybox:1.37
    command: ["sh", "-c", "sleep 3600"]
    securityContext:
      runAsUser: 2000
      readOnlyRootFilesystem: true
      allowPrivilegeEscalation: false
    volumeMounts:
    - name: tmp
      mountPath: /tmp
  volumes:
  - name: tmp
    emptyDir: {}
EOF
k wait --for=condition=Ready pod/hard
k exec hard -- id -u
k exec hard -- touch /etc/x
k exec hard -- touch /tmp/x && echo tmp ok
k exec hard -- grep NoNewPrivs /proc/1/status

Expected:

2000
touch: /etc/x: Read-only file system
tmp ok
NoNewPrivs: 1

Container level beats pod level (2000 instead of 1000). With a read-only root filesystem, every directory the app really writes to gets an emptyDir. allowPrivilegeEscalation: false sets no_new_privs: setuid programs like sudo no longer grant additional privileges.

Save your progress? Sign up for free

Drill 4: Capabilities and seccomp

cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: caps
spec:
  containers:
  - name: c
    image: busybox:1.37
    command: ["sh", "-c", "sleep 3600"]
    securityContext:
      capabilities:
        drop: ["ALL"]
        add: ["NET_BIND_SERVICE"]
      seccompProfile:
        type: RuntimeDefault
EOF
k run plain --image=busybox:1.37 -- sleep 3600
k wait --for=condition=Ready pod/caps pod/plain
k exec plain -- grep -E 'CapEff|Seccomp:' /proc/1/status
k exec caps -- grep -E 'CapEff|Seccomp:' /proc/1/status
k exec plain -- chown 1000 /tmp && echo chown ok
k exec caps -- chown 1000 /tmp

Expected:

CapEff: 00000000a80425fb
Seccomp:    0
CapEff: 0000000000000400
Seccomp:    2
chown ok
chown: /tmp: Operation not permitted

Both run as root, but caps only has NET_BIND_SERVICE left (bit 10 = 0x400, ports below 1024). Without CHOWN, not even root may change owners. You write capabilities without the CAP_ prefix. Seccomp: 0 shows: without a setting, a pod runs unfiltered; RuntimeDefault (2 = filter active) blocks dangerous system calls.

Save your progress? Sign up for free

Drill 5: Hardened notes-api

All of this together is what the Pod Security Standard restricted (Day 9) requires:

k create secret generic notes-key --from-literal=api-key=lab-key-123
cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: notes
spec:
  securityContext:
    runAsNonRoot: true
    seccompProfile:
      type: RuntimeDefault
  containers:
  - name: notes-api
    image: 172.16.0.126:30500/notes-api:1.0
    env:
    - name: NOTES_API_KEY
      valueFrom:
        secretKeyRef:
          name: notes-key
          key: api-key
    securityContext:
      readOnlyRootFilesystem: true
      allowPrivilegeEscalation: false
      capabilities:
        drop: ["ALL"]
EOF
k wait --for=condition=Ready pod/notes
k exec notes -- id
k exec notes -- wget -qO- localhost:8080/healthz

Expected:

pod/notes condition met
uid=65532 gid=0(root) groups=0(root)
ok

notes-api listens on 8080 and writes nothing to disk – so it needs neither capabilities nor an emptyDir.

Save your progress? Sign up for free

Drill 6: Your own ServiceAccount

Every namespace has a ServiceAccount default. Separate accounts split privileges per app.

k create serviceaccount notes-sa
cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: api-client
spec:
  serviceAccountName: notes-sa
  containers:
  - name: c
    image: nginx:1.29
    command: ["sleep", "3600"]
EOF
k wait --for=condition=Ready pod/api-client
k exec api-client -- ls /var/run/secrets/kubernetes.io/serviceaccount
k get pod api-client -o jsonpath='{.spec.volumes[0].projected.sources[0]}{"\n"}'

Expected:

ca.crt
namespace
token
{"serviceAccountToken":{"expirationSeconds":3607,"path":"token"}}

The token is no longer a Secret object but a projected, time-limited JWT (volume kube-api-access-…, as in Day 5 Drill 6). nginx is used here only for curl.

Save your progress? Sign up for free

Drill 7: Query the API with the token

k exec api-client -- sh -c 'D=/var/run/secrets/kubernetes.io/serviceaccount;
  curl -s --cacert $D/ca.crt -H "Authorization: Bearer $(cat $D/token)" \
  https://kubernetes.default.svc/api/v1/namespaces/tag08/pods' | grep -E 'message|code'

Expected:

  "message": "pods is forbidden: User \"system:serviceaccount:tag08:notes-sa\" cannot
  list resource \"pods\" in API group \"\" in the namespace \"tag08\"",
  "code": 403

403 means: the API server knows you (authentication ok), but you are not allowed to do anything (no RBAC rule). Without a token you would be system:anonymous; without --cacert, curl aborts with exit code 60 (unknown certificate). The same from outside, with a short-lived token:

TOKEN=$(k create token notes-sa --duration=10m)
k config view --raw \
  -o jsonpath='{.clusters[0].cluster.certificate-authority-data}' | base64 -d > ca.crt
curl -s --cacert ca.crt -H "Authorization: Bearer $TOKEN" \
  https://172.16.0.126:6443/api/v1/namespaces/tag08/pods | grep code
k auth can-i list pods --as=system:serviceaccount:tag08:notes-sa

Expected:

  "code": 403
no

k create token creates the token via the TokenRequest API; it is not stored anywhere. --audience defines who it is valid for.

Save your progress? Sign up for free

Drill 8: No token where none is needed

Most apps never talk to the Kubernetes API. In that case the token does not belong in the pod:

cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: no-token
spec:
  serviceAccountName: notes-sa
  automountServiceAccountToken: false
  containers:
  - name: c
    image: busybox:1.37
    command: ["sleep", "3600"]
EOF
k wait --for=condition=Ready pod/no-token
k exec no-token -- ls /var/run/secrets/kubernetes.io/serviceaccount

Expected:

ls: /var/run/secrets/kubernetes.io/serviceaccount: No such file or directory

The same field exists on the ServiceAccount (k patch sa notes-sa -p '{"automountServiceAccountToken":false}'); the value on the pod wins.

Save your progress? Sign up for free

Exercise: Speed Drills

  1. Pod secure (busybox:1.37, sleep 3600) runs as user 1001, group 1001 and without privilege escalation. Target time: 2 min
  2. ServiceAccount builder; Deployment ci (nginx:1.29) uses it. Target time: 45 s
  3. Create a token for builder, valid for one hour. Target time: 20 s

Solution

k run secure --image=busybox:1.37 $do --command -- sleep 3600 > secure.yaml
spec:
  securityContext:
    runAsUser: 1001
    runAsGroup: 1001
  containers:
  - name: secure
    image: busybox:1.37
    command: ["sleep", "3600"]
    securityContext:
      allowPrivilegeEscalation: false
k apply -f secure.yaml
k create sa builder
k create deploy ci --image=nginx:1.29
k set serviceaccount deploy/ci builder
k create token builder --duration=1h

Expected (shortened):

pod/secure created
serviceaccount/builder created
deployment.apps/ci created
deployment.apps/ci serviceaccount updated
eyJhbGciOiJSUzI1NiIs...

Save your progress? Sign up for free

Checkpoint

k exec secure -- id

Expected: uid=1001 gid=1001 groups=1001

k get deploy ci -o jsonpath='{.spec.template.spec.serviceAccountName}{"\n"}'

Expected: builder

k get pod notes -o jsonpath='{.spec.containers[0].securityContext.capabilities}{"\n"}'

Expected: {"drop":["ALL"]}

Challenge

Pod peek (busybox:1.37) runs with notes-sa but should have no API token in the default location. Instead it needs a token with audience vault under /var/run/vault/token that expires after 10 minutes.

Hint: A projected volume can supply a serviceAccountToken even without automount.

Solution

cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: peek
spec:
  serviceAccountName: notes-sa
  automountServiceAccountToken: false
  containers:
  - name: c
    image: busybox:1.37
    command: ["sleep", "3600"]
    volumeMounts:
    - name: vault-token
      mountPath: /var/run/vault
      readOnly: true
  volumes:
  - name: vault-token
    projected:
      sources:
      - serviceAccountToken:
          path: token
          audience: vault
          expirationSeconds: 600
EOF
k wait --for=condition=Ready pod/peek
k exec peek -- ls /var/run/vault
k exec peek -- cat /var/run/vault/token | cut -d. -f2 | base64 -d 2>/dev/null \
  | cut -c1-40

Expected:

token
{"aud":["vault"],"exp":1795017538,"iat":

600 seconds is the minimum for expirationSeconds. Vault (or any other service) verifies signature and audience via the TokenReview API.

Clean up:

k config set-context --current --namespace=default
k delete ns tag08

Save your progress? Sign up for free

Exam Tip

  • Fields like capabilities and readOnlyRootFilesystem exist only on the container, fsGroup only on the pod. When in doubt, k explain pod.spec.securityContext and k explain pod.spec.containers.securityContext.
  • serviceAccountName is immutable on a running pod – for a Deployment use k set serviceaccount, for a pod k replace --force.
  • Docs: kubernetes.io → Tasks → Configure Pods → “Configure a Security Context” and “Configure Service Accounts for Pods”.

Enterprise Note

  • Pod Security Admission with restricted on all app namespaces enforces the fields from Drill 5 (Day 9).
  • Build images with a fixed, numeric non-root USER (Day 2); otherwise the kubelet cannot verify runAsNonRoot.
  • ServiceAccount tokens are workload identity: cloud IAM (IRSA, Workload Identity) and Vault trust them via the audience.
  • automountServiceAccountToken: false as the default for all ServiceAccounts that do not need the API.