Skip to content
Hands-on IT IT Drills

Day 5: Volumes

Goal: You give pods ephemeral and persistent storage and choose the right volume type for it. · Duration: approx. 60 min · Prerequisite: Day 1

Briefing

CKAD: Application Design and Build (20 %) – “Utilize persistent and ephemeral volumes”.

ephemeral (lives with the pod)     persistent (lives without the pod)
  emptyDir                           PVC --> PV (dynamic via StorageClass
  projected / configMap / secret                 or static by hand)
  ephemeral (PVC per pod)

In the lab, local-path is the default StorageClass: it creates volumes as a directory on the node (RWO/RWOP only) and only binds once a pod uses the PVC.

mkdir -p ~/ckad/tag05 && cd ~/ckad/tag05
k create ns tag05
k config set-context --current --namespace=tag05

Drill 1: emptyDir in RAM

Two containers share a directory. The reader mounts it read-only.

cat > cache.yaml <<'EOF'
apiVersion: v1
kind: Pod
metadata:
  name: cache
spec:
  containers:
  - name: writer
    image: busybox:1.37
    command: ["sh", "-c", "while true; do date >> /cache/log; sleep 5; done"]
    volumeMounts:
    - name: cache
      mountPath: /cache
  - name: reader
    image: busybox:1.37
    command: ["sh", "-c", "sleep 3600"]
    volumeMounts:
    - name: cache
      mountPath: /cache
      readOnly: true
  volumes:
  - name: cache
    emptyDir:
      medium: Memory
      sizeLimit: 64Mi
EOF
k apply -f cache.yaml
k wait --for=condition=Ready pod/cache
sleep 10
k exec cache -c reader -- tail -2 /cache/log
k exec cache -c reader -- df -h /cache
k exec cache -c reader -- touch /cache/x

Expected:

Wed Nov 18 15:26:10 UTC 2026
Wed Nov 18 15:26:15 UTC 2026
Filesystem                Size      Used Available Use% Mounted on
tmpfs                    64.0M      4.0K     64.0M   0% /cache
touch: /cache/x: Read-only file system

medium: Memory turns the volume into a tmpfs, sizeLimit sets its size. Whatever is stored there counts toward the container’s memory usage. Try writing more:

k exec cache -c writer -- dd if=/dev/zero of=/cache/big bs=1M count=100

Expected (shortened):

dd: error writing '/cache/big': No space left on device
67104768 bytes (64.0MB) copied, 0.180261 seconds, 355.0MB/s

Without medium, the emptyDir lives on the node’s disk; there the kubelet monitors the sizeLimit and evicts the pod if it is exceeded.

Save your progress? Sign up for free

Drill 2: hostPath – only with care

k delete pod cache --now
cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: nodelog
spec:
  containers:
  - name: c
    image: busybox:1.37
    command: ["sh", "-c", "sleep 3600"]
    volumeMounts:
    - name: varlog
      mountPath: /host/var/log
      readOnly: true
  volumes:
  - name: varlog
    hostPath:
      path: /var/log
      type: Directory
EOF
k wait --for=condition=Ready pod/nodelog
k exec nodelog -- ls /host/var/log/pods | head -2

Expected (shortened):

calico-system_calico-node-zjlh8_2382af01-417c-4b72-bd87-6f45d3ed34ab
calico-system_csi-node-driver-pd2g4_02bed54d-7eef-47bc-ace0-424b1247cf08

Warning: hostPath opens up the node’s file system. A pod with write access to / or /var/run can take over the node. Almost never right for applications; Pod Security Standards (Day 9) forbid it from baseline upward.

k delete pod nodelog --now

Save your progress? Sign up for free

Drill 3: PVC with a StorageClass

cat <<'EOF' | k apply -f -
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: notes-data
spec:
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 1Gi
EOF
k get pvc notes-data
k describe pvc notes-data | tail -1

Expected (shortened):

NAME         STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   ...
notes-data   Pending                                      local-path     ...
  Normal  WaitForFirstConsumer  1s    persistentvolume-controller  waiting for first
  consumer to be created before binding

No storageClassName → the default class applies. Pending is not an error here: WaitForFirstConsumer waits until the scheduler knows which node the pod lands on.

Save your progress? Sign up for free

Drill 4: Data survives the pod

cat > store.yaml <<'EOF'
apiVersion: v1
kind: Pod
metadata:
  name: store
spec:
  containers:
  - name: c
    image: busybox:1.37
    command: ["sh", "-c", "echo start $(hostname) >> /data/log; sleep 3600"]
    volumeMounts:
    - name: data
      mountPath: /data
  volumes:
  - name: data
    persistentVolumeClaim:
      claimName: notes-data
EOF
k apply -f store.yaml
k wait --for=condition=Ready pod/store --timeout=90s
k get pvc notes-data
k delete pod store --now
k apply -f store.yaml
k wait --for=condition=Ready pod/store
k exec store -- cat /data/log

Expected (shortened):

NAME         STATUS   VOLUME                                     CAPACITY   ACCESS MODES
notes-data   Bound    pvc-454231cd-5f97-452b-9b90-aeda506caa82   1Gi        RWO
start store
start store

Two lines: the second pod kept writing to the first pod’s file. The local-path PV is tied to a node via nodeAffinity – the new pod necessarily lands there.

Now the counter-test with ReadWriteMany:

cat <<'EOF' | k apply -f -
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: shared
spec:
  accessModes: ["ReadWriteMany"]
  resources:
    requests:
      storage: 1Gi
EOF
sed 's/notes-data/shared/; s/name: store/name: rwx/' store.yaml | k apply -f -
k describe pvc shared | grep ProvisioningFailed

Expected (shortened):

  Warning  ProvisioningFailed  0s (x2 over 15s)  rancher.io/local-path_...  failed to
  provision volume with StorageClass "local-path": NodePath only supports ReadWriteOnce
  and ReadWriteOncePod (1.22+) access modes

Which access modes work is decided by the provisioner, not by Kubernetes.

k delete pod rwx --now
k delete pvc shared

Save your progress? Sign up for free

Drill 5: Static PV with ReadWriteOncePod

PVs are cluster-wide, hence a unique name. storageClassName: "" means “no class” – otherwise the default class would grab the PVC.

cat <<'EOF' | k apply -f -
apiVersion: v1
kind: PersistentVolume
metadata:
  name: tag05-static
spec:
  capacity:
    storage: 200Mi
  accessModes: ["ReadWriteOncePod"]
  persistentVolumeReclaimPolicy: Retain
  storageClassName: ""
  hostPath:
    path: /tmp/tag05-static
    type: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: static-claim
spec:
  storageClassName: ""
  volumeName: tag05-static
  accessModes: ["ReadWriteOncePod"]
  resources:
    requests:
      storage: 100Mi
EOF
k get pvc static-claim

Expected (shortened):

NAME           STATUS   VOLUME         CAPACITY   ACCESS MODES   STORAGECLASS
static-claim   Bound    tag05-static   200Mi      RWOP

volumeName binds explicitly to this PV. 100Mi requested, 200Mi received: the PVC occupies the whole PV. Now two pods on the same claim:

sed 's/notes-data/static-claim/; s/name: store/name: one/' store.yaml | k apply -f -
sed 's/notes-data/static-claim/; s/name: store/name: two/' store.yaml | k apply -f -
k get pods one two
k describe pod two | tail -1

Expected (after a few seconds, shortened):

NAME   READY   STATUS    RESTARTS   AGE
one    1/1     Running   0          9s
two    0/1     Pending   0          9s
... 2 node(s) unavailable due to PersistentVolumeClaim with ReadWriteOncePod access mode
already in-use by another pod. ...

RWO allows multiple pods on one node, RWOP exactly one pod.

k delete pod one two --now

Save your progress? Sign up for free

Drill 6: Projected volume

Four sources, one directory:

k create configmap app-cfg --from-literal=LOG_LEVEL=debug
k create secret generic app-key --from-literal=api-key=lab-key-123
cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: proj
  labels:
    app: notes
spec:
  containers:
  - name: c
    image: busybox:1.37
    command: ["sh", "-c", "sleep 3600"]
    volumeMounts:
    - name: all
      mountPath: /etc/app
      readOnly: true
  volumes:
  - name: all
    projected:
      sources:
      - configMap:
          name: app-cfg
      - secret:
          name: app-key
      - downwardAPI:
          items:
          - path: labels
            fieldRef:
              fieldPath: metadata.labels
      - serviceAccountToken:
          path: token
          audience: notes-api
          expirationSeconds: 3600
EOF
k wait --for=condition=Ready pod/proj
k exec proj -- ls /etc/app
k exec proj -- cat /etc/app/labels; echo

Expected:

LOG_LEVEL
api-key
labels
token
app="notes"

The token is a JWT with its own audience (aud: ["notes-api"]) and expires after one hour; the kubelet renews it in time. More on this on Day 8.

Save your progress? Sign up for free

Drill 7: Generic ephemeral volume

Like emptyDir, but from the provisioner – with a size, a StorageClass and its own disk.

cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: scratch
spec:
  containers:
  - name: c
    image: busybox:1.37
    command: ["sh", "-c", "sleep 3600"]
    volumeMounts:
    - name: work
      mountPath: /work
  volumes:
  - name: work
    ephemeral:
      volumeClaimTemplate:
        spec:
          accessModes: ["ReadWriteOnce"]
          storageClassName: local-path
          resources:
            requests:
              storage: 500Mi
EOF
k wait --for=condition=Ready pod/scratch --timeout=90s
k get pvc scratch-work -o jsonpath='{.metadata.ownerReferences[0].kind}{"\n"}'
k delete pod scratch --now
k get pvc

Expected (shortened):

Pod
NAME           STATUS   VOLUME         CAPACITY   ACCESS MODES   STORAGECLASS
notes-data     Bound    pvc-4542...    1Gi        RWO            local-path
static-claim   Bound    tag05-static   200Mi      RWOP

The PVC is named <pod>-<volume> and is owned by the pod. It disappears with the pod.

Save your progress? Sign up for free

Drill 8: subPath – one file instead of a folder

k create configmap nginx-extra \
  --from-literal=extra.conf='server { listen 8080; return 200 "tag05\n"; }'
cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: web
spec:
  containers:
  - name: nginx
    image: nginx:1.29
    volumeMounts:
    - name: extra
      mountPath: /etc/nginx/conf.d/extra.conf
      subPath: extra.conf
  volumes:
  - name: extra
    configMap:
      name: nginx-extra
EOF
k wait --for=condition=Ready pod/web
k exec web -- ls /etc/nginx/conf.d
k exec web -- curl -s localhost:8080

Expected:

default.conf
extra.conf
tag05

Without subPath (mountPath /etc/nginx/conf.d), the volume hides the folder and default.conf is gone. The catch: files mounted via subPath do not receive ConfigMap updates.

Save your progress? Sign up for free

Exercise: Speed Drills

  1. Pod tmp (busybox:1.37, sleep 3600) with emptyDir scratch under /scratch, at most 100Mi. Target time: 2 min
  2. PVC logs (100Mi, RWO) and pod logger that mounts it under /var/log/app. Target time: 3 min
  3. Pod info with label tier=backend that can read its labels under /etc/podinfo/labels. Target time: 2 min

Solution

Always start the skeleton with k run … $do, then add volumes and volumeMounts:

k run tmp --image=busybox:1.37 $do --command -- sleep 3600 > tmp.yaml
spec:
  containers:
  - name: tmp
    image: busybox:1.37
    command: ["sleep", "3600"]
    volumeMounts:
    - name: scratch
      mountPath: /scratch
  volumes:
  - name: scratch
    emptyDir:
      sizeLimit: 100Mi
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: logs
spec:
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 100Mi
---
apiVersion: v1
kind: Pod
metadata:
  name: logger
spec:
  containers:
  - name: logger
    image: busybox:1.37
    command: ["sh", "-c", "while true; do date >> /var/log/app/out; sleep 10; done"]
    volumeMounts:
    - name: logs
      mountPath: /var/log/app
  volumes:
  - name: logs
    persistentVolumeClaim:
      claimName: logs
metadata:
  name: info
  labels:
    tier: backend
spec:
  containers:
  - name: info
    image: busybox:1.37
    command: ["sleep", "3600"]
    volumeMounts:
    - name: podinfo
      mountPath: /etc/podinfo
  volumes:
  - name: podinfo
    downwardAPI:
      items:
      - path: labels
        fieldRef:
          fieldPath: metadata.labels

Save your progress? Sign up for free

Checkpoint

k get pvc logs -o jsonpath='{.status.phase}{"\n"}'

Expected: Bound

k exec info -- cat /etc/podinfo/labels; echo

Expected: tier="backend"

k get pv tag05-static -o jsonpath='{.spec.accessModes}{"\n"}'

Expected: ["ReadWriteOncePod"]

Challenge

A colleague creates a PVC reports (1Gi, RWO) with storageClassName: "". It stays Pending forever. Find the reason and fix it so that local-path serves it.

cat <<'EOF' | k apply -f -
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: reports
spec:
  storageClassName: ""
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 1Gi
EOF

Hint: The spec of a PVC is almost entirely immutable.

Solution

k describe pvc reports | tail -1

Expected (shortened):

Normal  FailedBinding  3s  persistentvolume-controller  no persistent volumes available
for this claim and no storage class is set

"" turns off dynamic provisioning, and there is no matching PV. A patch fails with spec is immutable after creation. So recreate it:

k get pvc reports -o yaml > reports.yaml
sed -i '/storageClassName: ""/d' reports.yaml
k replace --force -f reports.yaml
k get pvc reports

Expected: reports with STORAGECLASS local-path, status Pending (WaitForFirstConsumer) – now for the right reason.

Clean up (the PV is cluster-wide; because of Retain, the directory under /tmp stays on the node until it reboots):

k config set-context --current --namespace=default
k delete ns tag05
k delete pv tag05-static

Save your progress? Sign up for free

Exam Tip

  • The volumes of a running pod are immutable: k get pod x -o yaml > x.yaml, edit, k replace --force -f x.yaml.
  • PVC Pending? Read k describe pvc: WaitForFirstConsumer is normal, “no storage class is set” or a wrong access mode is not.
  • Docs: kubernetes.io → Concepts → Storage → Volumes (all types with a YAML example).

Enterprise Note

  • local-path is lab storage: if the node fails, the data is gone. Production uses CSI drivers (Ceph/Rook, Longhorn, cloud disks) with replication and snapshots.
  • For databases, use StatefulSets with volumeClaimTemplates instead of individual PVCs.
  • Block hostPath with Pod Security Admission or a policy engine (Kyverno, Gatekeeper).
  • RWX (NFS, CephFS) only if the application really tolerates parallel writers.