Skip to content
Hands-on IT IT Drills

Day 4: Multi-Container Pods

Goal: You build pods with init containers, sidecars, adapters and ambassadors and target individual containers precisely. · Duration: approx. 60 min · Prerequisite: Day 3

Briefing

CKAD: Application Design and Build (20 %) – “Understand multi-container Pod design patterns (e.g. sidecar, init and others)”. All containers of a pod share the network (localhost) and volumes, but run as separate processes with their own logs.

  Pod ────────────────────────────────────────────────────────────
  | init-1 -> init-2 -> [native sidecar] -> app  +  classic sidecar |
  |      one after       runs until         shared emptyDir,     |
  |      another, each   the very end       localhost            |
  |      to completion                                           |
  ─────────────────────────────────────────────────────────────────

Patterns: Init (prepare), Sidecar (extend, e.g. ship logs), Adapter (reshape output), Ambassador (broker connections to the outside).

Drill 1: Init containers

k create ns tag04
k config set-context --current --namespace=tag04
mkdir -p ~/ckad/tag04 && cd ~/ckad/tag04
cat > init.yaml <<'EOF'
apiVersion: v1
kind: Pod
metadata:
  name: web
spec:
  initContainers:
  - name: wait-db
    image: busybox:1.37
    command:
    - sh
    - -c
    - until nslookup db.tag04.svc.cluster.local; do echo warte auf db; sleep 2; done
  - name: content
    image: busybox:1.37
    command: ["sh", "-c", "echo 'Hallo aus dem Init-Container' > /work/index.html"]
    volumeMounts:
    - name: html
      mountPath: /work
  containers:
  - name: nginx
    image: nginx:1.29
    volumeMounts:
    - name: html
      mountPath: /usr/share/nginx/html
  volumes:
  - name: html
    emptyDir: {}
EOF
k apply -f init.yaml
k get pod web
k logs web -c wait-db --tail=2

Expected:

pod/web created
NAME   READY   STATUS     RESTARTS   AGE
web    0/1     Init:0/2   0          8s

warte auf db

Init:0/2: neither of the two init containers has finished. Create the Service that wait-db is waiting for:

k create service clusterip db --tcp=5432:5432
k get pod web
curl -s $(k get pod web -o jsonpath='{.status.podIP}')

Expected:

service/db created
NAME   READY   STATUS    RESTARTS   AGE
web    1/1     Running   0          17s
Hallo aus dem Init-Container

Warning: nslookup db with the short name ends with exit code 1 in busybox 1.37, even when the name resolves – the search domains return NXDOMAIN. So use the full name in the init container.

Save your progress? Sign up for free

Drill 2: Inspect init containers

J='{range .status.initContainerStatuses[*]}'
J+='{.name}{" "}{.state.terminated.reason}{"\n"}{end}'
k get pod web -o jsonpath="$J"
k logs web | head -1

Expected:

wait-db Completed
content Completed
Defaulted container "nginx" out of: nginx, wait-db (init), content (init)

Without -c, kubectl takes the first regular container and tells you so. While an init container is running, k logs web fails with “is waiting to start: PodInitializing” – then target it with -c wait-db.

Save your progress? Sign up for free

Drill 3: Classic sidecar with emptyDir

The application writes to a file, the sidecar prints it to stdout so that k logs can see it:

cat > logger.yaml <<'EOF'
apiVersion: v1
kind: Pod
metadata:
  name: logger
spec:
  containers:
  - name: app
    image: busybox:1.37
    command:
    - sh
    - -c
    - while true; do date '+%T GET /notes 200' >> /log/access.log; sleep 5; done
    volumeMounts:
    - name: logs
      mountPath: /log
  - name: log-shipper
    image: busybox:1.37
    command: ["tail", "-n+1", "-F", "/log/access.log"]
    volumeMounts:
    - name: logs
      mountPath: /log
      readOnly: true
  volumes:
  - name: logs
    emptyDir: {}
EOF
k apply -f logger.yaml
k get pod logger
k logs logger -c log-shipper --tail=2

Expected:

pod/logger created
NAME     READY   STATUS    RESTARTS   AGE
logger   2/2     Running   0          14s
15:48:51 GET /notes 200
15:48:56 GET /notes 200

Save your progress? Sign up for free

Drill 4: Into individual containers

k exec logger -c app -- ls -l /log
k exec logger -c log-shipper -- sh -c 'echo x >> /log/access.log'
k logs logger --all-containers --prefix --tail=1

Expected:

total 4
-rw-r--r--    1 root     root            72 Nov 18 15:48 access.log
sh: can't create /log/access.log: Read-only file system
command terminated with exit code 1
[pod/logger/log-shipper] 15:48:56 GET /notes 200

Same volume, different permissions: readOnly: true applies per volumeMount, i.e. per container. app only writes to the file, which is why --all-containers only returns lines from the sidecar.

Save your progress? Sign up for free

Drill 5: Adapter

An adapter reshapes the application’s output into a format another system expects – here key=value instead of free text. Pods are immutable, so add a third container under containers: in logger.yaml:

  - name: adapter
    image: busybox:1.37
    command:
    - sh
    - -c
    - |
      tail -n+1 -F /log/access.log \
        | awk '{print "time=" $1, "path=" $3, "status=" $4; fflush()}'
    volumeMounts:
    - name: logs
      mountPath: /log
      readOnly: true
k replace -f logger.yaml $now
k get pod logger
k logs logger -c adapter --tail=2
k get pod logger -o jsonpath='{.spec.containers[*].name}{"\n"}'

Expected (shortened):

pod "logger" force deleted from tag04 namespace
pod/logger replaced
NAME     READY   STATUS    RESTARTS   AGE
logger   3/3     Running   0          21s
time=15:51:38 path=/notes status=200
time=15:51:43 path=/notes status=200
app log-shipper adapter

Why fflush()? When awk writes to a pipe instead of a terminal, it buffers – without a flush you see nothing in k logs for a long time.

Save your progress? Sign up for free

Drill 6: Ambassador

An ambassador takes care of the application’s connection to the outside. Here the app only talks to localhost:8001; kubectl proxy in the ambassador handles the API server, token and TLS:

cat > amb.yaml <<'EOF'
apiVersion: v1
kind: Pod
metadata:
  name: client
spec:
  containers:
  - name: app
    image: busybox:1.37
    command:
    - sh
    - -c
    - while true; do wget -qO- -T 2 localhost:8001/version | grep gitV; sleep 10; done
  - name: api-ambassador
    image: registry.k8s.io/kubectl:v1.37.2
    args: ["proxy", "--port=8001"]
EOF
k apply -f amb.yaml
k logs client -c app
k logs client -c api-ambassador

Expected:

pod/client created
wget: can't connect to remote host (127.0.0.1): Connection refused
  "gitVersion": "v1.37.2",
Starting to serve on 127.0.0.1:8001

The first request goes nowhere: both containers start at the same time, and the proxy was not ready yet. Drill 7 solves that.

Save your progress? Sign up for free

Drill 7: Native sidecar

An init container with restartPolicy: Always is a native sidecar: it starts before the regular containers (in list order) but keeps running until the end. Move the ambassador in amb.yaml:

cat > amb.yaml <<'EOF'
apiVersion: v1
kind: Pod
metadata:
  name: client
spec:
  initContainers:
  - name: api-ambassador
    image: registry.k8s.io/kubectl:v1.37.2
    args: ["proxy", "--port=8001"]
    restartPolicy: Always
  containers:
  - name: app
    image: busybox:1.37
    command:
    - sh
    - -c
    - while true; do wget -qO- -T 2 localhost:8001/version | grep gitV; sleep 10; done
EOF
k replace -f amb.yaml $now
k get pod client
k logs client -c app

Expected (shortened):

pod/client replaced
NAME     READY   STATUS    RESTARTS   AGE
client   2/2     Running   0          15s
  "gitVersion": "v1.37.2",
  "gitVersion": "v1.37.2",

READY 2/2 counts the native sidecar too. If you need a real readiness guarantee, give the sidecar a startupProbe – the regular containers only start once it succeeds.

Warning: A tcpSocket probe on port 8001 fails here with “connect: connection refused”: the kubelet checks the pod IP, but kubectl proxy only listens on 127.0.0.1.

Save your progress? Sign up for free

Drill 8: Shutdown order

The real benefit of native sidecars shows with Jobs: once the application is done, the kubelet terminates the sidecars – a classic sidecar would keep the Job running forever.

cat > export.yaml <<'EOF'
apiVersion: batch/v1
kind: Job
metadata:
  name: export
spec:
  template:
    spec:
      restartPolicy: Never
      initContainers:
      - name: prep
        image: busybox:1.37
        command: ["sh", "-c", "echo prep fertig; sleep 2"]
      - name: log-shipper
        image: busybox:1.37
        restartPolicy: Always
        command:
        - sh
        - -c
        - trap 'exit 0' TERM; tail -n+1 -F /log/job.log & wait
        volumeMounts:
        - name: logs
          mountPath: /log
      containers:
      - name: app
        image: busybox:1.37
        command:
        - sh
        - -c
        - for i in 1 2 3; do echo Export Teil $i >> /log/job.log; sleep 3; done
        volumeMounts:
        - name: logs
          mountPath: /log
      volumes:
      - name: logs
        emptyDir: {}
EOF
k apply -f export.yaml
k wait --for=condition=complete job/export --timeout=60s
k get job export

Expected:

job.batch/export created
job.batch/export condition met
NAME     STATUS     COMPLETIONS   DURATION   AGE
export   Complete   1/1           18s        18s

Start and end times of all containers:

S='{.name} {.state.terminated.startedAt} {.state.terminated.finishedAt} '
S="$S"'{.state.terminated.exitCode}{"\n"}'
P=$(k get pods -l job-name=export -o name)
k get $P -o jsonpath="{range .status.initContainerStatuses[*]}$S{end}"
k get $P -o jsonpath="{range .status.containerStatuses[*]}$S{end}"

Expected:

prep 2026-11-18T15:50:17Z 2026-11-18T15:50:19Z 0
log-shipper 2026-11-18T15:50:19Z 2026-11-18T15:50:29Z 0
app 2026-11-18T15:50:20Z 2026-11-18T15:50:29Z 0

Startup: prep to completion, then the sidecar, then app. Shutdown: first app, then the sidecar (multiple sidecars in reverse order). Why the trap? A shell as PID 1 ignores SIGTERM. Without it, the kubelet waits out the 30-second grace period and kills the sidecar with exit code 137 – in the test, the Job took 49 instead of 18 seconds.

Save your progress? Sign up for free

Exercise: Speed Drills

  1. Pod two with container web (nginx:1.29) and container sh (busybox:1.37, sleep 3600). Target time: 90 s
  2. Show the last two lines of the container adapter in pod logger. Target time: 15 s
  3. List the directory /log in the container log-shipper of logger. Target time: 15 s

Solution

k run two --image=nginx:1.29 $do > two.yaml
vim two.yaml

Change the container section like this:

  containers:
  - image: nginx:1.29
    name: web
  - image: busybox:1.37
    name: sh
    command: ["sleep", "3600"]
k apply -f two.yaml
k logs logger -c adapter --tail=2
k exec logger -c log-shipper -- ls /log

Expected (shortened):

pod/two created
time=15:52:28 path=/notes status=200
time=15:52:33 path=/notes status=200
access.log

Save your progress? Sign up for free

Checkpoint

k get pod two -o jsonpath='{.spec.containers[*].name}'

Expected: web sh

k get pod client -o jsonpath='{.spec.initContainers[0].restartPolicy}'

Expected: Always

k get pod logger

Expected: READY 3/3, STATUS Running

Challenge

A Deployment notes with notes-api:1.0 should not receive the API key as an environment variable: an init container writes it to an in-memory emptyDir, and notes-api reads it via NOTES_API_KEY_FILE.

Hint: emptyDir.medium: Memory creates the volume as a tmpfs.

Solution

cat > notes.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: notes
  name: notes
spec:
  replicas: 1
  selector:
    matchLabels:
      app: notes
  template:
    metadata:
      labels:
        app: notes
    spec:
      initContainers:
      - name: write-key
        image: busybox:1.37
        command: ["sh", "-c", "echo lab-key-123 > /secrets/api-key"]
        volumeMounts:
        - name: secrets
          mountPath: /secrets
      containers:
      - image: 172.16.0.126:30500/notes-api:1.0
        name: notes-api
        env:
        - name: NOTES_API_KEY_FILE
          value: /secrets/api-key
        volumeMounts:
        - name: secrets
          mountPath: /secrets
          readOnly: true
      volumes:
      - name: secrets
        emptyDir:
          medium: Memory
EOF
k apply -f notes.yaml
k rollout status deployment/notes
k logs deployment/notes | head -1

Expected (shortened):

deployment.apps/notes created
deployment "notes" successfully rolled out
Defaulted container "notes-api" out of: notes-api, write-key (init)
time=2026-11-18T15:52:39.722Z level=INFO msg="api key loaded" source=file

source=file confirms the file route. A plaintext key in the init container is, of course, only a lab value – on Day 6 it comes from a Secret.

Clean up:

k config set-context --current --namespace=default
k delete ns tag04

Save your progress? Sign up for free

Exam Tip

  • Native sidecars go under initContainers with restartPolicy: Always – not under containers. In the docs under “Sidecar Containers” (Concepts → Workloads → Pods).
  • With multiple containers, always use -c <name> with k logs and k exec; container names come from k get pod <p> -o jsonpath='{.spec.containers[*].name}'.
  • You cannot add a container to a running pod: change the file and run k replace -f … $now. For Deployments, k apply is enough.

Enterprise Note

  • Service meshes (Istio, Linkerd) inject proxy sidecars automatically via an admission webhook; Istio now uses native sidecars for this.
  • Log sidecars are expensive (one container per pod). More common is a DaemonSet agent (Fluent Bit, Vector) that reads the nodes’ container logs.
  • Every sidecar needs requests and limits – they count toward the pod and therefore toward quotas and scheduling (Day 7).