Day 13: Minimizing Images, SBOM and Trivy
Goal: You build notes-api as a minimal image without a shell, find vulnerabilities and misconfigurations with Trivy and generate an SBOM. · Duration: approx. 75 min · Prerequisite: Day 9
Briefing
- Curriculum domain “CKS: Supply Chain Security”: minimize base images, understand the supply chain (SBOM), scan workloads and images for vulnerabilities.
- Every package in the image is attack surface – and work for the scanner. What isn’t in there can neither be attacked nor reported.
- Trivy scans images (packages, Go modules), configuration (Dockerfile, manifests) and SBOMs. Exit codes turn that into a CI gate.
- Result for later days:
notes-api:1.0andnotes-api:1.0-minin the registry172.16.0.126:30500.
golang:1.27-alpine (267 MB) --build--> notes-api (Go binary, static)
|
+-------------------------------+-------------------------+
v v v
alpine:3.23 -> notes-api:1.0 distroless/static -> 1.0-min scratch
16 packages, shell 6 packages, no shell 0 packages
Everything on cp1:
mkdir -p ~/cks/tag13 && cd ~/cks/tag13
git clone https://github.com/thomaszachmann/cks-zum-anfassen
k create ns tag13
k config set-context --current --namespace=tag13Drill 1: Comparing base images
podman pull docker.io/library/alpine:3.23
podman pull gcr.io/distroless/static-debian13:nonroot
podman images --format '{{.Repository}}:{{.Tag}} {{.Size}}' \
| grep -E 'alpine:3.23|distroless'
podman run --rm docker.io/library/alpine:3.23 sh -c 'id -u; apk info | wc -l'
podman run --rm gcr.io/distroless/static-debian13:nonroot sh
podman inspect gcr.io/distroless/static-debian13:nonroot --format '{{.Config.User}}'Expected:
docker.io/library/alpine:3.23 8.72 MB
gcr.io/distroless/static-debian13:nonroot 3.44 MB
0
16
Error: crun: executable file `sh` not found in $PATH: No such file or directory: ...
65532
alpine brings a package manager and a BusyBox shell and runs as root.
The distroless image static contains only CA certificates,
time zones and /etc/passwd – no shell, no package manager,
user 65532. That’s enough for statically linked Go binaries.
Save your progress? Sign up for free
Drill 2: Building notes-api on alpine and pushing it
The starting point is the familiar multi-stage Dockerfile:
cd ~/cks/tag13/cks-zum-anfassen/notes-api
cat > Dockerfile <<'EOF'
FROM docker.io/library/golang:1.27-alpine AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
ARG VERSION=dev
RUN CGO_ENABLED=0 go build -ldflags="-s -w -X main.version=${VERSION}" \
-o /out/notes-api .
FROM docker.io/library/alpine:3.23
COPY --from=build /out/notes-api /usr/local/bin/notes-api
USER 65532:65532
EXPOSE 8080
ENTRYPOINT ["notes-api"]
EOF
podman build -t notes-api:1.0 --build-arg VERSION=1.0 .
podman tag notes-api:1.0 172.16.0.126:30500/notes-api:1.0
podman push --tls-verify=false 172.16.0.126:30500/notes-api:1.0Expected (abridged):
[2/2] COMMIT notes-api:1.0
Successfully tagged localhost/notes-api:1.0
...
Writing manifest to image destination
CGO_ENABLED=0 produces a static binary – that’s the only
reason it later runs without libc in distroless or
scratch.
Save your progress? Sign up for free
Drill 3: Minimized – distroless, non-root, no shell
Only the second stage changes:
D=gcr.io/distroless/static-debian13:nonroot
sed -e "s|^FROM docker.io/library/alpine:3.23|FROM $D|" \
-e 's|/usr/local/bin/notes-api|/notes-api|' \
-e 's|^ENTRYPOINT .*|ENTRYPOINT ["/notes-api"]|' Dockerfile > Dockerfile.min
tail -5 Dockerfile.min
podman build -t notes-api:1.0-min --build-arg VERSION=1.0 -f Dockerfile.min .
podman images --format '{{.Repository}}:{{.Tag}} {{.Size}}' | grep localhost/notesExpected (abridged):
FROM gcr.io/distroless/static-debian13:nonroot
COPY --from=build /out/notes-api /notes-api
USER 65532:65532
EXPOSE 8080
ENTRYPOINT ["/notes-api"]
...
Successfully tagged localhost/notes-api:1.0-min
localhost/notes-api:1.0-min 15.2 MB
localhost/notes-api:1.0 20.5 MB
The USER line is redundant (the base image already sets
65532), but it makes the intent visible in the Dockerfile. Does it run –
and what’s missing?
podman run -d --name min -p 8081:8080 -e NOTES_API_KEY=lab-key-123 notes-api:1.0-min
curl -s localhost:8081/healthz; echo
podman exec min id
podman run --rm --entrypoint sh notes-api:1.0-min
podman rm -f minExpected (abridged):
ok
Error: crun: executable file `id` not found in $PATH: No such file or directory: ...
Error: crun: executable file `sh` not found in $PATH: No such file or directory: ...
min
lab-key-123 is a pure lab value. Without a shell, an
attacker who gains code execution cannot load sh,
wget or apk add. Now push:
podman tag notes-api:1.0-min 172.16.0.126:30500/notes-api:1.0-min
podman push --tls-verify=false 172.16.0.126:30500/notes-api:1.0-min
curl -s http://172.16.0.126:30500/v2/notes-api/tags/listExpected:
{"name":"notes-api","tags":["1.0","1.0-min"]}
And scratch? Even smaller, but without CA certificates.
notes-api calls an HTTPS address under /upstream:
sed "s|^FROM $D|FROM scratch|" Dockerfile.min > Dockerfile.scratch
podman build -q -t notes-api:1.0-scratch --build-arg VERSION=1.0 -f Dockerfile.scratch .
podman run -d --name sc -p 8082:8080 -e NOTES_API_KEY=lab-key-123 \
-e NOTES_UPSTREAM=https://kubernetes.io notes-api:1.0-scratch
curl -s localhost:8082/upstream; echo
podman rm -f sc
podman images --format '{{.Tag}} {{.Size}}' localhost/notes-api:1.0-scratchExpected (abridged):
{"error":"Get \"https://kubernetes.io\": tls: failed to verify certificate: x509:
certificate signed by unknown authority"}
sc
1.0-scratch 11.7 MB
With 1.0-min, the same call returns
{"status":200,...}. distroless is the better compromise:
almost as small as scratch, but with certificates, time
zones and a user.
Save your progress? Sign up for free
Drill 4: Installing Trivy and the first scan
Trivy as a binary from the project, with a checksum (as of 2026-11-18: v0.76.2):
cd /tmp
V=0.76.2
U=https://github.com/aquasecurity/trivy/releases/download/v$V
ARCH=$(dpkg --print-architecture | sed 's/amd64/64bit/;s/arm64/ARM64/')
curl -sSLO $U/trivy_${V}_Linux-${ARCH}.tar.gz
curl -sSLO $U/trivy_${V}_checksums.txt
grep " trivy_${V}_Linux-${ARCH}.tar.gz$" trivy_${V}_checksums.txt | sha256sum -c
tar xzf trivy_${V}_Linux-${ARCH}.tar.gz trivy
sudo install -m 755 trivy /usr/local/bin/trivy
trivy --versionExpected:
trivy_0.76.2_Linux-64bit.tar.gz: OK
Version: 0.76.2
The first scan downloads the vulnerability database (takes a moment).
--scanners vuln turns off the slow secret search,
-q the log lines:
cd ~/cks/tag13
R=172.16.0.126:30500/notes-api
trivy image -q --scanners vuln $R:1.0
trivy image -q --scanners vuln $R:1.0-min | grep -E 'debian|gobinary'Expected (abridged):
│ 172.16.0.126:30500/notes-api:1.0 (alpine 3.23.8) │ alpine │ 1 │
│ usr/local/bin/notes-api │ gobinary │ 2 │
...
172.16.0.126:30500/notes-api:1.0 (alpine 3.23.8)
Total: 1 (UNKNOWN: 0, LOW: 0, MEDIUM: 1, HIGH: 0, CRITICAL: 0)
...
usr/local/bin/notes-api (gobinary)
Total: 2 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 2, CRITICAL: 0)
...
│ 172.16.0.126:30500/notes-api:1.0-min (debian 13.7) │ debian │ 0 │
│ notes-api │ gobinary │ 2 │
notes-api (gobinary)
The base image is clean now; what remains are the Go modules in the binary – no base image makes those go away (challenge).
And the HTTP registry? Trivy asks docker, containerd, podman in turn
and finally the registry (remote). For addresses from
private networks (RFC 1918, e.g. 172.16.0.0/12), Trivy’s registry
library falls back to HTTP by itself. With a name instead of the IP,
that doesn’t work:
trivy image -q --scanners vuln 172.16.0.126.nip.io:30500/notes-api:1.0-min 2>&1 \
| grep 'remote error'
trivy image -q --scanners vuln --insecure \
172.16.0.126.nip.io:30500/notes-api:1.0-min | grep ^TotalExpected (abridged):
* remote error: Get "https://172.16.0.126.nip.io:30500/v2/": http: server gave
HTTP response to HTTPS client
Total: 2 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 2, CRITICAL: 0)
nip.io publicly resolves
172.16.0.126.nip.io to 172.16.0.126.
--insecure allows HTTP and unverified TLS – in production
only for test registries.
Save your progress? Sign up for free
Drill 5: Filtering – severity, unfixed, exit code
An older image shows why filters are necessary (the first scan takes a while):
I=docker.io/library/nginx:1.25
trivy image -q --scanners vuln $I | grep ^Total
trivy image -q --scanners vuln --severity HIGH,CRITICAL $I | grep ^Total
trivy image -q --scanners vuln --severity HIGH,CRITICAL --ignore-unfixed $I \
| grep ^TotalExpected:
Total: 945 (UNKNOWN: 74, LOW: 267, MEDIUM: 347, HIGH: 236, CRITICAL: 21)
Total: 257 (HIGH: 236, CRITICAL: 21)
Total: 117 (HIGH: 99, CRITICAL: 18)
--ignore-unfixed hides vulnerabilities for which no
update exists yet – you can only act on the rest. For CI, the exit code
is what counts:
trivy image -q --scanners vuln --severity HIGH,CRITICAL --exit-code 1 $R:1.0-min \
> /dev/null
echo $?
trivy image -q --scanners vuln --severity CRITICAL --exit-code 1 $R:1.0-min > /dev/null
echo $?Expected:
1
0
Machine-readable details via JSON and jq:
trivy image -q --scanners vuln --severity HIGH,CRITICAL -f json $R:1.0 \
| jq -r '.Results[] | .Target as $t | .Vulnerabilities[]?
| [$t, .PkgName, .VulnerabilityID, .FixedVersion] | @tsv'Expected:
usr/local/bin/notes-api golang.org/x/text CVE-2026-56851 0.41.0
usr/local/bin/notes-api golang.org/x/text CVE-2026-56852 0.39.0
Warning: No findings doesn’t mean clean.
trivy image busybox:1.37reports[report] Supported files for scanner(s) not foundandNot scanned– BusyBox has no package database that Trivy could read.
Save your progress? Sign up for free
Drill 6: Scanning configuration – Dockerfile and manifest
trivy config checks files against built-in rules
(DS-… for Dockerfiles, KSV-… for
Kubernetes):
trivy config -q ~/cks/tag13/cks-zum-anfassen/notes-api | grep -E '^Dockerfile|^DS'
grep -v '^USER' cks-zum-anfassen/notes-api/Dockerfile > Dockerfile.root
mkdir -p bad && mv Dockerfile.root bad/Dockerfile
trivy config -q bad | grep -E '^DS'Expected (abridged):
Dockerfile (dockerfile)
DS-0026 (LOW): Add HEALTHCHECK instruction in your Dockerfile
Dockerfile.min (dockerfile)
DS-0026 (LOW): Add HEALTHCHECK instruction in your Dockerfile
...
DS-0002 (HIGH): Specify at least 1 USER command in Dockerfile with non-root user as ...
DS-0026 (LOW): Add HEALTHCHECK instruction in your Dockerfile
Now a Deployment as k create generates it:
k create deployment notes --image=172.16.0.126:30500/notes-api:1.0 --port=8080 \
$do > notes.yaml
trivy config -q notes.yaml | grep -E '^Failures'
trivy config -q --severity HIGH,CRITICAL --exit-code 1 notes.yaml | grep '^KSV'
echo ${PIPESTATUS[0]}Expected (abridged):
Failures: 18 (UNKNOWN: 0, LOW: 11, MEDIUM: 4, HIGH: 3, CRITICAL: 0)
KSV-0014 (HIGH): Container 'notes-api' of Deployment 'notes' should set
'securityContext.readOnlyRootFilesystem' to true
KSV-0118 (HIGH): container notes in default namespace is using the default security ...
KSV-0118 (HIGH): deployment notes in default namespace is using the default security ...
1
It says default namespace because $do
doesn’t write metadata.namespace. A hardened version with
the minimal image (SecurityContext as on Day 9):
cat > notes-min.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
name: notes
spec:
replicas: 1
selector:
matchLabels:
app: notes
template:
metadata:
labels:
app: notes
spec:
securityContext:
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
containers:
- name: notes-api
image: 172.16.0.126:30500/notes-api:1.0-min
env:
- name: NOTES_API_KEY
value: lab-key-123
ports:
- containerPort: 8080
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: ["ALL"]
EOF
trivy config -q --severity HIGH,CRITICAL --exit-code 1 notes-min.yaml | grep Misconf -A3
echo ${PIPESTATUS[0]}
trivy config -q notes-min.yaml | grep -E '^Failures'Expected (abridged):
│ Target │ Type │ Misconfigurations │
├────────────────┼────────────┼───────────────────┤
│ notes-min.yaml │ kubernetes │ 0 │
0
Failures: 8 (UNKNOWN: 0, LOW: 7, MEDIUM: 1, HIGH: 0, CRITICAL: 0)
What remains: resource limits, UID > 10000, the namespace and
KSV-0125 (“untrusted registry”) – Day 14 shows how to
enforce allowed registries in the cluster
(ValidatingAdmissionPolicy).
Save your progress? Sign up for free
Drill 7: Generating and scanning an SBOM
An SBOM (Software Bill of Materials) lists every component in the image. Trivy writes CycloneDX and SPDX:
trivy image -q -f cyclonedx -o notes-1.0.cdx.json $R:1.0
trivy image -q -f spdx-json -o notes-1.0.spdx.json $R:1.0
jq -r '.bomFormat, .specVersion, (.components | length)' notes-1.0.cdx.json
jq -r '.spdxVersion, (.packages | length)' notes-1.0.spdx.json
jq -r '.components[] | [.name, .version] | @tsv' notes-1.0.cdx.json \
| grep -E 'zlib|busybox|x/text'Expected:
CycloneDX
1.7
27
SPDX-2.3
28
busybox-binsh 1.37.0-r30
busybox 1.37.0-r30
zlib 1.3.2-r1
golang.org/x/text v0.29.0
The SBOM itself is now a scan target – without pulling the image again. If a new CVE shows up tomorrow, a new run against the saved file is enough:
trivy sbom -q notes-1.0.cdx.json | grep ^Total
trivy sbom -q --severity HIGH,CRITICAL notes-1.0.spdx.json | grep ^TotalExpected:
Total: 1 (UNKNOWN: 0, LOW: 0, MEDIUM: 1, HIGH: 0, CRITICAL: 0)
Total: 2 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 2, CRITICAL: 0)
Total: 2 (HIGH: 2, CRITICAL: 0)
The same findings as in the image scan: one Total line
per target (alpine packages, Go binary); with the filter, only the Go
binary remains.
An alternative from the Kubernetes project: bom
(kubernetes-sigs/bom, writes SPDX), here v0.8.0 with a checksum:
cd /tmp
U=https://github.com/kubernetes-sigs/bom/releases/download/v0.8.0
ARCH=$(dpkg --print-architecture)
curl -sSLO $U/bom-${ARCH}-linux
curl -sSLO $U/checksums.txt
grep " bom-${ARCH}-linux$" checksums.txt | sha256sum -c
sudo install -m 755 bom-${ARCH}-linux /usr/local/bin/bom
cd ~/cks/tag13
bom generate --image $R:1.0-min --format json --output notes-min.bom.spdx.json
jq -r '.packages[] | select(.versionInfo != null) | [.name, .versionInfo] | @tsv' \
notes-min.bom.spdx.json | grep -vE '^sha256|^172'
trivy sbom -q notes-min.bom.spdx.json | grep ^TotalExpected (abridged):
bom-amd64-linux: OK
level=info msg="bom v0.8.0: Generating SPDX Bill of Materials"
...
base-files 13.8+deb13u7
ca-certificates 20250419+deb13u1
media-types 13.0.0
netbase 6.5
...
golang.org/x/text v0.29.0
stdlib 1.27.4
tzdata 2026f-0+deb13u1
tzdata-legacy 2026f-0+deb13u1
Total: 2 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 2, CRITICAL: 0)
Six Debian packages instead of 16 alpine packages – and none of them
is a shell. bom, too, pulls from the HTTP registry without
trouble (same library as Trivy).
Save your progress? Sign up for free
Drill 8: Test deploy without a shell
k apply -f notes-min.yaml
k rollout status deploy notes
k expose deploy notes --port=8080
k run probe --image=busybox:1.37 --restart=Never -- wget -qO- http://notes:8080/healthz
k exec deploy/notes -- shExpected (abridged):
deployment "notes" successfully rolled out
service/notes exposed
pod/probe created
error: Internal error occurred: ... exec failed: unable to start container process:
exec: "sh": executable file not found in $PATH
k logs probe shows ok. For troubleshooting,
you bring the shell along as an ephemeral container. Because of
runAsNonRoot it needs a UID;
--profile=restricted sets the remaining fields to
match:
printf 'securityContext:\n runAsUser: 65532\n' > dbg.yaml
P=$(k get pod -l app=notes -o name)
k debug $P --image=busybox:1.37 --target=notes-api --profile=restricted \
--custom=dbg.yaml -c dbg -- sleep 300Wait a few seconds until the debug container is running – otherwise
ps shows nothing yet:
k exec $P -c dbg -- psExpected (k debug and
ps):
Targeting container "notes-api". If you don't see processes from this container it
may be because the container runtime doesn't support this feature.
PID USER TIME COMMAND
1 65532 0:00 /notes-api
20 65532 0:00 sleep 300
38 65532 0:00 ps
Why? The shell exists only while you’re debugging – it is not part of the image that every copy of the Pod ships.
Save your progress? Sign up for free
Exercise: Speed Drills
- Count the CRITICAL vulnerabilities in
nginx:1.25that have a fix. Target time: 30 s - Scan all images of the Pods in
tag13for HIGH and CRITICAL. Target time: 90 s - According to the CycloneDX SBOM, which version of
golang.org/x/textis innotes-api:1.0-min? Target time: 60 s - List only the IDs of the HIGH findings from
trivy configfornotes.yaml. Target time: 45 s
Solution
trivy image -q --scanners vuln --severity CRITICAL --ignore-unfixed \
docker.io/library/nginx:1.25 | grep ^Total
for i in $(k get pods -o jsonpath='{.items[*].spec.containers[*].image}' \
| tr ' ' '\n' | sort -u); do
echo "== $i"; trivy image -q --scanners vuln --severity HIGH,CRITICAL $i | grep ^Total
done
trivy image -q -f cyclonedx $R:1.0-min \
| jq -r '.components[] | select(.name=="golang.org/x/text") | .version'
trivy config -q --severity HIGH notes.yaml | grep -oE '^KSV-[0-9]+' | sort -uExpected:
Total: 18 (CRITICAL: 18)
== 172.16.0.126:30500/notes-api:1.0-min
Total: 2 (HIGH: 2, CRITICAL: 0)
== busybox:1.37
v0.29.0
KSV-0014
KSV-0118
For busybox:1.37 the Total line is missing
– “not scanned”, not “clean” (Drill 5).
Save your progress? Sign up for free
Checkpoint
trivy --versionExpected: first line Version: 0.76.2,
below it the state of the databases.
curl -s http://172.16.0.126:30500/v2/notes-api/tags/listExpected:
{"name":"notes-api","tags":["1.0","1.0-min"]} – both images
stay in the registry; Day 14 and later days use them. If the registry
Pod restarts, repeat Drills 2 and 3 (push).
k exec deploy/notes -- shExpected:
exec: "sh": executable file not found in $PATH
trivy config -q --severity HIGH,CRITICAL --exit-code 1 notes-min.yaml > /dev/null
echo $?Expected: 0
jq -r .bomFormat notes-1.0.cdx.jsonExpected: CycloneDX
Challenge
The two HIGH vulnerabilities in golang.org/x/text are in
the binary, not in the base image. Build
notes-api:1.0.1-min without these vulnerabilities and prove
it with Trivy – without pushing the image to the registry.
Hint: podman save writes an image as a
tar archive, trivy image --input scans it.
Solution
There is no Go installed on cp1 – go get
runs in the build image:
cd ~/cks/tag13 && cp -r cks-zum-anfassen/notes-api notes-fix && cd notes-fix
podman run --rm -v $PWD:/src:Z -w /src docker.io/library/golang:1.27-alpine \
go get golang.org/x/[email protected]
grep x/text go.mod
podman build -q -t notes-api:1.0.1-min --build-arg VERSION=1.0.1 -f Dockerfile.min .
podman save -o ../fix.tar notes-api:1.0.1-min
trivy image -q --scanners vuln --severity HIGH,CRITICAL --exit-code 1 \
--input ../fix.tar | grep -E 'debian|gobinary'
echo ${PIPESTATUS[0]}Expected (abridged):
go: upgraded golang.org/x/sync v0.17.0 => v0.22.0
go: upgraded golang.org/x/text v0.29.0 => v0.41.0
golang.org/x/text v0.41.0 // indirect
...
│ ../fix.tar (debian 13.7) │ debian │ 0 │
│ notes-api │ gobinary │ 0 │
0
The fix belongs in the repository (go.mod), not in the
image: a new base image wouldn’t have changed anything.
Clean up (the two images in the registry stay):
k delete ns tag13
k config set-context --current --namespace=default
podman rmi notes-api:1.0-scratch notes-api:1.0.1-min
rm -rf ~/cks/tag13/fix.tar ~/cks/tag13/notes-fix ~/cks/tag13/badSave your progress? Sign up for free
Exam Tip
- Typical task: “Find all images with CRITICAL vulnerabilities and
delete the affected Pods” –
trivy image --severity CRITICALper image;grep ^Totalkeeps the output short.-qand--scanners vulnsave time. - Read the task carefully:
--ignore-unfixed, a specific severity or an SBOM format (cyclonedx,spdx-json) changes the result. - Docs: trivy.dev (“Vulnerability”, “Misconfiguration”, “SBOM”), kubernetes.io “Debug Running Pods” (ephemeral containers), github.com/kubernetes-sigs/bom.
Enterprise Note
- Scans run in the pipeline (exit code as a gate) and continuously in the registry (Harbor with Trivy) or in the cluster (Trivy Operator) – new CVEs hit old images too.
- SBOMs are generated at build time, signed and stored as an attestation next to the image (cosign, Day 14); tools like Dependency-Track evaluate them centrally.
- Base images come from a curated, mirrored catalog (distroless, Chainguard, UBI Micro) with a fixed update cadence instead of arbitrary Docker Hub tags.
- The scanner is part of the supply chain too: besides the checksum
file (
trivy_<version>_checksums.txt), Trivy releases carry a Sigstore signature (*.sigstore.json); in production both are verified and the version is pinned. - Exceptions (
.trivyignore) only with a justification, an expiry date and a review – otherwise the gate becomes a formality.