Skip to content
Hands-on IT IT Drills

Day 15: Docker tools – docker, Buildx, Compose, Colima, Scaffdog

Goal: You point the Docker CLI at Docker or Podman on purpose with contexts, build for two platforms into your registry with Buildx, start notes with Docker Compose, know Colima as the Docker foundation on the Mac and generate a project scaffold with Scaffdog. · Duration: approx. 120 min · Prerequisite: Day 13 (Docker CE), Day 7

Briefing

  • The Docker CLI is just a client of the Docker Engine API. Where it talks to is determined by DOCKER_HOST, --context and the active context – in that order.
  • Buildx is the CLI’s build plugin and talks to BuildKit. A dedicated builder (BuildKit in a container) can build for several platforms and push directly to a registry; Bake builds several images from one file.
  • Docker Compose (plugin docker compose) starts several containers from a compose.yaml – the original behind podman compose from Day 4.
  • Colima starts a small Linux VM with Docker Engine on the Mac (via Lima) and sets up a Docker context for it.
  • Scaffdog generates files from Markdown templates – for example a new service scaffold with a Dockerfile that passes hadolint right away.
 docker ─┬─ DOCKER_HOST / --context ─► Docker CE  │  Podman socket  │  Colima VM
         ├─ buildx ─► builder ct-lab (moby/buildkit) ─► registry.ct.lab:5000
         └─ compose ─► compose.yaml ─► db, migrate, api, web
 scaffdog (Node in a container) ─► notes-report/{Dockerfile,main.go,…}

Drill 1: docker – the CLI and its contexts

Profile docker (CLI)
Task Client for the Docker Engine API: build, run, manage
Special feature Talks to any API endpoint; extensions as CLI plugins
Project github.com/docker/cli (Apache-2.0), docs: docs.docker.com/reference/cli
Version in this book 29.9.2 (package docker-ce-cli, installed since Day 13)

Since Day 4, your ~/.bashrc has a DOCKER_HOST pointing to the Podman socket. In a new shell, docker therefore talks to Podman:

source ~/ct-lab/setup/env.sh
mkdir -p ~/ct-lab/tag15 && cd ~/ct-lab/tag15
cp -R "$LAB/code/notes/." .
echo "$DOCKER_HOST"
docker version --format '{{.Client.Version}} / {{.Server.Version}}'
docker context ls

Expected:

unix:///run/user/1000/podman/podman.sock
29.9.2 / 5.7.0
NAME        DESCRIPTION                               DOCKER ENDPOINT                …
default *   Current DOCKER_HOST based configuration   unix:///run/user/1000/podman/pod…
Warning: DOCKER_HOST environment variable overrides the active context. To use a contex…

Client 29.9.2, server 5.7.0 – that’s Podman. Cleaner than an environment variable is a context: a named endpoint you choose per command with --context.

docker context create podman --description "Podman rootless" \
  --docker "host=unix://$XDG_RUNTIME_DIR/podman/podman.sock"
unset DOCKER_HOST
docker context ls
docker version --format '{{.Client.Version}} / {{.Server.Version}}'
docker --context podman version --format '{{.Server.Version}}'

Expected:

podman
Successfully created context "podman"
NAME        DESCRIPTION                               DOCKER ENDPOINT                …
default *   Current DOCKER_HOST based configuration   unix:///var/run/docker.sock
podman      Podman rootless                           unix:///run/user/1000/podman/pod…
29.9.2 / 29.9.2
5.7.0

As long as DOCKER_HOST is set, it wins against every context – which is why the CLI already warns on docker context ls. Without it, default is Docker CE again. Contexts live in ~/.docker/contexts; with docker context use podman you switch permanently. In this shell we stay with the default and name Podman explicitly.

Save your progress? Sign up for free

Drill 2: docker – build, run, inspect

docker build -q -t ct-lab/notes-api:0.1.0 --build-arg VERSION=0.1.0 notes-api
docker image ls ct-lab/notes-api --format '{{.Repository}}:{{.Tag}} {{.Size}}'
docker run -d --name ct15-api -p 127.0.0.1:8095:8080 \
  -e NOTES_API_KEY=lab-api-key-123 ct-lab/notes-api:0.1.0
sleep 6
curl -s localhost:8095/version; echo
docker ps --filter name=ct15-api --format '{{.Names}}\t{{.Status}}\t{{.Ports}}'
docker exec ct15-api /notes-api healthcheck; echo "Exit: $?"
docker logs ct15-api 2>&1 | tail -1
docker stats --no-stream --format '{{.Name}} {{.MemUsage}}' ct15-api
docker rm -f ct15-api

Expected (shortened; IDs and times will differ for you):

sha256:122f0da849f2…
ct-lab/notes-api:0.1.0 16.4MB
71dbb13839c2…
0.1.0
ct15-api    Up 6 seconds (healthy)  127.0.0.1:8095->8080/tcp
Exit: 0
time=… level=INFO msg=listening addr=:8080 version=0.1.0 pid=1
ct15-api 1.977MiB / 3.911GiB
ct15-api

You know these commands from podman – Podman copied them from Docker. Differences: Docker CE has a daemon running as root, images are named without the localhost/ prefix, and the image ends up in Docker’s own storage, not in Podman’s. Since Docker 23, docker build is already Buildx with the built-in builder default.

Save your progress? Sign up for free

Drill 3: Buildx – a builder of your own

Profile Docker Buildx
Task Builds with BuildKit: multiple platforms, cache, attestations, Bake
Special feature Builder drivers docker, docker-container, kubernetes, remote
Project github.com/docker/buildx (Apache-2.0)
Version in this book v0.38.1 (package docker-buildx-plugin), BuildKit v0.34.1
docker buildx version
docker buildx ls

Expected:

github.com/docker/buildx v0.38.1 0dbe87f36ed472fe36b6796c3f70b4210a731fc5
NAME/NODE     DRIVER/ENDPOINT   STATUS     BUILDKIT   PLATFORMS
default*      docker
 \_ default    \_ default       running    v0.34.0    linux/amd64 (+3)
podman        docker-container
 \_ podman     \_ podman        inactive

Buildx shows a builder for every context – including podman. The builder default lives in the Docker daemon and can only build into Docker’s storage. A builder of your own with the docker-container driver starts BuildKit as a container. It should trust your lab CA and – because of 127.0.0.1 registry.ct.lab – use the host’s network:

cat > buildkitd.toml <<EOF
# tag15/buildkitd.toml – state at end of Day 15: BuildKit trusts the lab CA
[registry."$REG"]
  ca = ["$LAB/registry/certs/ca.crt"]
EOF
docker buildx create --name ct-lab --driver docker-container \
  --driver-opt image=docker.io/moby/buildkit:v0.34.1 --driver-opt network=host \
  --buildkitd-config buildkitd.toml --bootstrap 2>&1 | tail -2
docker buildx inspect ct-lab | grep -E '^Driver|^Status|^BuildKit version|^Platforms'

Expected:

#1 DONE 23.8s
ct-lab
Driver:        docker-container
Driver Options:        image="docker.io/moby/buildkit:v0.34.1" network="host"
Status:                running
BuildKit version:      v0.34.1
Platforms:             linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386

When creating the builder, Buildx copies the CA file from buildkitd.toml into the container. Without --use, default remains the active builder; you select the new one with --builder ct-lab.

Save your progress? Sign up for free

Drill 4: Buildx – two platforms into your own registry

For --push, Buildx needs a login – docker login does that and stores it in ~/.docker/config.json, where crane and cosign have also been reading it since Day 7:

echo "$REG_PASS" | docker login "$REG" -u "$REG_USER" --password-stdin 2>&1 | tail -1
docker buildx build --builder ct-lab --platform linux/amd64,linux/arm64 \
  --build-arg VERSION=0.1.0 -t "$REG/lab/notes-api:0.1.0" --push notes-api 2>&1 \
  | grep -E 'pushing manifest|ERROR' | tail -1 | cut -c1-88
docker buildx imagetools inspect "$REG/lab/notes-api:0.1.0" \
  | grep -E '^MediaType|Platform'

Expected (shortened):

Login Succeeded
#16 pushing manifest for registry.ct.lab:5000/lab/notes-api:0.1.0@sha256:769ddc950…
MediaType: application/vnd.oci.image.index.v1+json
  Platform:    linux/amd64
  Platform:    linux/arm64
  Platform:    unknown/unknown
  Platform:    unknown/unknown
  • The builder has no arm64 platform and doesn’t need one: the Dockerfile builds with FROM --platform=$BUILDPLATFORM on amd64 and lets Go cross-compile with GOARCH=$TARGETARCH. A RUN on arm64, by contrast, would need QEMU (binfmt).
  • Under the tag sits an index with both images. The unknown/unknown entries are the provenance attestations that Buildx pushes along by default.
  • With the built-in builder default, this wouldn’t work into your registry: you would have to give the Docker daemon the CA separately.

Save your progress? Sign up for free

Drill 5: Buildx – Bake for all three images

Bake reads the builds from a file (HCL, JSON or compose.yaml) and builds them in parallel. The file is in the companion code:

cp "$LAB/code/tag15/notes/docker-bake.hcl" .
grep -E '^(variable|group|target)|platforms' docker-bake.hcl
docker buildx bake --print 2>/dev/null \
  | jq -c '.target | to_entries[] | {t: .key, p: .value.platforms}'

Expected:

variable "REG" {
variable "VERSION" {
group "default" {
target "_common" {
  platforms = ["linux/amd64"]
target "api" {
  platforms = ["linux/amd64", "linux/arm64"]   # Go kompiliert quer, ohne QEMU
target "worker" {
target "web" {
{"t":"api","p":["linux/amd64","linux/arm64"]}
{"t":"web","p":["linux/amd64"]}
{"t":"worker","p":["linux/amd64"]}

_common is an abstract target that the others inherit via inherits. Worker (Python) and web (Node) run RUN steps on the target platform and therefore stay on amd64. --print shows the resolved result without building.

docker buildx bake --builder ct-lab --push --progress=quiet && echo gepusht
for r in notes-api notes-worker notes-web; do echo "$r: $(crane ls "$REG/lab/$r")"; done

Expected (gepusht = pushed):

gepusht
notes-api: 0.1.0
notes-worker: 0.1.0
notes-web: 0.1.0

Save your progress? Sign up for free

Drill 6: Docker Compose – notes with one command

Profile Docker Compose
Task Multiple containers, networks and volumes from one compose.yaml
Special feature CLI plugin docker compose; builds with Buildx, waits for health checks
Project github.com/docker/compose (Apache-2.0), specification: compose-spec.io
Version in this book v5.6.1 (package docker-compose-plugin)

The depth is in “Docker Compose hands-on”. Here, the interplay is what counts: Compose builds the images from source, waits for the database, runs the migration and only starts web once api is healthy.

cp "$LAB/code/tag15/notes/compose.yaml" .
grep -E '^  [a-z]+:|apparmor|condition' compose.yaml
docker compose version
docker compose up -d --build --wait 2>&1 | grep -E 'Built|Healthy' | head -4
docker compose ps -a --format '{{.Service}}\t{{.Status}}'

Expected:

  db:
      - apparmor=unconfined    # Ubuntu 26.04: docker-default verbietet gosu Signale
  migrate:
        condition: service_healthy
  api:
        condition: service_completed_successfully
  web:
        condition: service_healthy      # HEALTHCHECK aus dem Dockerfile
  pgdata:
Docker Compose version v5.6.1
 Image ct-lab/notes-api:0.1.0 Built
 Image ct-lab/notes-web:0.1.0 Built
 Container ct15-db-1 Healthy
 Container ct15-api-1 Healthy
api Up 3 seconds (healthy)
db  Up 14 seconds (healthy)
migrate Exited (0) 3 seconds ago
web Up 1 second

Warning: Without apparmor=unconfined, Postgres hangs at startup with full CPU and never becomes healthy. The AppArmor profile docker-default forbids the entrypoint helper gosu and Postgres from sending signals to their own processes – the same trap as with nerdctl-default on Day 6 (sudo journalctl -k | grep DENIED).

curl -s -X POST -H 'X-API-Key: lab-api-key-123' -H 'Content-Type: application/json' \
  -d '{"text":"aus Docker Compose"}' localhost:8096/notes; echo
until curl -fs -o /dev/null localhost:3006/healthz; do sleep 1; done
curl -s localhost:3006/api/notes | jq -r '.[].text'
docker compose down 2>&1 | tail -1
docker compose up -d --wait >/dev/null 2>&1
curl -s localhost:8096/notes | jq -r '.[].text'
docker compose down 2>&1 | tail -1

Expected:

{"id":"34d9711a-…","text":"aus Docker Compose"}
aus Docker Compose
 Network ct15_default Removed
aus Docker Compose
 Network ct15_default Removed

down removes containers and network, the volume ct15_pgdata stays – the note survives. Only down -v deletes the data too. --wait waits until all services are running and healthy; web has no health check and counts as ready once started, hence the loop before the curl.

Save your progress? Sign up for free

Drill 7: Colima – Docker on the Mac

Profile Colima
Task Container runtime on macOS (and Linux) in a Lima VM
Special feature Docker or containerd, optionally k3s; creates the Docker context colima
Project github.com/abiosoft/colima (MIT)
Version in this book v0.10.3, Lima 2.2.1

Docker Engine needs a Linux kernel. Docker Desktop brings a VM for that, Colima is the lean, free alternative. On the Mac, not in the VM ct-lab:

brew install colima docker docker-buildx docker-compose
mkdir -p ~/.docker && [ -f ~/.docker/config.json ] || echo '{}' > ~/.docker/config.json
jq --arg d "$(brew --prefix)/lib/docker/cli-plugins" '.cliPluginsExtraDirs = [$d]' \
  ~/.docker/config.json > /tmp/config.json && mv /tmp/config.json ~/.docker/config.json
colima start --cpu 2 --memory 4 --disk 40 --vm-type vz --vz-rosetta

Homebrew puts the plugins docker-buildx and docker-compose under $(brew --prefix)/lib/docker/cli-plugins; cliPluginsExtraDirs tells the CLI where to find them. --vm-type vz uses Apple’s Virtualization.framework, --vz-rosetta runs amd64 binaries on Apple silicon with Rosetta instead of QEMU. Then:

colima status
docker context ls
docker run --rm docker.io/library/alpine:3.24 uname -m
docker buildx version && docker compose version

Expected (shortened; tested with Colima on Linux and QEMU – on the Mac you’ll see vz, aarch64, /Users/… and the versions from Homebrew):

time="…" level=info msg="colima is running using QEMU"
time="…" level=info msg="arch: x86_64"
time="…" level=info msg="runtime: docker"
time="…" level=info msg="docker socket: unix:///home/…/.config/colima/default/docker.so…
NAME       DESCRIPTION                               DOCKER ENDPOINT                 …
colima *   colima                                    unix:///home/…/.config/colima/…
default    Current DOCKER_HOST based configuration   unix:///var/run/docker.sock
x86_64
github.com/docker/buildx v0.38.1 0dbe87f36ed472fe36b6796c3f70b4210a731fc5
Docker Compose version v5.6.1

Colima creates the context colima and makes it the active one. Everything from this day works the same way on the Mac with it – Buildx, Compose, contexts. More commands:

colima list                      # all profiles with CPU, RAM, disk, runtime
colima ssh                       # shell in the VM
colima start --kubernetes        # additionally k3s, context for kubectl
colima stop / colima delete      # stop / delete the VM including its data
docker context use default       # back to Docker Desktop or another context

Save your progress? Sign up for free

Drill 8: Scaffdog – scaffolds from Markdown

Profile Scaffdog
Task Generate files and folders from Markdown templates
Special feature Template = .md file; questions in the front matter, placeholders {{ }}
Project github.com/scaffdog/scaffdog (MIT), docs: scaff.dog
Version in this book 4.1.0 (npm), run in node:26.13.0-alpine

Scaffdog is a Node program. Instead of installing Node on the VM, it runs via npx in a throwaway container, with your UID, so that the files belong to you:

cp -R "$LAB/code/tag15/notes/.scaffdog" .
sed -n '1,10p' .scaffdog/go-dienst.md
grep '^# `' .scaffdog/go-dienst.md
docker pull -q docker.io/library/node:26.13.0-alpine
docker run --rm -u "$(id -u):$(id -g)" -e HOME=/tmp -e NO_COLOR=1 \
  -v "$PWD":/work -w /work \
  docker.io/library/node:26.13.0-alpine npx --yes [email protected] list

Expected:

---
name: 'go-dienst'
root: '.'
output: '.'
questions:
  name: 'Name des Dienstes (z. B. notes-report)?'
  port:
    message: 'Port im Container?'
    initial: '8080'
---
# `{{ inputs.name }}/go.mod`
# `{{ inputs.name }}/main.go`
# `{{ inputs.name }}/Dockerfile`
# `{{ inputs.name }}/.dockerignore`
docker.io/library/node:26.13.0-alpine
- go-dienst (.scaffdog/go-dienst.md, 4 templates, 2 questions)
1 file found.

-u sets your UID so that the generated files belong to you, HOME=/tmp gives npm a writable cache, -v and -w mount the project folder as /work and make it the working directory. Every heading # `path` with the code block below it becomes a file. Without -a, Scaffdog asks interactively; with -a name:value you answer the questions in the script:

docker run --rm -u "$(id -u):$(id -g)" -e HOME=/tmp -e NO_COLOR=1 \
  -v "$PWD":/work -w /work \
  docker.io/library/node:26.13.0-alpine \
  npx --yes [email protected] generate go-dienst -a name:notes-report -a port:8097 --force
hadolint notes-report/Dockerfile && echo "hadolint: keine Befunde"
docker buildx build --builder ct-lab --platform linux/amd64,linux/arm64 \
  -t "$REG/lab/notes-report:0.1.0" --push --progress=quiet notes-report && echo gepusht
docker run -d -q --name ct15-report -p 127.0.0.1:8097:8097 \
  "$REG/lab/notes-report:0.1.0" >/dev/null
sleep 2; curl -s localhost:8097/; docker rm -f ct15-report >/dev/null

Expected (keine Befunde = no findings, läuft = running):

     ✔ notes-report/go.mod
     ✔ notes-report/main.go
     ✔ notes-report/Dockerfile
     ✔ notes-report/.dockerignore
hadolint: keine Befunde
sha256:4ef28eb63696…
gepusht
notes-report läuft

From an empty folder to a signable multi-platform image in three commands. The template belongs in the team repo: every new service starts with the same rules (scratch, non-root, pinned base) instead of copying an old Dockerfile.

Save your progress? Sign up for free

Checkpoint

The Docker context podman points to the Podman socket:

cd ~/ct-lab/tag15
unset DOCKER_HOST
docker context ls --format '{{.Name}} {{.DockerEndpoint}}'

Expected:

default unix:///var/run/docker.sock
podman unix:///run/user/1000/podman/podman.sock

The builder ct-lab is running:

docker buildx inspect ct-lab | grep '^Status'

Expected:

Status:                running

notes-api is in the registry for two platforms:

docker buildx imagetools inspect "$REG/lab/notes-api:0.1.0" | grep Platform

Expected:

  Platform:    linux/amd64
  Platform:    linux/arm64
  Platform:    unknown/unknown
  Platform:    unknown/unknown

The four images are in lab/, next to the regsync configuration from Day 9:

crane catalog "$REG" | grep '^lab/'

Expected:

lab/notes-api
lab/notes-report
lab/notes-web
lab/notes-worker
lab/regsync-config

The scaffold from Scaffdog passes hadolint:

hadolint notes-report/Dockerfile; echo "Exit: $?"

Expected:

Exit: 0

Challenge

Add notes-report to the Bake file (target report, amd64 and arm64) and build only this target as version 0.2.0 into your registry.

Hint: Variables in a Bake file can be overridden by an environment variable of the same name; you pass the target as an argument.

Solution

cat >> docker-bake.hcl <<'EOF'

target "report" {
  inherits  = ["_common"]
  context   = "notes-report"
  platforms = ["linux/amd64", "linux/arm64"]
  tags      = ["${REG}/lab/notes-report:${VERSION}"]
}
EOF
VERSION=0.2.0 docker buildx bake --builder ct-lab --push --progress=quiet report \
  && echo gepusht
crane ls "$REG/lab/notes-report"

Expected:

gepusht
0.1.0
0.2.0

The target isn’t in the group default – a plain docker buildx bake still builds only the three notes images.

Save your progress? Sign up for free

Enterprise Note

  • Docker Desktop requires a paid license for larger companies. Colima, Rancher Desktop or Podman Desktop are free alternatives; check beforehand which features (extensions, Scout integration, Enhanced Container Isolation) the team needs.
  • Builders in CI: A shared BuildKit (driver remote or kubernetes) with a registry cache (--cache-to type=registry,…) saves minutes per build.
  • QEMU only when cross-compiling isn’t possible – native arm64 runners are faster and more reliable.
  • Contexts instead of DOCKER_HOST: named contexts make it visible where a command goes – important as soon as remote hosts (host=ssh://…) come into play.
  • Maintain templates: Scaffdog templates belong versioned in the platform repo, with a CI job that builds and lints every generated scaffold.