Appendix A: Cheat Sheet
All 34 tools on a few pages: installation and the commands you
practiced in the book. The number in parentheses is the day. Variables
like $REG are set by
source ~/ct-lab/setup/env.sh; the registry is
registry.ct.lab:5000, login
lab/lab-registry-pw-123. When in doubt, the
project’s docs or --help on the VM take precedence.
Installation at a glance
| Tool | Version in this book | Source | Day |
|---|---|---|---|
| podman, Quadlet | 5.7.0 | Ubuntu package podman |
1–3 |
| podman-compose | 1.5.0 | Ubuntu package podman-compose |
4 |
| lazydocker, ctop | 0.25.2, 0.7.7 (outdated) | Release binary | 4 |
| buildah | 1.42.1 | Ubuntu package buildah |
5 |
| ko | 0.19.1 | Release binary, plus golang-go |
5 |
| ctr, nerdctl, buildctl | v2.4.2, 2.4.2, v0.34.1 | nerdctl-full bundle | 6 |
| buildctl, Lima (Mac) | 0.34.1, 2.2.1 | Homebrew, VM via template://buildkit |
6 |
| crane, krane | 0.22.1 | Release binary (go-containerregistry) | 7, 10 |
| skopeo | 1.21.0 | Ubuntu package skopeo |
8 |
| regctl, regsync | v0.11.7 | Release binary (regclient) | 8, 9 |
| oras | 1.3.6 | Release binary | 9 |
| kind, crictl | v0.33.0, v1.37.0 | Release binary | 10 |
| dive, diffoci | 0.13.1, v0.1.8 | Release binary | 11 |
| container-diff | v0.19.0 (archived) | Self-built (go mod download) |
11 |
| hadolint, dockle | 2.15.1, 0.4.15 | Release binary | 12 |
| ShellCheck | 0.11.0 | Ubuntu package shellcheck |
12 |
| trivy, grype | 0.76.1, 0.121.0 | Release binary | 13 |
| docker scout | v1.29.0, Docker CE 29.9.2 | Docker apt repository + CLI plugin | 13 |
| syft, cosign | 1.55.0, v3.1.4 | Release binary | 14 |
| docker, Buildx, Compose | 29.9.2, v0.38.1, v5.6.1 | Docker apt repository (plugins) | 13, 15 |
| Colima (Mac), Lima | v0.10.3, 2.2.1 | Homebrew | 15 |
| Scaffdog | 4.1.0 | npm, via npx in node:26.13.0-alpine |
15 |
Every release binary in four steps – download, verify checksum,
unpack, install (0). URL and checksum are in
setup/tools.lock:
sudo apt-get install -y podman buildah skopeo jq apache2-utils # packages (0)
arch=$(dpkg --print-architecture) # one binary by hand (0, 7, 8)
read -r _ _ url sum < <(grep "^regctl $arch " "$LAB/setup/tools.lock")
curl -fsSLo /tmp/regctl "$url"
echo "$sum /tmp/regctl" | sha256sum -c -
sudo install -m 0755 /tmp/regctl /usr/local/bin/regctl && rm /tmp/regctl
sha256sum -c --ignore-missing checksums.txt # against the project's file (0, 4)Podman and systemd (Days 1–4)
podman
podman pull docker.io/library/nginx:1.30.6-alpine # always fully qualified (1)
podman run -d --name web -p 127.0.0.1:8080:80 docker.io/library/nginx:1.30.6-alpine
podman ps --format '{{.Names}}\t{{.Status}}\t{{.Ports}}'
podman logs --tail 1 web
podman top web user huser pid hpid args # UID in the container and on the host (1)
podman build --format docker -t localhost/notes-api:0.1.0 \
--build-arg VERSION=0.1.0 notes-api # Docker format keeps HEALTHCHECK (1)
podman image inspect localhost/notes-api:0.1.0 \
--format '{{.Config.Healthcheck.Test}} {{.ManifestType}}'
podman healthcheck run notes-api && echo "gesund"
podman save localhost/notes-api:0.1.0 | sudo podman load # rootless → rootful (1)
podman info --format '{{.Store.GraphRoot}}'
podman image prune -f # only the user's unnamed images (1)Rootless in detail, pods and Kubernetes YAML (2):
podman unshare cat /proc/self/uid_map # 0 → 1000, 1 → 100000 … (2)
podman run -d --name pg-bind --user 70:70 -e POSTGRES_PASSWORD=lab-db-pw-123 \
-v ./pgdata-bind:/var/lib/postgresql:U docker.io/library/postgres:18.6-alpine
podman unshare rm -rf pgdata-bind # delete files with foreign UIDs (2)
podman pod create --name notes -p 127.0.0.1:8080:8080 -p 127.0.0.1:3000:3000
podman run -d --pod notes --name notes-db … docker.io/library/postgres:18.6-alpine
podman kube generate notes -f notes-pod.yaml
podman kube play --replace notes-pod.yaml
podman kube down notes-pod.yaml # volumes remain (without --force)Observing (4):
podman stats --no-stream --format 'table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}'
podman pod stats --no-stream --format '{{.Pod}} {{.Name}} {{.MemUsage}}'
podman events --since 30s --until 1s --filter container=notes-api \
--format '{{.Status}} {{.Name}}'Quadlet
Units in ~/.config/containers/systemd/,
daemon-reload after every change (3):
# ~/.config/containers/systemd/notes-api.container – template (3)
[Unit]
Requires=notes-migrate.container
After=notes-migrate.container
[Container]
ContainerName=notes-api
Image=localhost/notes-api:0.1.0
Pod=notes.pod
Environment=NOTES_API_KEY=lab-api-key-123
Notify=healthy
AutoUpdate=local
[Service]
Restart=on-failure/usr/libexec/podman/quadlet -dryrun -user # generated units, typos (3, 16)
QUADLET_UNIT_DIRS=/tmp/qtest /usr/libexec/podman/quadlet -dryrun -user
systemctl --user daemon-reload
systemctl --user start notes-pod.service # not enable: the units are generated
systemctl --user list-units 'notes*' --all --no-pager
journalctl --user -u notes-api.service --no-pager -o cat
sudo loginctl enable-linger "$USER" # services without a login session (3)
podman auto-update --dry-run
systemctl --user enable --now podman-auto-update.timerPodman socket and podman-compose
systemctl --user enable --now podman.socket # Docker API 1.41 (4)
S="$XDG_RUNTIME_DIR/podman/podman.sock"
curl -s --unix-socket "$S" http://d/_ping; echo
export DOCKER_HOST="unix://$XDG_RUNTIME_DIR/podman/podman.sock" # in ~/.bashrc
podman --remote ps --format '{{.Names}}'
sudo apt-get install -y podman-compose # provider for podman compose (4)
podman compose up -d
podman compose logs worker | tail -1
podman compose downlazydocker
lazydocker # reads DOCKER_HOST (4)
lazydocker -c | grep -E '^commandTemplates:|dockerCompose:' # default valuesIn ~/.config/lazydocker/config.yml:
commandTemplates: →
dockerCompose: podman compose. Keys:
1…6 panels, [ ] tabs
(logs, stats …), x menu, q quit.
ctop
ctop -v && ctop # no release since 2022 – outdated (4)Building (Days 5–6)
buildah
sudo apt-get install -y buildah
buildah build --format docker -t localhost/notes-api:0.2.0 \
--build-arg VERSION=0.2.0 notes-api # without --layers: one layer (5)
buildah build -q --format docker --timestamp 0 … # reproducible image ID (5)
b=$(buildah from --name api-build docker.io/library/golang:1.27-alpine)
buildah copy "$b" notes-api /src
buildah run --workingdir /src -e CGO_ENABLED=0 "$b" -- \
go build -trimpath -ldflags="-s -w -X main.version=0.2.0" -o /out/notes-api .
f=$(buildah from --format docker --name api-final scratch)
buildah copy --from "$b" --chmod 555 "$f" /out/notes-api /notes-api
buildah config --user 65532:65532 --entrypoint '["/notes-api"]' \
--healthcheck 'CMD /notes-api healthcheck' "$f"
buildah commit --format docker --rm "$f" localhost/notes-api:0.2.0-buildah
buildah rm "$b" && buildah containers
buildah unshare sh -c '
m=$(buildah mount api-final); ls -ln "$m"; buildah umount api-final'
buildah push localhost/notes-api:0.2.0 oci:./out:notes-api # OCI: health check gone
buildah push localhost/notes-api:0.2.0 \
docker-archive:./out/notes-api.tar:localhost/notes-api:0.2.0ko
sudo apt-get install -y golang-go
ko build --push=false . # without KO_DOCKER_REPO, build only (5)
ko build --local . # via DOCKER_HOST to ko.local/…
export VERSION=0.2.0 # for {{.Env.VERSION}} in .ko.yaml
KO_DOCKER_REPO=ko.local/notes-api ko build --bare -t 0.2.0 --sbom-dir ../out/sbom .
KO_DOCKER_REPO=localhost/notes-api ko build --bare -t 0.2.0-ko --push=false \
--tarball ../out/ko.tar .
podman load -i ../out/ko.tar
podman run -d --name api-ko --health-cmd '["/ko-app/notes-api", "healthcheck"]' …ctr
sudo systemctl daemon-reload && sudo systemctl enable --now containerd
sudo ctr ns ls
sudo ctr images pull docker.io/library/nginx:1.30.6-alpine # no short names
sudo ctr run -d docker.io/library/nginx:1.30.6-alpine web
sudo ctr tasks ls
sudo ctr tasks exec --exec-id e1 web nginx -v
sudo ctr tasks kill web && sudo ctr tasks rm web && sudo ctr containers rm web
podman save localhost/notes-api:0.2.0 | sudo ctr images import - # Podman → containerdnerdctl
sudo nerdctl run -d --name web -p 127.0.0.1:8081:80 \
docker.io/library/nginx:1.30.6-alpine
sudo nerdctl ps && sudo nerdctl images
sudo nerdctl network ls
sudo nerdctl compose -f compose.yaml -f compose.nerdctl.yaml up -d db # db first (6)
sudo nerdctl compose -f compose.yaml -f compose.nerdctl.yaml up -d
sudo nerdctl build -t localhost/notes-api:0.2.0 --build-arg VERSION=0.2.0 notes-api
sudo systemctl disable --now containerd buildkitbuildctl
sudo systemctl enable --now buildkit
sudo buildctl debug workers -v
sudo buildctl build --frontend dockerfile.v0 \
--local context=notes-api --local dockerfile=notes-api \
--opt build-arg:VERSION=0.2.0 \
--output type=oci,dest=out/notes-api-oci.tar,name=localhost/notes-api:0.2.0-buildctl \
--export-cache type=local,dest=out/buildcache
sudo buildctl build … --import-cache type=local,src=out/buildcache
sudo buildctl build … --output type=docker,name=localhost/notes-api:0.3.0,dest=- \
| sudo ctr images import -
sudo buildctl prune --allStandalone BuildKit with Lima (Mac)
brew install buildkit lima # buildctl on the Mac (6)
limactl start --name=buildkit --cpus=2 --memory=2 --disk=20 template://buildkit
export BUILDKIT_HOST="unix://$HOME/.lima/buildkit/sock/buildkitd.sock" # ~/.zshrc
buildctl debug workers
buildctl build --frontend dockerfile.v0 --local context=. --local dockerfile=. \
--output type=oci,dest=myapp.tar,name=localhost/myapp:0.1.0
buildctl build … --output "type=image,name=$REG/lab/myapp:0.1.0,push=true" # from 7 on
limactl stop buildkit; limactl list; limactl start buildkitBuildKit only builds: no images, no run.
Run the result via podman load -i myapp.tar.
Registry and moving images (Days 7–10)
Your own registry
./run-registry.sh [kind] # restart the registry (7, 10)
curl -sS -u "$REG_USER:$REG_PASS" "https://$REG/v2/_catalog"; echo
curl -sS -u "$REG_USER:$REG_PASS" "https://$REG/v2/notes/notes-api/tags/list"; echo
podman exec registry registry garbage-collect --dry-run /etc/distribution/config.yml| Tool | Writes login to | Also reads |
|---|---|---|
| podman, buildah, skopeo | $XDG_RUNTIME_DIR/containers/auth.json |
~/.docker/config.json |
| crane, krane | ~/.docker/config.json |
Podman auth.json |
| regctl | ~/.regctl/config.json |
~/.docker/config.json |
| oras, cosign | ~/.docker/config.json |
– |
With DOCKER_CONFIG set, crane and oras read and write
there instead of in ~/.docker; podman login
then writes there too, as long as REGISTRY_AUTH_FILE isn’t
set (16).
crane
echo "$REG_PASS" | crane auth login "$REG" -u "$REG_USER" --password-stdin
crane catalog "$REG"
crane ls docker.io/library/nginx | grep '^1\.30\.6-alpine'
crane digest docker.io/library/nginx:1.30.6-alpine # HEAD: doesn't count
crane digest --platform linux/arm64 docker.io/library/nginx:1.30.6-alpine
crane manifest "$REG/library/nginx:1.30.6-alpine" | jq -r .mediaType
crane config --platform linux/arm64 "$REG/library/nginx:1.30.6-alpine" | jq .config
crane copy docker.io/library/nginx:1.30.6-alpine "$REG/library/nginx:1.30.6-alpine"
crane tag "$REG/lab/notes-api:oci" stabil
crane delete "$REG/lab/notes-api@$(crane digest "$REG/lab/notes-api:oci")"
crane mutate "$REG/lab/notes-api:0.1.0" --label lab.tag=07 -t "$REG/lab/notes-api:mut"
crane append -b "$REG/lab/notes-api:0.1.0" -f extra.tar -t "$REG/lab/notes-api:extra"
crane export "$REG/lab/notes-api:extra" - | tar -t
crane validate --remote "$REG/lab/notes-api:extra"
echo "$REG" | crane auth get; echoskopeo
skopeo inspect "docker://$REG/library/nginx:1.30.6-alpine" | jq '{Digest, Architecture}'
skopeo inspect --config --raw "docker://$REG/notes/notes-api:0.1.0" \
| jq -c .config.Healthcheck.Test # --raw: original config (8)
skopeo copy -q --all --preserve-digests "docker://$REG/library/nginx:1.30.6-alpine" \
"docker://$REG/mirror/nginx:1.30.6-alpine-all" # a true mirror
skopeo copy -q "docker://$REG/notes/notes-api:0.1.0" oci:api-oci:0.1.0
podman unshare skopeo copy -q dir:api-dir \
containers-storage:localhost/notes-api:von-dir # the only way rootless (8)
skopeo sync --src docker --dest dir --scoped "$REG/notes/notes-api" backup/
skopeo list-tags "docker://$REG/notes/notes-api" | jq -c .Tags
skopeo delete "docker://$REG/backup/notes-web:0.1.0"
skopeo --policy policy-lab.json copy -q "docker://$REG/notes/notes-api:0.1.0" dir:p1Transports: docker:// (registry), dir:,
oci:path:tag, docker-archive:file,
containers-storage: (Podman storage).
regctl
echo "$REG_PASS" | regctl registry login "$REG" -u "$REG_USER" --pass-stdin
regctl registry whoami "$REG" # only knows ~/.regctl/config.json
regctl tag ls --include '1\.30\.6-alpine.*' docker.io/library/nginx
regctl image ratelimit docker.io/library/nginx:1.30.6-alpine
regctl manifest head "$REG/library/nginx:1.30.6-alpine"
regctl manifest get "$REG/notes/notes-api:0.1.0" --format raw-body | jq -r .mediaType
regctl image digest --platform linux/arm64 "$REG/library/nginx:1.30.6-alpine"
regctl image copy "$REG/notes/notes-api:0.1.0" "$REG/lab/notes-api:rc"
regctl image mod "$REG/lab/notes-api:rc" --to-oci --label lab.tag=08 \
--create 0.1.0-oci08 # health check stays (8)
regctl image get-file "$REG/notes/notes-api:0.1.0" /etc/ssl/certs/ca-certificates.crt
regctl image export "$REG/notes/notes-api:0.1.0" api-regctl.tar
regctl tag rm "$REG/lab/notes-api:rc" # only the tag
regctl repo ls "$REG"
regctl artifact list "$REG/release/notes-api:0.1.0"regsync
regsync config -c regsync.yml # shows passwords in plain text!
regsync check -c regsync.yml
regsync once -c regsync.yml
regsync once --missing -c regsync.yml
timeout 15 regsync server -c regsync.yml
systemd-analyze verify --user systemd/regsync.service systemd/regsync.timer# regsync.yml – template (9)
version: 1
creds:
- registry: '{{ env "REG" }}'
user: '{{ env "REG_USER" }}'
pass: '{{ env "REG_PASS" }}'
defaults:
ratelimit: {min: 20, retry: 15m}
sync:
- source: docker.io/library/postgres
target: '{{ env "REG" }}/mirror/postgres'
type: repository
platform: linux/arm64
tags:
allow: ['18\.6-alpine']oras
echo "$REG_PASS" | oras login -u "$REG_USER" --password-stdin "$REG"
oras repo ls "$REG" && oras repo tags "$REG/notes/notes-api"
oras push "$REG/lab/regsync-config:tag09" \
--artifact-type application/vnd.ct-lab.config regsync.yml:application/yaml
oras pull -o /tmp/zurueck "$REG/lab/regsync-config:tag09"
oras manifest fetch "$REG/charts/notes:0.1.0" | jq .
oras attach --artifact-type application/spdx+json "$REG/notes/notes-api:0.1.0" \
notes-api.spdx.json:application/spdx+json
oras discover "$REG/notes/notes-api:0.1.0"
oras discover --artifact-type text/markdown "$REG/notes/notes-web:0.1.0"
oras cp -r "$REG/notes/notes-api:0.1.0" "$REG/release/notes-api:0.1.0" # with referrers
oras manifest delete --force "$REG/notes/notes-api@$S"kind
export KIND_EXPERIMENTAL_PROVIDER=podman
CONTAINERS_CONF_OVERRIDE=$PWD/kind/containers-kind.conf \
kind create cluster --config kind/kind-ct-lab.yaml # rootless (10)
N=ct-lab-control-plane
podman exec $N kubectl get pods -A # kubectl in the node (10)
podman save -q -o api-020.tar localhost/notes-api:0.2.0
kind load image-archive api-020.tar --name ct-lab
kind delete cluster --name ct-labcrictl
podman cp /usr/local/bin/crictl ct-lab-control-plane:/usr/local/bin/crictl
N=ct-lab-control-plane
podman exec $N crictl pods # crictl in the node (10)
podman exec $N crictl ps
podman exec $N crictl images
CID=$(podman exec $N crictl ps --name notes-api -q)
podman exec $N crictl inspect "$CID" | jq -r .status.imageRef
podman exec $N crictl logs --tail 1 "$CID"
podman exec $N crictl exec "$CID" /notes-api healthcheck
podman exec $N crictl pull --creds lab:lab-registry-pw-123 \
registry.ct.lab:5000/notes/notes-web:0.1.0
podman exec $N crictl rmi registry.ct.lab:5000/notes/notes-web:0.1.0krane
krane version && krane ls "$REG/notes/notes-api" # crane + cloud keychains (10)
DOCKER_CONFIG=/tmp/leer env -u XDG_RUNTIME_DIR krane ls "$REG/notes/notes-api"krane reads the Docker config, Google, GitHub, ECR, ACR – no
kubeconfig, no pull secrets. A pull secret is a Docker config: mount it
as a file and point DOCKER_CONFIG at it.
Checking and securing (Days 11–14)
dive
dive docker.io/library/nginx:1.30.6-alpine # UI via DOCKER_HOST
CI=true dive podman://docker.io/library/nginx:1.30.6-alpine # calls podman save
dive --ci --source podman localhost/notes-api:schlecht # rules from .dive-ci (11)
dive --source podman localhost/notes-api:schlecht -j dive.json
dive --ci docker-archive://nginx.tar# .dive-ci – template (11)
rules:
lowestEfficiency: 0.98
highestWastedBytes: 10MB
highestUserWastedPercent: 0.05diffoci
diffoci diff --semantic podman://localhost/notes-api:0.1.0 \
podman://localhost/notes-api:0.2.0 # exit 1 on differences (11)
diffoci diff --semantic --verbose podman://… podman://…
diffoci diff --semantic podman://localhost/notes-api:0.1.0 "$REG/notes/notes-api:0.1.0"container-diff
Archived, only for comparison with diffoci (11):
podman run --rm -v "$PWD/bin:/out:Z" -e CGO_ENABLED=0 \
docker.io/library/golang:1.27-alpine sh -c '
m=github.com/GoogleContainerTools/[email protected]
cd "$(go mod download -json "$m" | sed -n "s/.*\"Dir\": \"\(.*\)\",/\1/p")"
go build -o /out/container-diff .'
./bin/container-diff diff api-010.tar api-020.tar --type=file --type=sizehadolint
hadolint --no-color notes-api/Dockerfile # reads .hadolint.yaml in the folder (12)
hadolint -t error notes-api/Containerfile.lint
hadolint --no-fail notes-api/Containerfile.lint
hadolint --disable-ignore-pragma notes-api/Containerfile.lint
hadolint -f sarif notes-api/Containerfile.lint -o lint.sarifExceptions: # hadolint ignore=DL3008 above the line,
--ignore DL3059 for one run, ignored:,
override: and trustedRegistries: in
.hadolint.yaml.
dockle
export DOCKLE_HOST="unix://$XDG_RUNTIME_DIR/podman/podman.sock" # not DOCKER_HOST
dockle --no-color -c 1 localhost/notes-worker:0.1.0 # .dockleignore in the folder (12)
dockle --no-color --accept-key KEY_SHA512 --exit-code 1 \
docker.io/library/nginx:1.30.6-alpine
dockle --no-color --input api.tar # archive from podman save
dockle -f json localhost/notes-api:0.1.0 | jq -c '.summary'ShellCheck
sudo apt-get install -y shellcheck
shellcheck push-alle.sh # findings with SC code, suggestion (12)
shellcheck -x push-alle.sh # check source files too
shellcheck -S warning -f gcc tag12/*.sh # warnings only, format for editor/CI
shellcheck -f json1 push-alle.sh | jq -c '.comments[] | [.line, .code]'Exceptions: # shellcheck disable=SC2086 above the line,
# shellcheck source=file for source,
source-path=SCRIPTDIR and
external-sources=true in .shellcheckrc.
trivy
trivy image -q --scanners vuln --severity HIGH,CRITICAL --ignore-unfixed \
--exit-code 1 localhost/notes-api:0.1.0 # gate, reads .trivyignore (13)
trivy image -q --scanners vuln --show-suppressed localhost/notes-api:0.1.0
trivy image -q --scanners vuln --ignorefile '' localhost/notes-api:0.1.1
trivy image -q --scanners secret --exit-code 1 localhost/notes-api:schlecht
trivy fs -q --scanners vuln --table-mode summary .
trivy config -q notes-api/Containerfile.besser
trivy sbom -q --table-mode summary nginx.cdx.json
trivy image -q --skip-db-update … && trivy clean --vuln-db# .trivyignore – template (13): every exception with a reason and an expiry
CVE-2026-56851 exp:2026-12-31
grype
grype db update # approx. 4–5 min, 3.2 GB (13)
grype -q podman:docker.io/library/nginx:1.30.6-alpine
grype -q "registry:$REG/notes/notes-api:0.1.0" --only-fixed --by-cve
grype -q podman:localhost/notes-api:0.1.0 --only-fixed --fail-on high # exit 2
grype -q sbom:nginx.cdx.json --only-fixed --fail-on critical
grype db statusdocker scout
Needs Docker CE and a Docker account; sees only Docker’s storage (13):
docker scout quickview docker.io/library/nginx:1.30.6-alpine
docker scout cves --only-severity critical,high --only-fixed --exit-code \
docker.io/library/nginx:1.30.6-alpine
syft -q podman:docker.io/library/nginx:1.30.6-alpine -o spdx-json \
| docker scout cves sbom://
docker scout recommendations docker.io/library/nginx:1.30.6-alpinesyft
syft -q podman:localhost/notes-api:0.1.0 # table, Go modules from the binary
syft -q dir:. -o purls
syft -q "registry:$REG/notes/notes-api:0.1.0" -o syft-json=api.syft.json \
-o cyclonedx-json=api.cdx.json -o spdx-json=api.spdx.json # three formats (14)
syft convert api.syft.json -o cyclonedx-json=konvertiert.cdx.jsoncosign
export COSIGN_PASSWORD=lab-cosign-pw-123
cosign generate-key-pair # in ~/ct-lab/keys, never in the repo (14)
cosign signing-config create --out lab-signing-config.json # without Rekor/TSA
img="$REG/notes/notes-api@$(crane digest "$REG/notes/notes-api:0.1.0")" # digest!
cosign sign --key ~/ct-lab/keys/cosign.key --signing-config lab-signing-config.json \
--yes -a version=0.1.0 "$img"
cosign verify --key ~/ct-lab/keys/cosign.pub --insecure-ignore-tlog=true "$img"
cosign attest --key ~/ct-lab/keys/cosign.key --signing-config lab-signing-config.json \
--yes --type cyclonedx --predicate api.cdx.json "$img"
cosign verify-attestation --key ~/ct-lab/keys/cosign.pub \
--insecure-ignore-tlog=true --type cyclonedx "$img"
cosign tree "$img"
cosign download attestation "$img" # verifies nothing
cosign verify … --registry-cacert "$LAB/registry/certs/ca.crt" "$img"Docker tools (Day 15)
docker and contexts
docker context create podman --docker "host=unix://$XDG_RUNTIME_DIR/podman/podman.sock"
docker context ls # DOCKER_HOST overrides every context
unset DOCKER_HOST # then the active context applies (15)
docker --context podman ps # per command
docker context use podman # permanently
docker version --format '{{.Client.Version}} / {{.Server.Version}}'
docker build -q -t ct-lab/notes-api:0.1.0 notes-api
docker ps --format '{{.Names}}\t{{.Status}}'
docker stats --no-stream; docker system dfBuildx and Bake
docker buildx ls
docker buildx create --name ct-lab --driver docker-container \
--driver-opt image=docker.io/moby/buildkit:v0.34.1 --driver-opt network=host \
--buildkitd-config buildkitd.toml --bootstrap # registry CA in the TOML (15)
echo "$REG_PASS" | docker login "$REG" -u "$REG_USER" --password-stdin
docker buildx build --builder ct-lab --platform linux/amd64,linux/arm64 \
-t "$REG/lab/notes-api:0.1.0" --push notes-api
docker buildx imagetools inspect "$REG/lab/notes-api:0.1.0"
docker buildx bake --print # resolved Bake file
docker buildx bake --builder ct-lab --push # group default
VERSION=0.2.0 docker buildx bake --builder ct-lab --push report # one targetDocker Compose
docker compose up -d --build --wait # build, start, health checks (15)
docker compose ps --format '{{.Service}}\t{{.Status}}'
docker compose logs api; docker compose exec api /notes-api healthcheck
docker compose down # -v also deletes the volumesColima (Mac)
brew install colima docker docker-buildx docker-compose
colima start --cpu 2 --memory 4 --disk 40 --vm-type vz --vz-rosetta # (15)
colima status; colima list; colima ssh
colima stop; colima delete # delete the VM including images
docker context use default # back to another contextThe CLI finds plugins from Homebrew via
"cliPluginsExtraDirs" in ~/.docker/config.json
($(brew --prefix)/lib/docker/cli-plugins).
Scaffdog
docker run --rm -u "$(id -u):$(id -g)" -e HOME=/tmp \
-v "$PWD":/work -w /work \
docker.io/library/node:26.13.0-alpine npx --yes [email protected] list # (15)
# instead of list: generate go-dienst -a name:notes-report -a port:8097 [--dry-run|--force]Pitfalls on Ubuntu 26.04
All of them occurred in the lab and are shown in the respective chapter.
AppArmor
- pasta when removing networks (2, 4): When you
remove the last container of a custom network rootless (including
kube down,docker compose down), Podman reportsrootless netns: kill network process: permission denied– thepastaprofile denies Podman’s signal. The objects are gone anyway; Docker Compose needs a seconddown. - nerdctl-default, docker-default and gosu (6, 15):
Under
nerdctland Docker CE, Postgres hangs with full CPU – the profilesnerdctl-defaultanddocker-defaultforbidgosu(and Postgres itself) from sending signals to their own threads and processes (journalctl -k | grep DENIED). Lab solution:security_opt: [apparmor=unconfined]fordbincompose.nerdctl.yamlorcompose.yaml(Day 15); in production, a custom profile or an image without gosu. - skopeo and
containers-storage:(8, 16):kernel.apparmor_restrict_unprivileged_userns = 1allows user namespaces only for programs with a profile; podman and buildah have one, skopeo doesn’t →Error during unshare(...): Operation not permitted. Remedy:podman unshare skopeo copy …orpodman save. - Rootless containerd (6):
containerd-rootless-setuptool.sh checkfails because/usr/local/bin/rootlesskithas no profile. In this book, containerd stays rootful.
Names and formats
- Short names (1, 16):
registries.confhas nounqualified-search-registries,shortnames.confdoesn’t knownginxandpostgres. A short name only matches images that are already local – on a fresh machine, the same build fails. Always writedocker.io/library/…. - HEALTHCHECK disappears (1, 5, 8, 12, 16): Without
--format docker, Podman and Buildah build OCI and only warn.buildah push … oci:,skopeo copy --format ociand ko lose it too;regctl image mod --to-ocidoesn’t. - ctr doesn’t know short names (6):
nginx:1.30.6-alpinebecomesparse "dummy://nginx…": invalid port.
Packages and services
- docker-compose-v2 from apt (4): pulls in
docker.ioandcontainerd– don’t install it; use podman-compose or the Docker Compose binary. - Ports after stopping (4): After
systemctl --user stop notes-pod.service, wait a good 3 s, otherwise you getinternal libpod errorfor the next stack on the same ports. - Ports below 1024 (2, 16): rootless
Listen failed … Permission denied(net.ipv4.ip_unprivileged_port_start = 1024); as a Quadlet service, the message only appears in the unit’s journal. - buildkitd (6): exits with exit 1 when stopped,
systemctl is-active buildkitshowsfailed–sudo systemctl reset-failed buildkit. - Go in Ubuntu (5):
golang-gois 1.26, notes-api requires 1.27 –GOTOOLCHAIN=autodownloads the matching toolchain to~/go/pkg/mod.
Logins, TLS, registry
- Podman login in RAM (7):
$XDG_RUNTIME_DIR/containers/auth.jsonis gone after a reboot. regctl and oras never read it (table above). - Certificate without SAN (7, 16): curl accepts the
CN, none of the Go tools do:
x509: certificate relies on legacy Common Name field. Issue the certificate withsubjectAltName. - Registry 3.1.2 without Referrers API (9, 14): oras
and cosign create the tag
sha256-<digest>; take it along when copying (oras cp -r). - cosign v3 (14):
--tlog-upload=falseno longer works – an empty signing config and--insecure-ignore-tlog=truewhen verifying. - Docker Hub (7): anonymous: 100 pulls per hour and
IP; HEAD (
crane digest,regctl image digest) doesn’t count.
Tool quirks
- dockle (12): ignores
DOCKER_HOSTand takeslocalhost/…for a registry – setDOCKLE_HOSTor use--inputwith an archive. - grype (13): local Podman images only with the
podman:prefix and a running Podman socket. - kind (10):
kind load docker-imagedoesn’t find Podman’slocalhost/images (→image-archive); the node inherits the VM’s/etc/hosts(→base_hosts_file = "none"viaCONTAINERS_CONF_OVERRIDE). - dive (11): deleted folders don’t count as waste – look at the layer sizes as well.
- hadolint (12):
-t nonestill returned exit 1 – use--no-fail. - Quadlet (3, 16): typos make the unit silently
disappear (
Unit … not found); only the dry run names the line. Generated units can’t beenabled.