Skip to content
Hands-on IT IT Drills

Day 0: The training ground

Goal: Create the lab VM, install and verify all tools according to a fixed principle and get to know the demo application notes. · Duration: approx. 60 min · Prerequisite: none

Briefing

  • You work in an Ubuntu 26.04 VM ct-lab, as user ubuntu in ~/ct-lab. On the Mac you create it with Multipass.
  • Two ways of installing: Ubuntu package (apt, signed via the archive) or release binary from GitHub (verify it yourself with SHA-256, then to /usr/local/bin).
  • setup/tools.lock pins every binary with URL and checksum for amd64 and arm64. Today you use it to install all tools, with the four steps from Drill 4 in a short loop.
 GitHub                           VM ct-lab (Ubuntu 26.04, arm64)
 ┌──────────────────────┐        ┌──────────────────────────────────────────┐
 │ thomaszachmann/      │  git   │ ~/ct-lab/code   companion code (orig.)   │
 │ container-tools-     │ clone  │ ~/ct-lab/setup  env.sh, tools.lock       │
 │ zum-anfassen         │───────►│ ~/ct-lab/notes  notes-api, -web, -worker │
 └──────────────────────┘        │ apt:  podman buildah skopeo jq …         │
 Mac: multipass shell ──────────►│ /usr/local/bin: crane trivy cosign …     │
                                 └──────────────────────────────────────────┘

Drill 1: The lab VM

On the Mac (Multipass from multipass.run):

multipass launch 26.04 --name ct-lab --cpus 2 --memory 4G --disk 40G
multipass shell ct-lab

From here on, everything runs inside the VM:

grep -E '^(PRETTY_NAME|VERSION_ID)' /etc/os-release
uname -m
nproc && free -h | head -2

Expected (abridged, values will differ on your machine):

PRETTY_NAME="Ubuntu 26.04.1 LTS"
VERSION_ID="26.04"
aarch64
2
               total        used        free      shared  buff/cache   available
Mem:           3.9Gi       823Mi       426Mi       5.4Mi       3.0Gi       3.1Gi

aarch64 means: on a Mac with Apple Silicon you need the arm64 variant of a binary everywhere. On an Intel machine or in the cloud this shows x86_64 (amd64).

Save your progress? Sign up for free

Drill 2: Packages from Ubuntu

Podman, Buildah and Skopeo come from the same project (github.com/containers) and are well maintained in Ubuntu. On top of that: jq for JSON, htpasswd from apache2-utils and ShellCheck for shell scripts (Day 12):

sudo apt-get update
sudo apt-get install -y podman buildah skopeo jq apache2-utils shellcheck
podman --version
buildah --version
skopeo --version
jq --version
dpkg -l apache2-utils | tail -1 | awk '{print $2, $3}'
shellcheck --version | sed -n 2p

Expected (abridged):

podman version 5.7.0
buildah version 1.42.1 (image-spec 1.1.1, runtime-spec 1.2.1)
skopeo version 1.21.0-dev
jq-1.8.1
apache2-utils 2.4.66-2ubuntu2.6
version: 0.11.0

Installed along with them are crun (the OCI runtime), passt/pasta (networking for rootless containers) and conmon. apt verifies a package itself: the package lists are signed by the Ubuntu archive, and every .deb is listed there with its checksum.

Save your progress? Sign up for free

Drill 3: Companion code into the VM

The companion code is public on GitHub. You clone it inside the VM to ~/ct-lab/code:

mkdir -p ~/ct-lab && cd ~/ct-lab
git clone https://github.com/thomaszachmann/container-tools-zum-anfassen.git code
ls code

Expected (abridged):

Cloning into 'code'...
LICENSE  README.md  anhb  notes  setup  tag01  tag02  tag03  tag04  …

The setup files and the demo application get their own copies, so you can change them without losing the original:

cd ~/ct-lab
cp -R code/setup setup && cp -R code/notes notes
ls ~/ct-lab ~/ct-lab/setup
grep -v '^#' setup/env.sh

Expected (abridged):

/home/ubuntu/ct-lab:
code  notes  setup

/home/ubuntu/ct-lab/setup:
env.sh  tools.lock
export LAB="${LAB:-$HOME/ct-lab}"
export NOTES="$LAB/notes"
export REG_HOST="${REG_HOST:-registry.ct.lab}"
export REG_PORT="${REG_PORT:-5000}"
export REG="$REG_HOST:$REG_PORT"              # eigene Registry ab Tag 7
export REG_USER=lab
export REG_PASS="${REG_PASS:-lab-registry-pw-123}"   # Lab-Wert

env.sh sets the variables that all drills use. $LAB/code stays unchanged: that is where you pick up the previous day’s state every morning.

Save your progress? Sign up for free

Drill 4: The installation principle

You always install a release binary in four steps: download, verify the checksum, unpack, install. Using crane (Day 7) from github.com/google/go-containerregistry as an example:

mkdir -p /tmp/dl && cd /tmp/dl
V=v0.22.1
BASE=https://github.com/google/go-containerregistry/releases/download/$V
curl -fsSLO "$BASE/go-containerregistry_Linux_arm64.tar.gz"
curl -fsSLO "$BASE/checksums.txt"
grep Linux_arm64 checksums.txt
sha256sum -c --ignore-missing checksums.txt

Expected:

898c0cff975f…  go-containerregistry_Linux_arm64.tar.gz
go-containerregistry_Linux_arm64.tar.gz: OK
  • -f makes curl fail on HTTP errors instead of saving an error page; -L follows GitHub’s redirect to the download server.
  • --ignore-missing checks only the files from checksums.txt that you actually downloaded.

The same checksum is in tools.lock. And this is what it looks like when it does not match:

grep '^crane arm64' ~/ct-lab/setup/tools.lock | awk '{print $4}'
echo "$(printf '0%.0s' {1..64})  go-containerregistry_Linux_arm64.tar.gz" \
  | sha256sum -c -; echo "Exit: $?"

Expected:

898c0cff975f898a33e8c4580bdafb0e7c02c7faa33374e946762f97c4ab7110
go-containerregistry_Linux_arm64.tar.gz: FAILED
sha256sum: WARNING: 1 computed checksum did NOT match
Exit: 1

Exit code 1 – a script stops here. Only after that do you unpack and install:

tar -tzf go-containerregistry_Linux_arm64.tar.gz
tar -xzf go-containerregistry_Linux_arm64.tar.gz crane krane
sudo install -m 0755 crane krane /usr/local/bin/
crane version && krane version
cd ~ && rm -rf /tmp/dl

Expected:

LICENSE
README.md
crane
gcrane
krane
0.22.1
0.22.1

install copies, sets the mode and replaces an old version in one step. The checksum proves that the file is the one the project published – not that the project is trustworthy. You verify signatures on Day 14.

Security: curl … | sudo bash from a README skips step 2. In the lab, that does not happen.

Save your progress? Sign up for free

Drill 5: Install all tools

tools.lock has one line per tool and architecture: name, architecture, URL, SHA-256. For every line of your architecture you do the four steps from Drill 4, in a loop. read splits the line into its fields, exit 1 aborts immediately on a wrong checksum. First the tools that ship as a single binary:

A=$(dpkg --print-architecture) && mkdir -p /tmp/dl && cd /tmp/dl
grep -E "^(cosign|ctop|diffoci|hadolint|kind|regctl|regsync) $A " \
  ~/ct-lab/setup/tools.lock | while read -r name _ url sum; do
  curl -fsSLo "$name" "$url"
  echo "$sum  $name" | sha256sum -c || exit 1
  sudo install -m 0755 "$name" /usr/local/bin/
done

Expected:

cosign: OK
ctop: OK
diffoci: OK
hadolint: OK
kind: OK
regctl: OK
regsync: OK

Then the archives in which the binary has the same name as the tool:

grep -E "^(crictl|dive|dockle|grype|ko|lazydocker|oras|syft|trivy) $A " \
  ~/ct-lab/setup/tools.lock | while read -r name _ url sum; do
  curl -fsSLo "$name.tar.gz" "$url"
  echo "$sum  $name.tar.gz" | sha256sum -c || exit 1
  tar -xzf "$name.tar.gz" "$name"
  sudo install -m 0755 "$name" /usr/local/bin/
done

Expected (takes a few minutes, trivy is the largest download):

crictl.tar.gz: OK
dive.tar.gz: OK
dockle.tar.gz: OK
grype.tar.gz: OK
ko.tar.gz: OK
lazydocker.tar.gz: OK
oras.tar.gz: OK
syft.tar.gz: OK
trivy.tar.gz: OK

You installed crane in Drill 4. That leaves nerdctl-full, a bundle that is unpacked entirely into /usr/local:

read -r _ _ url sum < <(grep "^nerdctl-full $A " ~/ct-lab/setup/tools.lock)
curl -fsSLo nerdctl-full.tar.gz "$url"
echo "$sum  nerdctl-full.tar.gz" | sha256sum -c
sudo tar -C /usr/local -xzf nerdctl-full.tar.gz
cd ~ && rm -rf /tmp/dl

Expected:

nerdctl-full.tar.gz: OK

nerdctl-full is a bundle: nerdctl, containerd, buildctl/buildkitd, runc and the CNI plugins end up under /usr/local. None of it is started today – that comes on Day 6.

Save your progress? Sign up for free

Drill 6: Check the ground, note the versions

From now on, every day starts with source ~/ct-lab/setup/env.sh; the file only sets variables. Then a look at the ground:

source ~/ct-lab/setup/env.sh
podman version --format '{{.Client.Version}}'
ls -d "$NOTES/notes-api"
df -h ~ | tail -1

Expected (abridged, free space will differ):

5.7.0
/home/ubuntu/ct-lab/notes/notes-api
/dev/sda1        38G  6.7G   32G  18% /

For Day 13 you need at least 8 GB free (trivy and grype databases). Now the versions of all installed tools – this is the state the book was tested with:

crane version; echo "$(regctl version --format '{{.VCSTag}}')"
echo "$(regsync version --format '{{.VCSTag}}')"; oras version | head -1
cosign version 2>&1 | grep GitVersion; dive --version; diffoci --version
trivy --version | head -1; grype version | grep '^Version'
syft version | grep '^Version'; dockle --version; hadolint --version
nerdctl --version; containerd --version | cut -d' ' -f1,3
buildctl --version | cut -d' ' -f1,3; crictl --version; ko version
lazydocker --version | head -1; ctop -v; kind version | cut -d' ' -f1,2

Expected:

0.22.1
v0.11.7
v0.11.7
Version:        1.3.6
GitVersion:    v3.1.4
dive 0.13.1
diffoci version v0.1.8
Version: 0.76.1
Version:             0.121.0
Version:       1.55.0
dockle version 0.4.15
Haskell Dockerfile Linter 2.15.1
nerdctl version 2.4.2
containerd v2.4.2
buildctl v0.34.1
crictl version v1.37.0
0.19.1
Version: 0.25.2
ctop version 0.7.7, build 11a1cb1 go1.18
kind v0.33.0

On top of that, Ubuntu provides Podman 5.7.0, Buildah 1.42.1, Skopeo 1.21.0, crun 1.21 and pasta. Some tools are missing on purpose: you build container-diff yourself on Day 11 (the project is archived), Docker CE with docker scout, Buildx and Compose comes on Day 13, Colima (Mac) and Scaffdog (in the Node container) on Day 15.

Save your progress? Sign up for free

Drill 7: The demo application notes

cd ~/ct-lab/notes
ls -A . notes-api
grep -h '^FROM' notes-*/Dockerfile
grep -h -E '^(EXPOSE|USER)' notes-*/Dockerfile

Expected:

.:
.dockerignore  .env.example  .gitignore  .hadolint.yaml  README.md  notes-api  notes-web
notes-worker

notes-api:
.dockerignore  Dockerfile  go.mod  go.sum  main.go  main_test.go  metrics.go  store.go
FROM --platform=$BUILDPLATFORM golang:${GO_VERSION}-alpine AS base
FROM base AS test
FROM base AS build
FROM scratch AS final
FROM node:26-alpine AS build
FROM node:26-alpine AS prod
FROM python:${PYTHON_VERSION}-slim AS build
FROM python:${PYTHON_VERSION}-slim
EXPOSE 8080
USER 65532:65532
USER node
EXPOSE 3000
USER 65532:65532
Service Port Important variables Special feature
notes-api (Go, scratch) 8080 NOTES_API_KEY, NOTES_DB_URL migrate, healthcheck
notes-web (Node) 3000 NOTES_API_URL, NOTES_API_KEY proxies /api/… to api
notes-worker (Python) – NOTES_API_URL, INTERVAL summarizes notes
Postgres 18.6-alpine 5432 POSTGRES_PASSWORD volume on /var/lib/postgresql

Without NOTES_DB_URL, notes-api stores everything in memory. The files are named Dockerfile; Podman and Buildah read them just like a Containerfile. Building happens tomorrow – today not a single container is running yet:

podman ps -a && podman images

Expected:

CONTAINER ID  IMAGE       COMMAND     CREATED     STATUS      PORTS       NAMES
REPOSITORY  TAG         IMAGE ID    CREATED     SIZE

Save your progress? Sign up for free

Checkpoint

command -v podman buildah skopeo jq htpasswd crane krane regctl regsync oras cosign \
  dive diffoci trivy grype syft dockle hadolint nerdctl containerd buildctl crictl ko \
  lazydocker ctop kind shellcheck | wc -l

Expected: all 27 tools in the PATH.

27
ls -d ~/ct-lab/code/notes ~/ct-lab/notes/notes-api

Expected:

/home/ubuntu/ct-lab/code/notes
/home/ubuntu/ct-lab/notes/notes-api
podman ps -aq | wc -l

Expected: no container yet.

0

Challenge

Prove that the loop from Drill 5 rejects a tampered file: with a copy of tools.lock in which the checksum for crane is wrong, crane must not be unpacked, and the loop must end with a non-zero exit code.

Hint: sed writes the modified copy, and the loop then reads the copy instead of the original.

Solution

A=$(dpkg --print-architecture) && mkdir -p /tmp/dl && cd /tmp/dl
sed "/^crane $A /s/[0-9a-f]\{64\}\$/$(printf '0%.0s' {1..64})/" \
  ~/ct-lab/setup/tools.lock > tools.lock
grep "^crane $A " tools.lock | while read -r name _ url sum; do
  curl -fsSLo "$name.tar.gz" "$url"
  echo "$sum  $name.tar.gz" | sha256sum -c || exit 1
  tar -xzf "$name.tar.gz" crane krane
done; echo "Exit: $?"
ls
cd ~ && rm -rf /tmp/dl

Expected:

crane.tar.gz: FAILED
sha256sum: WARNING: 1 computed checksum did NOT match
Exit: 1
crane.tar.gz  tools.lock

The abort happens before tar and sudo install: there is no unpacked crane, and the installed version stays untouched.

Save your progress? Sign up for free

Enterprise Note

  • In production, release binaries come from an internal mirror (Artifactory, Nexus, your own registry with oras, Day 9), not freshly from GitHub on every installation.
  • A lock file with checksums belongs in Git and is updated via pull request, for example by Renovate – with a review of who approves which version.
  • Where projects sign their releases (cosign, regclient), also verify the signature: it proves the origin, the checksum only the integrity (Day 14).
  • Golden images or Ansible roles instead of manual installation: every machine gets the same versions, and an inventory knows where which version runs.
  • Ubuntu packages lag behind upstream, but they receive security backports. For Podman on servers that is usually the better trade-off than the newest version.