Skip to content
Hands-on IT IT Drills

Day 13: Compose in CI

Goal: The same stack runs in a pipeline: build, unit tests, start, test against the running stack, pass the exit code through, clean up without a trace – in parallel, too. · Duration: approx. 75 min · Prerequisite: Day 12

Briefing

  • compose.ci.yaml is layered over compose.yaml: no ports to the outside, but instead a service tests that checks api and web from within the network frontend.
  • up --wait is the “stack healthy” gate, run --rm tests the test run. The test’s exit code becomes the pipeline’s exit code.
  • Every run gets its own project name with -p. That way two pipelines share one Docker daemon without seeing each other.
  • Cleanup always happens, even on red: trap with down -v.
 ci/pipeline.sh   -p ci-<PID>
  ├─ lint          up --dry-run
  ├─ unit tests    --profile test build api-test
  ├─ stack         up -d --build --wait
  ├─ tests         run --rm tests ──► api:8080, web:3000
  └─ trap EXIT     down -v --remove-orphans, delete images ci-<PID>-*

Drill 1: Ports out with !reset

source ~/compose-lab/setup/env.sh
docker compose -p notes down
mkdir -p ~/compose-lab/tag13 && cd ~/compose-lab/tag13
cp -R "$LAB/code/notes/." . && cp -R "$LAB/code/tag12/notes/." .

On a CI runner, several pipelines often run side by side. A published port like 127.0.0.1:8080 can only be taken by one of them. The tests run inside the Compose network anyway, so they don’t need ports:

cat > compose.ci.yaml <<'EOF'
# compose.ci.yaml – CI, only via -f together with compose.yaml (Day 13)
services:
  api:
    ports: !reset []
  proxy:
    ports: !reset []
  tests:
    image: curlimages/curl:8.23.0
    profiles: [ci]
    entrypoint: ["/bin/sh", "/tests/integration.sh"]
    volumes:
      - ./tests:/tests:ro
    environment:
      API_URL: http://api:8080
      WEB_URL: http://web:3000
      NOTES_API_KEY_FILE: /run/secrets/api_key
    secrets: [api_key]
    networks: [frontend]
    depends_on:
      api:
        condition: service_healthy
      web:
        condition: service_healthy
EOF

!reset [] empties the list from compose.yaml instead of extending it (Day 7). The profile ci keeps tests out of a normal up: without a profile, up --wait would start the test container too, and it would run once alongside every up. run starts a service with a profile even without --profile.

COMPOSE_FILE saves you the double -f for the rest of the day (Day 7):

export COMPOSE_FILE=compose.yaml:compose.ci.yaml
docker compose up -d --build --wait --quiet-pull --quiet-build
docker compose ps --format 'table {{.Service}}\t{{.Ports}}'
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/healthz

Expected (shortened, IDs and times differ on your machine):

 Image notes-worker:dev Built
 Image notes-api:dev Built
 Image notes-web:dev Built
 Container notes-db-1 Healthy
 Container notes-migrate-1 Exited
 Container notes-api-1 Healthy
 Container notes-web-1 Healthy
 Container notes-worker-1 Healthy
 Container notes-proxy-1 Healthy
SERVICE   PORTS
api       8080/tcp
db        5432/tcp
proxy     80/tcp, 443/tcp, 2019/tcp, 443/udp
web       3000/tcp
worker
000

No more 127.0.0.1:…->… lines: the ports are only open inside the Compose network; curl from the Mac gets no connection (000). --quiet-pull and --quiet-build keep the CI log short; errors still show up.

Save your progress? Sign up for free

Drill 2: Integration tests as a service

The test script runs in the image curlimages/curl – POSIX sh and curl, nothing else:

mkdir -p tests
cat > tests/integration.sh <<'EOF'
#!/bin/sh
# integration.sh – integration tests against the running stack (Day 13)
# Runs in the service tests (curlimages/curl). Exit code = number of failures.
API_URL=${API_URL:-http://api:8080}
WEB_URL=${WEB_URL:-http://web:3000}
KEY=$(cat "${NOTES_API_KEY_FILE:-/run/secrets/api_key}") || exit 1
TEXT="ci-$(date +%s)-$$"
JSON='Content-Type: application/json'
fail=0

ok()  { echo "✔ $1"; }
bad() { echo "✘ $1"; fail=$((fail + 1)); }
# status <method> <url> [curl options…] – prints only the HTTP status
status() {
  m=$1 u=$2; shift 2
  curl -s -o /dev/null -w '%{http_code}' -X "$m" "$@" "$u"
}
# expect <name> <actual> <expected>
expect() {
  if [ "$2" = "$3" ]; then ok "$1"; else bad "$1 (erwartet $3, bekommen $2)"; fi
}

expect "api /healthz" "$(status GET "$API_URL/healthz")" 200
expect "web /healthz" "$(status GET "$WEB_URL/healthz")" 200
expect "POST ohne Key → 401" \
  "$(status POST "$API_URL/notes" -H "$JSON" -d '{"text":"x"}')" 401
expect "POST mit Key → 201" "$(status POST "$API_URL/notes" -H "$JSON" \
  -H "X-API-Key: $KEY" -d "{\"text\":\"$TEXT\"}")" 201
if curl -s "$API_URL/notes" | grep -q "$TEXT"; then
  ok "GET /notes enthält die Notiz"
else
  bad "GET /notes enthält die Notiz nicht"
fi
expect "POST über web → 201" \
  "$(status POST "$WEB_URL/api/notes" -H "$JSON" -d '{"text":"via web"}')" 201
if curl -s "$API_URL/metrics" | grep -q '^notes_created_total [1-9]'; then
  ok "/metrics zählt notes_created_total"
else
  bad "/metrics ohne notes_created_total > 0"
fi
echo "Fehler: $fail"
exit "$fail"
EOF
chmod +x tests/integration.sh
shellcheck tests/integration.sh; echo "Exit: $?"
docker compose run --rm tests; echo "Exit: $?"

Expected (shortened):

Exit: 0
 Container notes-api-1 Running
 Container notes-db-1 Running
 Container notes-web-1 Running
 Container notes-db-1 Healthy
 Container notes-migrate-1 Started
 Container notes-migrate-1 Exited
 Container notes-api-1 Healthy
 Container notes-web-1 Healthy
 Container notes-tests-run-ed87b0979ea4 Created
✔ api /healthz
✔ web /healthz
✔ POST ohne Key → 401
✔ POST mit Key → 201
✔ GET /notes enthält die Notiz
✔ POST über web → 201
✔ /metrics zählt notes_created_total
Fehler: 0
Exit: 0

Like up, run waits for the depends_on conditions – and in the process also starts migrate once more (idempotent, Day 5). The test container is called notes-tests-run-… and is gone after the run thanks to --rm. Every note gets a unique text so that a hit from an earlier run doesn’t count as a success.

Save your progress? Sign up for free

Drill 3: Red must be red

A test that never fails isn’t a test. run -e overrides a variable only for this run – here the key points to an empty file:

docker compose run --rm -e NOTES_API_KEY_FILE=/dev/null tests 2>/dev/null
echo "Exit: $?"
docker compose ps -a --format '{{.Service}}' | sort | tr '\n' ' '; echo

Expected:

✔ api /healthz
✔ web /healthz
✔ POST ohne Key → 401
✘ POST mit Key → 201 (erwartet 201, bekommen 401)
✘ GET /notes enthält die Notiz nicht
✔ POST über web → 201
✔ /metrics zählt notes_created_total
Fehler: 2
Exit: 2
api db migrate proxy web worker

The container’s exit code arrives unchanged on your side – exactly what the pipeline needs. No tests-run-… is left behind.

Save your progress? Sign up for free

Drill 4: --exit-code-from and the migrate trap

The alternative to up -d plus run: start everything in the foreground and take over the exit code of the test service. According to docker compose up --help, --exit-code-from implies the option --abort-on-container-exit – “Stops all containers if any container was stopped”. Try it with the whole stack:

docker compose down
docker compose --profile ci up --exit-code-from tests
# after half a minute without progress: Ctrl-C
echo "Exit: $?"
docker compose --profile ci ps -a --format 'table {{.Service}}\t{{.Status}}'

Expected (shortened):

Attaching to api-1, db-1, migrate-1, proxy-1, tests-1, web-1, worker-1
…
migrate-1  | time=… level=INFO msg="migration complete" table=notes
migrate-1 exited with code 0
 Compose Stopping Aborting on container exit...
 …
db-1 exited with code 0
 Container notes-db-1 Error dependency db failed to start
^C
Exit: 130
SERVICE   STATUS
api       Created
db        Exited (0) About a minute ago
migrate   Exited (0) About a minute ago
proxy     Created
tests     Created
web       Created
worker    Created

The one-off job migrate ends as planned with 0 – for --abort-on-container-exit that’s enough to stop everything. The tests never ran, and Compose then hung until you aborted. In a CI pipeline that would be a job running until the timeout. The solution: name only the test service. Compose starts its dependencies, but only attaches to tests:

docker compose up --exit-code-from tests tests; echo "Exit: $?"
docker compose --profile ci ps -a --format 'table {{.Service}}\t{{.Status}}'

Expected (shortened):

Attaching to tests-1
 Container notes-tests-1 Started
tests-1  | ✔ api /healthz
tests-1  | ✔ web /healthz
tests-1  | ✔ POST ohne Key → 401
tests-1  | ✔ POST mit Key → 201
tests-1  | ✔ GET /notes enthält die Notiz
tests-1  | ✔ POST über web → 201
tests-1  | ✔ /metrics zählt notes_created_total
tests-1  | Fehler: 0
tests-1 exited with code 0
 Compose Stopping Aborting on container exit...
 Container notes-tests-1 Stopped
Exit: 0
SERVICE   STATUS
api       Up 3 seconds (healthy)
db        Up 5 seconds (healthy)
migrate   Exited (0) 4 seconds ago
tests     Exited (0) Less than a second ago
web       Up 2 seconds (healthy)

worker and proxy are missing – they aren’t dependencies of tests. That’s why the pipeline sticks with up -d --wait for the whole stack and run --rm tests. The variant --abort-on-container-failure only stops on a non-zero exit; if the test ends green, up keeps running until someone aborts it.

Save your progress? Sign up for free

Drill 5: Output for CI logs

--progress is a global option (before the subcommand) and, according to --help, supports auto, tty, plain, json and quiet:

docker compose down >/dev/null 2>&1
for p in plain json quiet; do
  docker compose --progress "$p" up -d --wait > "p-$p.log" 2>&1
  echo "$p: Exit $? · $(wc -l < "p-$p.log" | tr -d ' ') Zeilen"
  docker compose down >/dev/null 2>&1
done
head -3 p-plain.log; head -2 p-json.log; cat p-quiet.log; rm p-*.log

Expected:

plain: Exit 0 · 60 Zeilen
json: Exit 0 · 60 Zeilen
quiet: Exit 0 · 0 Zeilen
 Network notes_frontend Creating
 Network notes_backend Creating
 Network notes_backend Creating
{"id":"Network notes_backend","status":"Working","text":"Creating"}
{"id":"Network notes_frontend","status":"Working","text":"Creating"}

Without a terminal, Compose 5.6 writes some lines twice (here notes_backend); in this book the outputs are therefore de-duplicated. json is suited for filtering with jq, quiet for steps where the exit code is enough. You turn off colors with the global option --ansi never or, for up, with --no-color.

Save your progress? Sign up for free

Drill 6: Parallel pipelines with -p

Two pipelines on the same daemon – that’s everyday life on self-hosted runners:

for p in ci-a ci-b; do docker compose -p "$p" --progress quiet up -d --wait & done
wait; echo "beide oben"
docker compose ls --filter name=ci-
docker network ls --filter name=ci- --format '{{.Name}}' | sort
docker volume ls --filter name=ci- --format '{{.Name}}' | sort
docker image ls --format '{{.Repository}}:{{.Tag}}' | grep '^ci-' | sort
for p in ci-a ci-b; do
  echo "$p: $(docker compose -p "$p" run --rm tests 2>/dev/null | tail -1)"
done

Expected (shortened):

beide oben
NAME   STATUS       CONFIG FILES
ci-a   running(5)   /Users/du/compose-lab/tag13/compose.yaml,/Users/du/…/compose.ci.yaml
ci-b   running(5)   /Users/du/compose-lab/tag13/compose.yaml,/Users/du/…/compose.ci.yaml
ci-a_backend
ci-a_frontend
ci-a_monitoring
ci-b_backend
ci-b_frontend
ci-b_monitoring
ci-a_caddy_config
ci-a_caddy_data
ci-a_pgdata
ci-b_caddy_config
ci-b_caddy_data
ci-b_pgdata
ci-a-api:dev
ci-a-web:dev
ci-a-worker:dev
ci-b-api:dev
ci-b-web:dev
ci-b-worker:dev
ci-a: Fehler: 0
ci-b: Fehler: 0

The project name is part of everything: containers, networks, volumes – and thanks to image: ${COMPOSE_PROJECT_NAME}-api:${TAG:-dev} (Day 8) the images too. -p sets this value for interpolation, even when Compose hands the build over to Bake (verified live). Two databases, two networks, no contact. And without compose.ci.yaml?

docker compose -f compose.yaml up -d --wait >/dev/null 2>&1   # notes on port 8080
docker compose -p ci-c -f compose.yaml up -d --wait 2>&1 | grep 'port is'

Expected (shortened):

Error response from daemon: failed to set up container networking: driver failed
programming external connectivity on endpoint ci-c-api-1 (…): Bind for 127.0.0.1:8080
failed: port is already allocated

Clean up – volumes and the project’s own images are part of it:

for p in ci-a ci-b ci-c; do docker compose -p "$p" --progress quiet down -v; done
docker image ls --format '{{.Repository}}:{{.Tag}}' | grep '^ci-' \
  | xargs docker image rm | grep -c Untagged

Expected:

9

down --rmi all would be wrong here: it would also remove postgres:18.6-alpine and caddy:2.11.7-alpine, which other projects still need.

Save your progress? Sign up for free

Drill 7: The pipeline as a script

What was manual work just now goes into a script that runs the same locally and on the runner:

mkdir -p ci
cat > ci/pipeline.sh <<'EOF'
#!/usr/bin/env bash
# pipeline.sh – the CI pipeline of notes, locally as on the runner (Day 13)
# Usage: ci/pipeline.sh [project-name]   default: ci-<PID>, unique per run
set -euo pipefail
cd "$(dirname "$0")/.."
P="${1:-ci-$$}"
dc() { docker compose -p "$P" -f compose.yaml -f compose.ci.yaml "$@"; }

# shellcheck disable=SC2329  # called via trap
cleanup() {
  echo "--- Aufräumen: $P"
  dc down -v --remove-orphans >/dev/null 2>&1 || true
  docker image ls --format '{{.Repository}}:{{.Tag}}' | { grep "^$P-" || true; } \
    | xargs -r docker image rm >/dev/null 2>&1 || true
}
trap cleanup EXIT

# Lab secrets, in case they're missing (fresh checkout). On a real runner the
# values come from the CI platform's secret store, never from the repository.
lab_secret() { [ -s "secrets/$1" ] || printf '%s' "$2" > "secrets/$1"; }
mkdir -p secrets
lab_secret api_key.txt lab-api-key-123
lab_secret db_password.txt lab-db-pw-123
lab_secret db_url.txt postgres://notes:lab-db-pw-123@db:5432/notes
lab_secret grafana_admin.txt lab-grafana-pw-123
lab_secret npmrc "$(printf '%s\n' '@notes-lab:registry=https://npm.lab.invalid/' \
  '//npm.lab.invalid/:_authToken=lab-npm-token-123')"

echo "--- Lint: $P"
# load and validate the model: a dry run, creates nothing and pulls nothing
if ! out=$(dc --dry-run up -d --no-build --pull never 2>&1); then
  echo "$out"; exit 1
fi
echo "--- Unit-Tests"
dc --profile test build --quiet api-test
echo "--- Stack starten"
dc up -d --build --wait --wait-timeout 180 --quiet-pull --quiet-build
echo "--- Integrationstests"
rc=0
dc run --rm --quiet-pull tests || rc=$?
echo "--- Ergebnis: Exit $rc"
exit "$rc"
EOF
chmod +x ci/pipeline.sh
shellcheck ci/pipeline.sh; echo "Exit: $?"
unset COMPOSE_FILE
ci/pipeline.sh > pipeline.log 2>&1; echo "Exit: $?"
grep -E '^---|Built|Healthy|Exited|✔|✘|Fehler' pipeline.log

Expected (shortened, the PID differs):

Exit: 0
Exit: 0
--- Lint: ci-62179
--- Unit-Tests
 Image ci-62179-api:test Built
--- Stack starten
 Image ci-62179-api:dev Built
 Image ci-62179-web:dev Built
 Image ci-62179-worker:dev Built
 Container ci-62179-db-1 Healthy
 Container ci-62179-migrate-1 Exited
 Container ci-62179-api-1 Healthy
 Container ci-62179-web-1 Healthy
 Container ci-62179-worker-1 Healthy
 Container ci-62179-proxy-1 Healthy
--- Integrationstests
✔ api /healthz
✔ web /healthz
✔ POST ohne Key → 401
✔ POST mit Key → 201
✔ GET /notes enthält die Notiz
✔ POST über web → 201
✔ /metrics zählt notes_created_total
Fehler: 0
--- Ergebnis: Exit 0
--- Aufräumen: ci-62179

Three decisions in the script:

  • trap cleanup EXIT runs on green, on red and on an abort in the middle of the script (set -e). The run leaves behind neither containers nor volumes nor images.
  • The lint is a dry run: --dry-run loads and checks the complete model (schema, interpolation, !reset) without creating anything. A typo like imagee: ends with services.tests.imagee false schema and exit code 1 (verified live).
  • The tests run against exactly the images that the previous step built – ci-62179-api:dev, not some latest.

The usual lint in pipelines is docker compose config -q: it prints nothing and ends with exit code 1 for an invalid file. Without -q, config prints the resolved model – with all interpolated values (Day 4) – into the CI log.

docker compose -f compose.yaml -f compose.ci.yaml config -q; echo "Exit: $?"

With valid files, the output stays empty except for Exit: 0. On an error, the message goes to stderr, and the exit code is 1.

Save your progress? Sign up for free

Drill 8: GitHub Actions and GitLab CI

Both platforms just call the script. The files are ready in the companion code; they were written against the documentation but haven’t run on real runners.

cp "$LAB/code/tag13/notes/ci/github-actions.yml" \
  "$LAB/code/tag13/notes/ci/gitlab-ci.yml" ci/
cat ci/github-actions.yml

Expected:

# ci/github-actions.yml – Beispiel (Tag 13), nicht ausgeführt.
# Im Repository als .github/workflows/ci.yml ablegen.
name: ci
on:
  push:
  pull_request:

jobs:
  integration:
    runs-on: ubuntu-latest        # Docker und das Compose-Plugin sind vorinstalliert
    timeout-minutes: 15           # ein hängendes up darf den Runner nicht blockieren
    steps:
      - uses: actions/checkout@v7
      - name: Secrets aus dem Secret-Store
        env:
          NOTES_API_KEY: ${{ secrets.NOTES_API_KEY }}
        run: |
          mkdir -p secrets
          printf '%s' "$NOTES_API_KEY" > secrets/api_key.txt
      - name: Pipeline
        run: ci/pipeline.sh "ci-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
cat ci/gitlab-ci.yml

Expected:

# ci/gitlab-ci.yml – Beispiel (Tag 13), nicht ausgeführt.
# Im Repository als .gitlab-ci.yml ablegen. Docker-Executor mit Docker-in-Docker:
# jeder Job hat seinen eigenen Docker-Daemon im Dienst docker.
integration:
  image: docker:29.9-cli
  services:
    - docker:29.9-dind
  variables:
    DOCKER_TLS_CERTDIR: "/certs"
  timeout: 15 minutes
  before_script:
    - apk add --no-cache bash
    - mkdir -p secrets
    - printf '%s' "$NOTES_API_KEY" > secrets/api_key.txt
  script:
    - ci/pipeline.sh "ci-${CI_JOB_ID}"

What you take away from this:

Point GitHub Actions GitLab CI
Docker daemon preinstalled on the runner service docker:29.9-dind, one per job
Timeout timeout-minutes timeout
Project name GITHUB_RUN_ID + attempt CI_JOB_ID
Secret secrets.NOTES_API_KEY masked CI/CD variable NOTES_API_KEY

Warning: With GitLab and Docker-in-Docker, according to the docs, the job container and the dind service don’t share a root file system. Bind mounts like ./tests and the file: secrets only work because the project lives in the build directory (/builds/…), which both can see. A path outside of it is empty or missing in the container – the same trap as with Colima and /tmp (Day 0) and as on the server tomorrow (Day 14).

The image docker:29.9-cli is Alpine without bash, hence apk add bash. The script wouldn’t need after_script: according to the GitLab docs, it doesn’t run on a timeout anyway, whereas the trap in the script runs whenever the script itself ends.

Save your progress? Sign up for free

Checkpoint

The checkpoint starts the normal stack on port 8080 and runs the pipeline next to it. First the CI model:

cd ~/compose-lab/tag13
unset COMPOSE_FILE
docker compose -f compose.yaml -f compose.ci.yaml config -q; echo "Exit: $?"

Expected:

Exit: 0

The development stack is running:

docker compose -f compose.yaml up -d --wait

Expected (shortened):

 Container notes-db-1 Healthy
…
 Container notes-proxy-1 Healthy
 Container notes-web-1 Healthy

The pipeline next to it, with its own project name:

ci/pipeline.sh ci-kp > ci-kp.log 2>&1; echo "Exit: $?"
grep 'Fehler:' ci-kp.log

Expected:

Exit: 0
Fehler: 0

Nothing is left of ci-kp:

docker ps -aq --filter label=com.docker.compose.project=ci-kp | wc -l
docker volume ls -q --filter label=com.docker.compose.project=ci-kp | wc -l
docker image ls -q --filter reference='ci-kp-*' | wc -l

Expected:

0
0
0

You’ll find the complete state in code/tag13/notes/ (compose.ci.yaml, tests/integration.sh, ci/).

Challenge

Run two pipelines at the same time: one from tag13, one from a copy that contains a configuration error – web no longer gets its API key. One must end green, the other red, and afterwards nothing may be left of either.

Hint: web’s wrapper reads /run/secrets/api_key (Day 10); a typo in the file name is enough, and you pass the project names to pipeline.sh as an argument.

Solution

cd ~/compose-lab
rm -rf tag13-rot && cp -R tag13 tag13-rot
sed -i '' 's#secrets/api_key)#secrets/apikey)#' tag13-rot/compose.yaml
grep -n 'apikey' tag13-rot/compose.yaml
tag13/ci/pipeline.sh ci-gruen > gruen.log 2>&1 & g=$!
tag13-rot/ci/pipeline.sh ci-rot > rot.log 2>&1 & r=$!
wait $g; echo "grün: Exit $?"; wait $r; echo "rot: Exit $?"
grep -E '✘|Fehler:|Ergebnis' rot.log
for p in ci-gruen ci-rot; do
  f="label=com.docker.compose.project=$p"
  echo "$p: $(docker ps -aq --filter "$f" | wc -l | tr -d ' ') Container," \
    "$(docker volume ls -q --filter "$f" | wc -l | tr -d ' ') Volumes"
done

Expected:

107:    command: ['NOTES_API_KEY="$$(cat /run/secrets/apikey)" exec node server.js']
grün: Exit 0
rot: Exit 1
✘ POST über web → 201 (erwartet 201, bekommen 401)
Fehler: 1
--- Ergebnis: Exit 1
ci-gruen: 0 Container, 0 Volumes
ci-rot: 0 Container, 0 Volumes

The error is insidious: web starts and is healthy, because /healthz doesn’t need a key. Only the integration test finds it. Both runs shared the daemon, base images and build cache, but no network and no volume. Clean up:

rm -rf ~/compose-lab/tag13-rot ~/compose-lab/gruen.log ~/compose-lab/rot.log
cd ~/compose-lab/tag13 && docker compose down

Save your progress? Sign up for free

Enterprise Note

  • On shared runners, a unique project name is mandatory, and so is cleanup. Better is a dedicated daemon per job (GitLab dind, short-lived VMs); dind, however, requires privileged – a security question on shared Kubernetes runners.
  • Build once, then push the same image (by digest) through test, staging and production. Today’s pipeline rebuilds on every run – fine for a lab.
  • On red, save the logs of all services before down deletes them (docker compose logs --no-color > stack.log in the trap), and upload them as an artifact (GitLab: artifacts: when: always).
  • Secrets come from the platform’s secret store, masked and only for protected branches. config without -q and set -x don’t belong in a CI log.
  • A timeout per job (timeout-minutes, timeout) and --wait-timeout for up prevent runs that block a runner for hours (Drill 4).