Day 4: Configuration
Goal: Know where every value in Compose and in the container comes from, enforce required values and give Postgres an init script via
configs:on its first start. · Duration: approx. 60 min · Prerequisite: Day 3
Briefing
- Two levels: Compose substitutes
${VAR}while reading the file (interpolation). Only what is listed underenvironment:orenv_file:gets into the container. .envin the project folder is used for interpolation only. The shell beats.env,--env-filereplaces.env.${VAR:-default}provides a default value,${VAR:?message}aborts if the value is missing or empty.$$is a dollar sign for the container.configs:put files into the container – from a file (file:) or directly from the Compose file (content:).
shell variables ─┐ ┌─ environment: ─┐
--env-file / .env┴─► ${…} in compose.yaml ─┤ ├─► container
(interpolation) └─ env_file: ────┘
db/init/10-readonly.sql ── configs: ──► /docker-entrypoint-initdb.d/
Drill 1: .env is for Compose, not for containers
Start of the day with the state of Day 3. Today you start with an
empty database (-v), because an init
script is supposed to run in Drill 7. Your backup from yesterday is in
~/compose-lab/tag03/backup.
source ~/compose-lab/setup/env.sh
docker compose -p notes down -v
mkdir -p ~/compose-lab/tag04 && cd ~/compose-lab/tag04
cp -R "$LAB/code/notes/." . && cp -R "$LAB/code/tag03/notes/." .
docker compose up -d db
until docker compose exec db pg_isready -h 127.0.0.1 -U notes -d notes >/dev/null; do
sleep 1
done
docker compose run --rm api migrate
docker compose up -d
sleep 3
curl -s -X POST localhost:8080/notes -H 'X-API-Key: lab-api-key-123' \
-H 'Content-Type: application/json' -d '{"text":"Tag 4 startet mit Daten"}'; echoExpected (shortened, IDs and times differ on your machine):
Container notes-db-1 Started
time=… level=INFO msg="migration complete" table=notes
Container notes-worker-1 Started
{"id":"8fdb2620-…","text":"Tag 4 startet mit Daten"}
What is in .env, and which of it does a container
see?
grep -v '^#' .env
docker compose exec worker printenv NOTES_API_KEY NOTES_API_URL
docker compose exec worker printenv POSTGRES_PASSWORD; echo "Exit: $?"
docker compose exec db printenv POSTGRES_PASSWORD NOTES_API_KEY; echo "Exit: $?"Expected:
NOTES_API_KEY=lab-api-key-123
POSTGRES_PASSWORD=lab-db-pw-123
lab-api-key-123
http://api:8080
Exit: 1
lab-db-pw-123
Exit: 1
worker knows the key because environment: takes it over
via ${NOTES_API_KEY} – not the password. db knows the
password, not the key. .env itself ends up in no
container.
Security: Whatever is in
environment:can be read by anyone with access to Docker:docker compose exec … printenv,docker inspectanddocker compose config, which prints the file with all substituted values. Such output does not belong in tickets or CI logs. Day 10 replaces the variables with secrets.
Save your progress? Sign up for free
Drill 2: Default values and required variables
First the problem: an empty variable in the shell beats
.env, and Compose substitutes an empty string without
warning.
NOTES_API_KEY= docker compose up -d worker
sleep 3
docker compose ps -a --format '{{.Service}}\t{{.Status}}'
docker compose logs --no-log-prefix worker api | tail -2Expected (shortened, times differ):
Container notes-api-1 Recreated
Container notes-worker-1 Recreated
Container notes-api-1 Started
Container notes-worker-1 Started
api Exited (1) 3 seconds ago
db Up 40 seconds
web Up 38 seconds
worker Exited (1) 2 seconds ago
time=… level=ERROR msg="start abgebrochen" err="NOTES_API_KEY oder NOTES_API_KEY_FILE
fehlt"
NOTES_API_KEY oder NOTES_API_KEY_FILE fehlt
api and worker crashed (“start aborted”, “NOTES_API_KEY or
NOTES_API_KEY_FILE missing”); the error only shows up at runtime. It’s
better if Compose aborts beforehand. The new file uses :?
everywhere and makes the worker interval configurable:
cat > compose.yaml <<'EOF'
# compose.yaml – Day 4 (Drill 2; configs follow in Drill 6)
name: notes
services:
api:
build: notes-api
environment:
NOTES_API_KEY: ${NOTES_API_KEY:?NOTES_API_KEY fehlt (.env)}
NOTES_DB_URL: "postgres://notes:\
${POSTGRES_PASSWORD:?POSTGRES_PASSWORD fehlt (.env)}@db:5432/notes"
ports:
- "127.0.0.1:${API_PORT:-8080}:8080"
networks: [frontend, backend]
depends_on:
- db
worker:
build: notes-worker
environment:
NOTES_API_URL: http://api:8080
NOTES_API_KEY: ${NOTES_API_KEY:?NOTES_API_KEY fehlt (.env)}
INTERVAL: ${WORKER_INTERVAL:-10}
depends_on:
- api
networks: [backend]
web:
build: notes-web
environment:
NOTES_API_URL: http://api:8080
NOTES_API_KEY: ${NOTES_API_KEY:?NOTES_API_KEY fehlt (.env)}
ports:
- "127.0.0.1:${WEB_PORT:-3000}:3000"
depends_on:
- api
networks: [frontend]
db:
image: postgres:18.6-alpine
environment:
POSTGRES_USER: notes
POSTGRES_DB: notes
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?POSTGRES_PASSWORD fehlt (.env)}
volumes:
- pgdata:/var/lib/postgresql
networks: [backend]
networks:
frontend: {}
backend:
internal: true
volumes:
pgdata: {}
EOF
NOTES_API_KEY= docker compose up -d worker
docker compose up -dExpected (shortened):
error while interpolating services.api.environment.NOTES_API_KEY: required variable
NOTES_API_KEY is missing a value: NOTES_API_KEY fehlt (.env)
…
Container notes-api-1 Recreated
Container notes-worker-1 Recreated
Container notes-api-1 Started
Container notes-worker-1 Started
With an empty key nothing starts; the message names the service, the
variable and your hint (“NOTES_API_KEY missing (.env)”). The second call
uses .env again and recreates only api and worker – web and
db haven’t changed. The DB URL is a YAML string in double quotes:
\ at the end of a line joins the lines without a space;
Compose interpolates afterwards.
| Expression | Variable missing | Variable empty |
|---|---|---|
${VAR:-std} |
std |
std |
${VAR-std} |
std |
empty |
${VAR:?text} |
abort with text |
abort with text |
${VAR?text} |
abort with text |
empty |
Now the default value:
WORKER_INTERVAL=3 docker compose up -d worker
sleep 7
docker compose logs --no-log-prefix --since 10s worker
WORKER_INTERVAL= docker compose up -d worker
docker compose logs --no-log-prefix worker | grep started | tail -1Expected (shortened):
Container notes-worker-1 Recreated
worker started, api=http://api:8080, interval=3s
summary: 1 notes, 5 words
summary: 1 notes, 5 words
summary: 1 notes, 5 words
Container notes-worker-1 Recreated
worker started, api=http://api:8080, interval=10s
With :-, the default value also applies to an empty
variable. ${WORKER_INTERVAL-10} would have produced an
empty string, and the worker would have failed on
int("").
Save your progress? Sign up for free
Drill 3: Who wins during interpolation?
According to the Compose docs, this applies when substituting
${…}:
- shell variables,
- otherwise files from
--env-file, - without
--env-file, the.envin the project folder (--project-directory, otherwise the folder of the first-ffile, otherwise the current directory).
echo 'WORKER_INTERVAL=5' > interval.env
docker compose --env-file interval.env up -d worker
docker compose --env-file .env --env-file interval.env up -d worker
docker compose logs --no-log-prefix worker | grep started | tail -1
WORKER_INTERVAL=7 docker compose --env-file .env --env-file interval.env up -d worker
docker compose logs --no-log-prefix worker | grep started | tail -1Expected (shortened):
error while interpolating services.api.environment.NOTES_API_KEY: required variable
NOTES_API_KEY is missing a value: NOTES_API_KEY fehlt (.env)
…
Container notes-worker-1 Recreated
worker started, api=http://api:8080, interval=5s
Container notes-worker-1 Recreated
worker started, api=http://api:8080, interval=7s
- Only
--env-file interval.env:.envis no longer read, the key is missing – your:?from Drill 2 catches that. - Compose reads multiple
--env-filein order; later ones override earlier ones. - The shell beats every file. In CI that’s handy: values come as environment variables from the pipeline’s secret store.
Clean up:
rm interval.env
docker compose up -d workerTip:
docker compose config --environmentshows which variables Compose uses for interpolation. The output also contains the values from.env.
Save your progress? Sign up for free
Drill 4: Who wins inside the container?
For the environment inside the container, the docs give this order (top wins):
| Rank | Source | Example |
|---|---|---|
| 1 | docker compose run -e |
run -e INTERVAL=1 |
| 2 | environment:/env_file: with an
interpolated value |
INTERVAL: ${WORKER_INTERVAL:-10} |
| 3 | environment: with a fixed value |
NOTES_API_URL: http://api:8080 |
| 4 | env_file: |
INTERVAL=2 in worker.env |
| 5 | ENV in the image |
ENV PATH=… in the Dockerfile |
To try it out, worker temporarily gets an env_file::
printf 'INTERVAL=2\nNOTES_HINT=aus worker.env\n' > worker.envIn the service worker, add the line
env_file: worker.env directly below
build: notes-worker. Then:
docker compose run --rm --no-deps --entrypoint printenv worker INTERVAL NOTES_HINT
docker compose run --rm --no-deps -e INTERVAL=1 --entrypoint printenv worker INTERVAL
WORKER_INTERVAL=4 docker compose run --rm --no-deps --entrypoint printenv \
worker INTERVALExpected (shortened, without the Container … Created
lines):
10
aus worker.env
1
4
environment: beats env_file: (10 instead of
2); what is only in worker.env still arrives (“aus
worker.env” = “from worker.env”). run -e beats everything.
--entrypoint printenv replaces the image’s
ENTRYPOINT ["notes-worker"], --no-deps doesn’t
start api as well.
Revert: delete the line env_file: worker.env again,
then
rm worker.envThe final state has no env_file:: a second source for
the same variables only makes troubleshooting harder.
Save your progress? Sign up for free
Drill 5: $$ – a dollar sign for the container
A mini project with just one service shows who replaces which
$:
mkdir -p dollar && cd dollar
cat > compose.yaml <<'EOF'
# dollar/compose.yaml – Day 4, Drill 5: who replaces the dollar sign?
name: notes-dollar
services:
hallo:
image: alpine:3.24
command: ["sh", "-c", "echo Compose: $HOME / Container: $$HOME"]
EOF
docker compose run --rm hallo
docker compose down
cd ..Expected (shortened, IDs differ):
Network notes-dollar_default Created
Container notes-dollar-hallo-run-c83f47aef9a8 Created
Compose: /Users/du / Container: /root
Network notes-dollar_default Removed
Compose replaced $HOME with your home directory on the
Mac while reading the file. $$HOME became
$HOME, and only the shell inside the container substituted
it. You need this for healthchecks with CMD-SHELL (Day 5)
and for wrapper scripts (Day 10).
Save your progress? Sign up for free
Drill 6: configs – an init script for Postgres
On its first start, the Postgres image runs all
*.sql and *.sh from
/docker-entrypoint-initdb.d/. The script creates a
read-only role:
mkdir -p db/init
cat > db/init/10-readonly.sql <<'EOF'
-- Läuft nur beim ersten Start mit leerem Volume (docker-entrypoint-initdb.d)
CREATE ROLE notes_ro NOLOGIN;
GRANT pg_read_all_data TO notes_ro;
EOFIn compose.yaml, db gets the config, and a
top-level block is added at the end of the file (whole file:
code/tag04/notes/compose.yaml):
db:
image: postgres:18.6-alpine
environment:
POSTGRES_USER: notes
POSTGRES_DB: notes
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?POSTGRES_PASSWORD fehlt (.env)}
volumes:
- pgdata:/var/lib/postgresql
configs:
- source: db_init
target: /docker-entrypoint-initdb.d/10-readonly.sql
networks: [backend]
# …
configs:
db_init:
file: ./db/init/10-readonly.sqlAlso change the first line to
# compose.yaml – state at the end of Day 4. Then:
docker compose config -q; echo "Exit: $?"
docker compose up -d
docker compose exec db ls -l /docker-entrypoint-initdb.d
docker compose exec db psql -U notes -d notes -c '\du'
docker compose logs --no-log-prefix db | grep SkippingExpected (shortened):
Exit: 0
Container notes-db-1 Recreated
Container notes-db-1 Started
total 4
-rw-r--r-- 1 root root 145 Oct 29 20:50 10-readonly.sql
List of roles
Role name | Attributes
-----------+------------------------------------------------------------
notes | Superuser, Create role, Create DB, Replication, Bypass RLS
PostgreSQL Database directory appears to contain a database; Skipping initialization
The file is there, the role is not: the volume already contains a
database, so the image skips initialization including the init scripts.
A file: config is technically a bind mount, so the
permissions from the Mac apply (644).
Save your progress? Sign up for free
Drill 7: Init script on an empty volume
For the script to run, Postgres needs an empty volume. With a backup before and a restore afterwards, nothing gets lost (Day 3):
mkdir -p backup
docker compose exec -T db pg_dump -U notes notes > backup/notes.sql
docker compose down -v
docker compose up -d db
until docker compose exec db pg_isready -h 127.0.0.1 -U notes -d notes >/dev/null; do
sleep 1
done
docker compose logs --no-log-prefix db | grep -E 'initdb.d|CREATE ROLE|GRANT'
docker compose exec db psql -U notes -d notes -c '\du notes_ro'
docker compose exec -T db psql -q -U notes -d notes -v ON_ERROR_STOP=1 \
< backup/notes.sql >/dev/null
docker compose up -d
sleep 3
curl -s localhost:8080/notes; echoExpected (shortened):
Volume notes_pgdata Removed
Container notes-db-1 Started
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/10-readonly.sql
CREATE ROLE
GRANT ROLE
List of roles
Role name | Attributes
-----------+--------------
notes_ro | Cannot login
Container notes-worker-1 Started
[{"id":"8fdb2620-…","text":"Tag 4 startet mit Daten"}]
Role created, note back. Init scripts are meant for the initial setup: whatever comes later belongs in a migration (Day 5) or is applied by hand.
Save your progress? Sign up for free
Checkpoint
All variables are set; up --dry-run reads and
interpolates the file like a real up, but changes
nothing:
cd ~/compose-lab/tag04
docker compose up --dry-run >/dev/null; echo "Exit: $?"Expected:
Exit: 0
An empty key aborts with a message:
NOTES_API_KEY='' docker compose up --dry-run 2>&1 | tail -1Expected (shortened):
error while interpolating services.worker.environment.NOTES_API_KEY: required variable
NOTES_API_KEY is missing a value: NOTES_API_KEY fehlt …
So does an empty password:
POSTGRES_PASSWORD='' docker compose up --dry-run 2>&1 | tail -1Expected (shortened):
error while interpolating services.db.environment.POSTGRES_PASSWORD: required variable
POSTGRES_PASSWORD is missing a value: POSTGRES_PASSWORD fehlt …
worker runs with the default interval:
docker compose logs worker | grep 'worker started' | tail -1Expected (shortened):
worker-1 | worker started, api=http://api:8080, interval=10s
The .env does not end up in the container wholesale:
docker compose exec worker printenv POSTGRES_PASSWORD; echo "Exit: $?"Expected:
Exit: 1
Init script mounted, role created:
docker compose exec db ls /docker-entrypoint-initdb.d
docker compose exec db psql -U notes -d notes -tAc \
"SELECT rolname FROM pg_roles WHERE rolname = 'notes_ro'"Expected:
10-readonly.sql
notes_ro
api is healthy:
curl -s localhost:8080/healthz; echoExpected:
ok
Challenge
Give Postgres a second init script 20-comment.sql, this
time without a separate file, directly in compose.yaml. It
should comment the database with
Projekt <Projektname>, Tag 4 (“Project [project
name], Day 4”) – Compose fills in the project name. Afterwards the role,
the comment and your note must be there.
Hint: Besides file:, the top-level
configs: also supports content:, and Compose
provides ${COMPOSE_PROJECT_NAME} for interpolation; the
script only runs on an empty volume.
Solution
cp compose.yaml compose.yaml.bakIn the service db, add a second config and, at the end,
its source (whole file:
code/tag04/challenge/compose.yaml):
configs:
- source: db_init
target: /docker-entrypoint-initdb.d/10-readonly.sql
- source: db_comment
target: /docker-entrypoint-initdb.d/20-comment.sql
# …
configs:
db_init:
file: ./db/init/10-readonly.sql
db_comment:
content: |
COMMENT ON DATABASE notes IS 'Projekt ${COMPOSE_PROJECT_NAME}, Tag 4';docker compose exec -T db pg_dump -U notes notes > backup/notes.sql
docker compose down -v
docker compose up -d db
until docker compose exec db pg_isready -h 127.0.0.1 -U notes -d notes >/dev/null; do
sleep 1
done
docker compose exec -T db psql -q -U notes -d notes -v ON_ERROR_STOP=1 \
< backup/notes.sql >/dev/null
docker compose up -d
docker compose logs --no-log-prefix db | grep 'initdb.d/'
docker compose exec db ls -l /docker-entrypoint-initdb.d
docker compose exec db psql -U notes -d notes -tAc "SELECT shobj_description(oid, \
'pg_database') FROM pg_database WHERE datname = 'notes'"
docker compose exec db psql -U notes -d notes -tAc 'SELECT count(*) FROM notes'Expected (shortened):
Volume notes_pgdata Removed
Container notes-db-1 Started
Container notes-web-1 Started
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/10-readonly.sql
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/20-comment.sql
total 8
-rw-r--r-- 1 root root 145 Oct 29 20:50 10-readonly.sql
-r--r--r-- 1 root root 54 Oct 29 20:51 20-comment.sql
Projekt notes, Tag 4
1
The scripts run in alphabetical order. Compose creates the
content: config as a file with the default mode
0444; the file: config keeps the permissions
from the Mac. Back to the state of the day – comment and role stay in
the database, and tomorrow’s start of the day tears down the stack:
mv compose.yaml.bak compose.yaml
docker compose up -dExpected (shortened):
Container notes-db-1 Recreated
Container notes-db-1 Started
Save your progress? Sign up for free
Enterprise Note
- Only lab and development values belong in
.env. The repository contains.env.example; real values come in CI as environment variables from the secret store and thereby beat every file. - One file per environment via
--env-file(dev, stage) or separate Compose files (Day 7). Required values with:?turn missing configuration into an error before startup instead of a crash afterwards. - Output of
docker compose configanddocker inspectcontains values in plaintext. In pipelines, only logconfig -qorconfig --no-interpolate. - Init scripts run exactly once. Schema and permission changes after that belong in versioned migrations (Flyway, Liquibase, golang-migrate) or in IaC for the database, not in new init scripts.
- In Kubernetes,
configs:correspond to a ConfigMap mounted as a file,environment:toenv:in the pod; the “who wins” problem there is calledenvFromversusenv.