Day 3: Volumes
Goal: Run Postgres as the service
dbwith a named volume, keep the data acrossdown, back it up and restore it into an empty database. · Duration: approx. 75 min · Prerequisite: Day 2
Briefing
- A named volume belongs to Docker, lives in the
Colima VM and survives every container. Compose names it
<project>_<name>, herenotes_pgdata. docker compose downleaves volumes alone,down -vdeletes them – database included.- Postgres 18 stores its data under
/var/lib/postgresql/18/docker. The volume therefore belongs on/var/lib/postgresql, not on…/data. - Bind mount = folder from the Mac inside the container, tmpfs = RAM, gone after a restart.
- A backup only counts as one once the restore has worked.
Mac ~/compose-lab/tag03 Colima VM
┌──────────────────────┐ ┌───────────────────────────────────────────┐
│ backup/notes.sql ◄─┼pg_dump─┤ notes-db-1 (network notes_backend) │
│ ──┼─psql──►│ /var/lib/postgresql ◄── volume │
└──────────────────────┘exec -T │ notes_pgdata │
└───────────────────────────────────────────┘
Drill 1: Postgres as a service
Start of the day with the state of Day 2:
source ~/compose-lab/setup/env.sh
docker compose -p notes down
mkdir -p ~/compose-lab/tag03 && cd ~/compose-lab/tag03
cp -R "$LAB/code/notes/." . && cp -R "$LAB/code/tag02/notes/." .New: the service db, the DB URL for api and the
top-level volumes::
cat > compose.yaml <<'EOF'
# compose.yaml – state at the end of Day 3
name: notes
services:
api:
build: notes-api
environment:
NOTES_API_KEY: ${NOTES_API_KEY}
NOTES_DB_URL: postgres://notes:${POSTGRES_PASSWORD}@db:5432/notes
ports:
- "127.0.0.1:${API_PORT:-8080}:8080"
networks: [frontend, backend]
depends_on:
- db
worker:
build: notes-worker
environment:
NOTES_API_URL: http://api:8080
NOTES_API_KEY: ${NOTES_API_KEY}
depends_on:
- api
networks: [backend]
web:
build: notes-web
environment:
NOTES_API_URL: http://api:8080
NOTES_API_KEY: ${NOTES_API_KEY}
ports:
- "127.0.0.1:${WEB_PORT:-3000}:3000"
depends_on:
- api
networks: [frontend]
db:
image: postgres:18.6-alpine
environment:
POSTGRES_USER: notes
POSTGRES_DB: notes
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
volumes:
- pgdata:/var/lib/postgresql
networks: [backend]
networks:
frontend: {}
backend:
internal: true
volumes:
pgdata: {}
EOF
docker compose config -q; echo "Exit: $?"Expected:
Exit: 0
db is attached only to the internal backend
and has no ports: – Postgres cannot be reached from the
Mac, only by api (Day 2). Now start only the database and wait:
docker compose up -d db
until docker compose exec db pg_isready -h 127.0.0.1 -U notes -d notes; do
sleep 1
done
docker volume ls --filter label=com.docker.compose.project=notesExpected (shortened, times differ on your machine):
Volume notes_pgdata Created
Network notes_backend Created
Container notes-db-1 Started
127.0.0.1:5432 - no response
127.0.0.1:5432 - accepting connections
DRIVER VOLUME NAME
local notes_pgdata
Why -h 127.0.0.1? On the first start, Postgres
initializes with a temporary server that listens only on the Unix
socket. Only the test over TCP tells you that api can connect.
Save your progress? Sign up for free
Drill 2: Create the table, start the stack
notes-api migrate creates the table. As a one-off
container with run --rm:
docker compose run --rm api migrate
docker compose up -d
sleep 3
docker compose ps --format '{{.Service}}\t{{.Status}}'
curl -s -X POST localhost:8080/notes -H 'X-API-Key: lab-api-key-123' \
-H 'Content-Type: application/json' -d '{"text":"Tag 3 liegt im Volume"}'; echo
docker compose logs apiExpected (shortened, IDs and times differ):
Container notes-api-run-00c7ed8bac49 Created
time=… level=INFO msg="migration complete" table=notes
Container notes-worker-1 Started
Container notes-web-1 Started
api Up 3 seconds (healthy)
db Up 16 seconds
web Up 3 seconds
worker Up 3 seconds
{"id":"07acfbc9-…","text":"Tag 3 liegt im Volume"}
api-1 | time=… level=INFO msg="api key loaded" source=env
api-1 | time=… level=INFO msg="store ready" store=postgres
api-1 | time=… level=INFO msg=listening addr=:8080 version=dev pid=1
run builds the image if it is missing, starts
depends_on services as well, publishes no ports and
replaces the command: /notes-api becomes
/notes-api migrate. migrate is idempotent; a
second call does no harm.
Warning: Until Day 5, the order – first db, wait, migrate, then the rest – exists only in this chapter. If api starts before Postgres accepts connections, it exits with exit code 1 (
database unreachable), exactly as in the challenge of Day 0.
Save your progress? Sign up for free
Drill 3: A look into the volume
docker compose volumes
docker volume inspect notes_pgdata
docker compose exec db printenv PGDATA
docker compose exec db ls /var/lib/postgresql
docker compose exec db du -sh /var/lib/postgresqlExpected (shortened):
DRIVER VOLUME NAME
local notes_pgdata
[
{
"CreatedAt": "2026-10-29T21:41:20+01:00",
"Driver": "local",
"Labels": {
"com.docker.compose.config-hash": "311194ae…",
"com.docker.compose.project": "notes",
"com.docker.compose.version": "5.6.0",
"com.docker.compose.volume": "pgdata"
},
"Mountpoint": "/var/lib/docker/volumes/notes_pgdata/_data",
"Name": "notes_pgdata",
"Options": null,
"Scope": "local"
}
]
/var/lib/postgresql/18/docker
18
46.4M /var/lib/postgresql
- The labels reveal the owner: project
notes, volumepgdata. That’s how you recognize, when cleaning up, what belongs to the lab. Mountpointis in the Colima VM, not on the Mac. You can only get to the files through a container.PGDATApoints to the subfolder18/docker. A single mount above it later allowspg_upgrade --linkfrom one version folder to the next.
Save your progress? Sign up for free
Drill 4: down and down -v
docker compose down
docker volume ls --filter label=com.docker.compose.project=notes
docker compose up -d db
until docker compose exec db pg_isready -h 127.0.0.1 -U notes -d notes >/dev/null; do
sleep 1
done
docker compose up -d
docker compose exec db psql -U notes -d notes -c 'SELECT text FROM notes'Expected (shortened):
Container notes-web-1 Removed
Container notes-worker-1 Removed
Container notes-api-1 Removed
Container notes-db-1 Removed
Network notes_frontend Removed
Network notes_backend Removed
DRIVER VOLUME NAME
local notes_pgdata
Container notes-db-1 Started
Container notes-web-1 Started
text
-----------------------
Tag 3 liegt im Volume
(1 row)
Containers and networks are gone, the volume is not – the note (“Day
3 lives in the volume”) is still there. With an existing volume,
Postgres starts in about a second; in the test, a direct
docker compose up -d worked too. But you can’t rely on
that.
| Command | Containers, networks | named volumes | anonymous volumes |
|---|---|---|---|
docker compose down |
removed | kept | kept |
docker compose down -v |
removed | deleted | deleted |
docker compose rm -sfv db |
db only | kept | db’s deleted |
You use down -v in Drill 7 – with a backup.
Save your progress? Sign up for free
Drill 5: The Postgres 18 trap
Many guides still mount on /var/lib/postgresql/data. Try
it:
docker compose down
sed -i.bak 's#pgdata:/var/lib/postgresql$#pgdata:/var/lib/postgresql/data#' \
compose.yaml
grep -n 'pgdata:/' compose.yaml
docker compose up -d db
sleep 3
docker compose ps -a --format '{{.Service}}\t{{.Status}}' db
docker compose logs --no-log-prefix db | head -9Expected (shortened):
43: - pgdata:/var/lib/postgresql/data
Container notes-db-1 Started
db Exited (1) 4 seconds ago
Error: in 18+, these Docker images are configured to store database data in a
format which is compatible with "pg_ctlcluster" (specifically, using
major-version-specific directory names). This better reflects how
PostgreSQL itself works, and how upgrades are to be performed.
See also https://github.com/docker-library/postgres/pull/1259
Counter to that, there appears to be PostgreSQL data in:
/var/lib/postgresql/data (unused mount/volume)
The image refuses because its data would end up under
/var/lib/postgresql/18/docker – outside your volume. And
there is a second trace:
docker container inspect notes-db-1 \
| jq -c '.[0].Mounts[] | {Type, Name, Destination}'Expected (shortened, IDs differ):
{"Type":"volume","Name":"notes_pgdata","Destination":"/var/lib/postgresql/data"}
{"Type":"volume","Name":"d3fa5ff65b66…","Destination":"/var/lib/postgresql"}
The image declares VOLUME /var/lib/postgresql. If
nothing is mounted there, Docker creates an anonymous
volume – without a name, without Compose labels. Back to the
correct path; rm -v takes the anonymous volume with it:
mv compose.yaml.bak compose.yaml
docker compose rm -sfv db
docker compose up -d db
until docker compose exec db pg_isready -h 127.0.0.1 -U notes -d notes >/dev/null; do
sleep 1
done
docker compose up -d
sleep 3
curl -s localhost:8080/notes; echoExpected (shortened):
Going to remove notes-db-1
Container notes-db-1 Removed
Container notes-db-1 Started
Container notes-worker-1 Started
[{"id":"07acfbc9-…","text":"Tag 3 liegt im Volume"}]
Save your progress? Sign up for free
Drill 6: Backup with pg_dump
pg_dump runs in the container; the output lands on the
Mac via redirection:
mkdir -p backup
docker compose exec -T db pg_dump -U notes notes > backup/notes.sql
ls -l backup
grep -E 'CREATE TABLE|COPY|Tag 3|PostgreSQL database dump' backup/notes.sql
printf '# Datenbank-Backups (Tag 3)\nbackup/\n' >> .gitignoreExpected (shortened, IDs and times differ):
-rw-r--r--@ 1 du staff 1377 Oct 29 21:43 notes.sql
-- PostgreSQL database dump
CREATE TABLE public.notes (
COPY public.notes (id, text, created_at) FROM stdin;
07acfbc9-… Tag 3 liegt im Volume 2026-10-29 20:41:37.000827+00
-- PostgreSQL database dump complete
-T turns off the pseudo-terminal that exec
otherwise allocates. Without a terminal, the bytes arrive in the file
unchanged, and in Drill 7 exec can read from stdin.
Security: A dump contains all data in plaintext. That’s why
backup/is in.gitignorefrom now on.
Save your progress? Sign up for free
Drill 7: Restore into an empty database
Now on purpose: delete the volume, start an empty database, load the backup.
docker compose down -v
docker compose up -d db
until docker compose exec db pg_isready -h 127.0.0.1 -U notes -d notes >/dev/null; do
sleep 1
done
docker compose exec db psql -U notes -d notes -c '\dt'
docker compose exec -T db psql -U notes -d notes -v ON_ERROR_STOP=1 \
< backup/notes.sql
docker compose up -d
sleep 3
curl -s localhost:8080/notes; echoExpected (shortened):
Volume notes_pgdata Removed
Network notes_frontend Removed
Volume notes_pgdata Created
Container notes-db-1 Started
Did not find any tables.
SET
…
CREATE TABLE
ALTER TABLE
COPY 1
ALTER TABLE
Container notes-worker-1 Started
[{"id":"07acfbc9-…","text":"Tag 3 liegt im Volume"}]
No migrate was needed: the dump contains the
CREATE TABLE. ON_ERROR_STOP=1 makes psql abort
with exit code 3 at the first error instead of silently carrying on.
Save your progress? Sign up for free
Drill 8: Bind mount and tmpfs
For comparison, you temporarily attach two more mounts to
db. Backup first:
cp compose.yaml compose.yaml.bakAdd to the service db (editor); afterwards it reads:
volumes:
- pgdata:/var/lib/postgresql
- ./backup:/backup
tmpfs:
- /scratch:size=16m
networks: [backend]docker compose up -d db
sleep 2
docker compose exec db df -h /var/lib/postgresql /backup /scratch
docker compose exec db pg_dump -U notes -f /backup/notes-bind.sql notes
ls -l backupExpected (shortened, sizes differ):
Container notes-db-1 Recreated
Container notes-db-1 Started
Filesystem Size Used Available Use% Mounted on
/dev/root 18.3G 10.3G 8.0G 56% /etc/resolv.conf
lima-a33ada9020d2b328
228.3G 220.3G 8.0G 97% /backup
tmpfs 16.0M 0 16.0M 0% /scratch
total 16
-rw-r--r-- 1 du staff 1377 Oct 29 21:44 notes-bind.sql
-rw-r--r--@ 1 du staff 1377 Oct 29 21:43 notes.sql
- Volume: disk of the Colima VM
(
/dev/root; BusyBoxdfnames the first mount point of this disk). - Bind mount: Colima’s Mac share
(
lima-…). The dump, written as root inside the container, belongs to you on the Mac. A relative path is resolved from the folder of thecompose.yaml– and only works under$HOME(Day 0). - tmpfs: RAM, limited to 16 MB here.
docker compose exec db sh -c 'echo hallo > /scratch/notiz && ls /scratch'
docker compose restart db
docker compose exec db ls -A /scratch | wc -lExpected:
notiz
Container notes-db-1 Started
0
After the restart, tmpfs is empty – good for temp files and sockets, never for data. Back to the state of the day:
mv compose.yaml.bak compose.yaml
docker compose up -d dbExpected:
Container notes-db-1 Recreated
Container notes-db-1 Started
Save your progress? Sign up for free
Checkpoint
The volume is mounted at /var/lib/postgresql:
cd ~/compose-lab/tag03
docker compose config --format json | jq '.services.db.volumes'Expected:
[
{
"type": "volume",
"source": "pgdata",
"target": "/var/lib/postgresql",
"volume": {}
}
]
db publishes no port, only 5432/tcp without
-> to the host:
docker compose ps db --format '{{.Ports}}'Expected:
5432/tcp
api stores data in Postgres:
docker compose logs api | grep -o 'store=[a-z]*'Expected:
store=postgres
Notes in the database:
docker compose exec db psql -U notes -d notes -tAc 'SELECT count(*) FROM notes'Expected (number may differ):
1
The backup and its entry in .gitignore:
grep -c 'CREATE TABLE public.notes' backup/notes.sql
grep -x 'backup/' .gitignoreExpected:
1
backup/
Challenge
Protect the database against an accidental
docker compose down -v: the volume should be named
notes-pgdata-keep (without the project prefix), contain
your note and survive down -v.
Hint: The top-level volumes: supports
external: true and name:; you create an
external volume yourself – with the label lab=compose-lab
so you find it when cleaning up.
Solution
cp compose.yaml compose.yaml.bak
docker compose downReplace the volumes: block at the end of the file (whole
file: code/tag03/challenge/compose.yaml):
volumes:
pgdata:
external: true # Compose never creates or deletes it
name: notes-pgdata-keep # name without project prefixdocker compose up -d db
docker volume create --label lab=compose-lab notes-pgdata-keep
docker compose up -d db
until docker compose exec db pg_isready -h 127.0.0.1 -U notes -d notes >/dev/null; do
sleep 1
done
docker compose exec -T db psql -q -U notes -d notes < backup/notes.sql >/dev/null
docker compose down -v
docker volume ls --filter name=notes
docker compose up -d db
sleep 3
docker compose exec db psql -U notes -d notes -tAc 'SELECT text FROM notes'Expected (shortened):
external volume "notes-pgdata-keep" not found
notes-pgdata-keep
Container notes-db-1 Started
Container notes-db-1 Removed
Network notes_backend Removed
DRIVER VOLUME NAME
local notes-pgdata-keep
local notes_pgdata
Container notes-db-1 Started
Tag 3 liegt im Volume
The first start fails on purpose: Compose never creates external
volumes itself. down -v removes only the container and
network. notes_pgdata stays as well – it is currently no
longer in the file, so Compose doesn’t touch it. Back to the state of
the day:
docker compose down
mv compose.yaml.bak compose.yaml
docker volume rm notes-pgdata-keepExpected (shortened):
Container notes-db-1 Removed
Network notes_backend Removed
notes-pgdata-keep
The volume notes_pgdata with your note stays for Day
4.
Save your progress? Sign up for free
Enterprise Note
pg_dumpis a logical backup for small databases. For large databases and point-in-time recovery there is WAL archiving with pgBackRest or WAL-G.- Backups leave the host, are encrypted and regularly restored into an empty instance – automated, with a check like in Drill 7.
- The
localdriver ties data to one host. Network storage viadriver_opts(NFS) is risky for Postgres (locking, latency); in production usually a dedicated disk or a managed database. - Major upgrades (18 → 19) go through
pg_upgradeor dump and restore. The layout/var/lib/postgresql/<version>/dockeris designed exactly for that. - External volumes are a contract like external networks (Day 2): who creates, backs up and deletes them must be documented.