Skip to content
Hands-on IT IT Drills

Day 3: Volumes

Goal: Run Postgres as the service db with a named volume, keep the data across down, back it up and restore it into an empty database. · Duration: approx. 75 min · Prerequisite: Day 2

Briefing

  • A named volume belongs to Docker, lives in the Colima VM and survives every container. Compose names it <project>_<name>, here notes_pgdata.
  • docker compose down leaves volumes alone, down -v deletes them – database included.
  • Postgres 18 stores its data under /var/lib/postgresql/18/docker. The volume therefore belongs on /var/lib/postgresql, not on …/data.
  • Bind mount = folder from the Mac inside the container, tmpfs = RAM, gone after a restart.
  • A backup only counts as one once the restore has worked.
 Mac ~/compose-lab/tag03          Colima VM
 ┌──────────────────────┐        ┌───────────────────────────────────────────┐
 │ backup/notes.sql   ◄─┼pg_dump─┤ notes-db-1 (network notes_backend)        │
 │                    ──┼─psql──►│   /var/lib/postgresql ◄── volume          │
 └──────────────────────┘exec -T │                           notes_pgdata    │
                                 └───────────────────────────────────────────┘

Drill 1: Postgres as a service

Start of the day with the state of Day 2:

source ~/compose-lab/setup/env.sh
docker compose -p notes down
mkdir -p ~/compose-lab/tag03 && cd ~/compose-lab/tag03
cp -R "$LAB/code/notes/." . && cp -R "$LAB/code/tag02/notes/." .

New: the service db, the DB URL for api and the top-level volumes::

cat > compose.yaml <<'EOF'
# compose.yaml – state at the end of Day 3
name: notes

services:
  api:
    build: notes-api
    environment:
      NOTES_API_KEY: ${NOTES_API_KEY}
      NOTES_DB_URL: postgres://notes:${POSTGRES_PASSWORD}@db:5432/notes
    ports:
      - "127.0.0.1:${API_PORT:-8080}:8080"
    networks: [frontend, backend]
    depends_on:
      - db

  worker:
    build: notes-worker
    environment:
      NOTES_API_URL: http://api:8080
      NOTES_API_KEY: ${NOTES_API_KEY}
    depends_on:
      - api
    networks: [backend]

  web:
    build: notes-web
    environment:
      NOTES_API_URL: http://api:8080
      NOTES_API_KEY: ${NOTES_API_KEY}
    ports:
      - "127.0.0.1:${WEB_PORT:-3000}:3000"
    depends_on:
      - api
    networks: [frontend]

  db:
    image: postgres:18.6-alpine
    environment:
      POSTGRES_USER: notes
      POSTGRES_DB: notes
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    volumes:
      - pgdata:/var/lib/postgresql
    networks: [backend]

networks:
  frontend: {}
  backend:
    internal: true

volumes:
  pgdata: {}
EOF
docker compose config -q; echo "Exit: $?"

Expected:

Exit: 0

db is attached only to the internal backend and has no ports: – Postgres cannot be reached from the Mac, only by api (Day 2). Now start only the database and wait:

docker compose up -d db
until docker compose exec db pg_isready -h 127.0.0.1 -U notes -d notes; do
  sleep 1
done
docker volume ls --filter label=com.docker.compose.project=notes

Expected (shortened, times differ on your machine):

 Volume notes_pgdata Created
 Network notes_backend Created
 Container notes-db-1 Started
127.0.0.1:5432 - no response
127.0.0.1:5432 - accepting connections
DRIVER    VOLUME NAME
local     notes_pgdata

Why -h 127.0.0.1? On the first start, Postgres initializes with a temporary server that listens only on the Unix socket. Only the test over TCP tells you that api can connect.

Save your progress? Sign up for free

Drill 2: Create the table, start the stack

notes-api migrate creates the table. As a one-off container with run --rm:

docker compose run --rm api migrate
docker compose up -d
sleep 3
docker compose ps --format '{{.Service}}\t{{.Status}}'
curl -s -X POST localhost:8080/notes -H 'X-API-Key: lab-api-key-123' \
  -H 'Content-Type: application/json' -d '{"text":"Tag 3 liegt im Volume"}'; echo
docker compose logs api

Expected (shortened, IDs and times differ):

 Container notes-api-run-00c7ed8bac49 Created
time=… level=INFO msg="migration complete" table=notes
 Container notes-worker-1 Started
 Container notes-web-1 Started
api Up 3 seconds (healthy)
db  Up 16 seconds
web Up 3 seconds
worker  Up 3 seconds
{"id":"07acfbc9-…","text":"Tag 3 liegt im Volume"}
api-1  | time=… level=INFO msg="api key loaded" source=env
api-1  | time=… level=INFO msg="store ready" store=postgres
api-1  | time=… level=INFO msg=listening addr=:8080 version=dev pid=1

run builds the image if it is missing, starts depends_on services as well, publishes no ports and replaces the command: /notes-api becomes /notes-api migrate. migrate is idempotent; a second call does no harm.

Warning: Until Day 5, the order – first db, wait, migrate, then the rest – exists only in this chapter. If api starts before Postgres accepts connections, it exits with exit code 1 (database unreachable), exactly as in the challenge of Day 0.

Save your progress? Sign up for free

Drill 3: A look into the volume

docker compose volumes
docker volume inspect notes_pgdata
docker compose exec db printenv PGDATA
docker compose exec db ls /var/lib/postgresql
docker compose exec db du -sh /var/lib/postgresql

Expected (shortened):

DRIVER    VOLUME NAME
local     notes_pgdata
[
    {
        "CreatedAt": "2026-10-29T21:41:20+01:00",
        "Driver": "local",
        "Labels": {
            "com.docker.compose.config-hash": "311194ae…",
            "com.docker.compose.project": "notes",
            "com.docker.compose.version": "5.6.0",
            "com.docker.compose.volume": "pgdata"
        },
        "Mountpoint": "/var/lib/docker/volumes/notes_pgdata/_data",
        "Name": "notes_pgdata",
        "Options": null,
        "Scope": "local"
    }
]
/var/lib/postgresql/18/docker
18
46.4M   /var/lib/postgresql
  • The labels reveal the owner: project notes, volume pgdata. That’s how you recognize, when cleaning up, what belongs to the lab.
  • Mountpoint is in the Colima VM, not on the Mac. You can only get to the files through a container.
  • PGDATA points to the subfolder 18/docker. A single mount above it later allows pg_upgrade --link from one version folder to the next.

Save your progress? Sign up for free

Drill 4: down and down -v

docker compose down
docker volume ls --filter label=com.docker.compose.project=notes
docker compose up -d db
until docker compose exec db pg_isready -h 127.0.0.1 -U notes -d notes >/dev/null; do
  sleep 1
done
docker compose up -d
docker compose exec db psql -U notes -d notes -c 'SELECT text FROM notes'

Expected (shortened):

 Container notes-web-1 Removed
 Container notes-worker-1 Removed
 Container notes-api-1 Removed
 Container notes-db-1 Removed
 Network notes_frontend Removed
 Network notes_backend Removed
DRIVER    VOLUME NAME
local     notes_pgdata
 Container notes-db-1 Started
 Container notes-web-1 Started
         text
-----------------------
 Tag 3 liegt im Volume
(1 row)

Containers and networks are gone, the volume is not – the note (“Day 3 lives in the volume”) is still there. With an existing volume, Postgres starts in about a second; in the test, a direct docker compose up -d worked too. But you can’t rely on that.

Command Containers, networks named volumes anonymous volumes
docker compose down removed kept kept
docker compose down -v removed deleted deleted
docker compose rm -sfv db db only kept db’s deleted

You use down -v in Drill 7 – with a backup.

Save your progress? Sign up for free

Drill 5: The Postgres 18 trap

Many guides still mount on /var/lib/postgresql/data. Try it:

docker compose down
sed -i.bak 's#pgdata:/var/lib/postgresql$#pgdata:/var/lib/postgresql/data#' \
  compose.yaml
grep -n 'pgdata:/' compose.yaml
docker compose up -d db
sleep 3
docker compose ps -a --format '{{.Service}}\t{{.Status}}' db
docker compose logs --no-log-prefix db | head -9

Expected (shortened):

43:      - pgdata:/var/lib/postgresql/data
 Container notes-db-1 Started
db  Exited (1) 4 seconds ago
Error: in 18+, these Docker images are configured to store database data in a
       format which is compatible with "pg_ctlcluster" (specifically, using
       major-version-specific directory names).  This better reflects how
       PostgreSQL itself works, and how upgrades are to be performed.

       See also https://github.com/docker-library/postgres/pull/1259

       Counter to that, there appears to be PostgreSQL data in:
         /var/lib/postgresql/data (unused mount/volume)

The image refuses because its data would end up under /var/lib/postgresql/18/docker – outside your volume. And there is a second trace:

docker container inspect notes-db-1 \
  | jq -c '.[0].Mounts[] | {Type, Name, Destination}'

Expected (shortened, IDs differ):

{"Type":"volume","Name":"notes_pgdata","Destination":"/var/lib/postgresql/data"}
{"Type":"volume","Name":"d3fa5ff65b66…","Destination":"/var/lib/postgresql"}

The image declares VOLUME /var/lib/postgresql. If nothing is mounted there, Docker creates an anonymous volume – without a name, without Compose labels. Back to the correct path; rm -v takes the anonymous volume with it:

mv compose.yaml.bak compose.yaml
docker compose rm -sfv db
docker compose up -d db
until docker compose exec db pg_isready -h 127.0.0.1 -U notes -d notes >/dev/null; do
  sleep 1
done
docker compose up -d
sleep 3
curl -s localhost:8080/notes; echo

Expected (shortened):

Going to remove notes-db-1
 Container notes-db-1 Removed
 Container notes-db-1 Started
 Container notes-worker-1 Started
[{"id":"07acfbc9-…","text":"Tag 3 liegt im Volume"}]

Save your progress? Sign up for free

Drill 6: Backup with pg_dump

pg_dump runs in the container; the output lands on the Mac via redirection:

mkdir -p backup
docker compose exec -T db pg_dump -U notes notes > backup/notes.sql
ls -l backup
grep -E 'CREATE TABLE|COPY|Tag 3|PostgreSQL database dump' backup/notes.sql
printf '# Datenbank-Backups (Tag 3)\nbackup/\n' >> .gitignore

Expected (shortened, IDs and times differ):

-rw-r--r--@ 1 du  staff  1377 Oct 29 21:43 notes.sql
-- PostgreSQL database dump
CREATE TABLE public.notes (
COPY public.notes (id, text, created_at) FROM stdin;
07acfbc9-…  Tag 3 liegt im Volume   2026-10-29 20:41:37.000827+00
-- PostgreSQL database dump complete

-T turns off the pseudo-terminal that exec otherwise allocates. Without a terminal, the bytes arrive in the file unchanged, and in Drill 7 exec can read from stdin.

Security: A dump contains all data in plaintext. That’s why backup/ is in .gitignore from now on.

Save your progress? Sign up for free

Drill 7: Restore into an empty database

Now on purpose: delete the volume, start an empty database, load the backup.

docker compose down -v
docker compose up -d db
until docker compose exec db pg_isready -h 127.0.0.1 -U notes -d notes >/dev/null; do
  sleep 1
done
docker compose exec db psql -U notes -d notes -c '\dt'
docker compose exec -T db psql -U notes -d notes -v ON_ERROR_STOP=1 \
  < backup/notes.sql
docker compose up -d
sleep 3
curl -s localhost:8080/notes; echo

Expected (shortened):

 Volume notes_pgdata Removed
 Network notes_frontend Removed
 Volume notes_pgdata Created
 Container notes-db-1 Started
Did not find any tables.
SET
…
CREATE TABLE
ALTER TABLE
COPY 1
ALTER TABLE
 Container notes-worker-1 Started
[{"id":"07acfbc9-…","text":"Tag 3 liegt im Volume"}]

No migrate was needed: the dump contains the CREATE TABLE. ON_ERROR_STOP=1 makes psql abort with exit code 3 at the first error instead of silently carrying on.

Save your progress? Sign up for free

Drill 8: Bind mount and tmpfs

For comparison, you temporarily attach two more mounts to db. Backup first:

cp compose.yaml compose.yaml.bak

Add to the service db (editor); afterwards it reads:

    volumes:
      - pgdata:/var/lib/postgresql
      - ./backup:/backup
    tmpfs:
      - /scratch:size=16m
    networks: [backend]
docker compose up -d db
sleep 2
docker compose exec db df -h /var/lib/postgresql /backup /scratch
docker compose exec db pg_dump -U notes -f /backup/notes-bind.sql notes
ls -l backup

Expected (shortened, sizes differ):

 Container notes-db-1 Recreated
 Container notes-db-1 Started
Filesystem                Size      Used Available Use% Mounted on
/dev/root                18.3G     10.3G      8.0G  56% /etc/resolv.conf
lima-a33ada9020d2b328
                        228.3G    220.3G      8.0G  97% /backup
tmpfs                    16.0M         0     16.0M   0% /scratch
total 16
-rw-r--r--  1 du  staff  1377 Oct 29 21:44 notes-bind.sql
-rw-r--r--@ 1 du  staff  1377 Oct 29 21:43 notes.sql
  • Volume: disk of the Colima VM (/dev/root; BusyBox df names the first mount point of this disk).
  • Bind mount: Colima’s Mac share (lima-…). The dump, written as root inside the container, belongs to you on the Mac. A relative path is resolved from the folder of the compose.yaml – and only works under $HOME (Day 0).
  • tmpfs: RAM, limited to 16 MB here.
docker compose exec db sh -c 'echo hallo > /scratch/notiz && ls /scratch'
docker compose restart db
docker compose exec db ls -A /scratch | wc -l

Expected:

notiz
 Container notes-db-1 Started
       0

After the restart, tmpfs is empty – good for temp files and sockets, never for data. Back to the state of the day:

mv compose.yaml.bak compose.yaml
docker compose up -d db

Expected:

 Container notes-db-1 Recreated
 Container notes-db-1 Started

Save your progress? Sign up for free

Checkpoint

The volume is mounted at /var/lib/postgresql:

cd ~/compose-lab/tag03
docker compose config --format json | jq '.services.db.volumes'

Expected:

[
  {
    "type": "volume",
    "source": "pgdata",
    "target": "/var/lib/postgresql",
    "volume": {}
  }
]

db publishes no port, only 5432/tcp without -> to the host:

docker compose ps db --format '{{.Ports}}'

Expected:

5432/tcp

api stores data in Postgres:

docker compose logs api | grep -o 'store=[a-z]*'

Expected:

store=postgres

Notes in the database:

docker compose exec db psql -U notes -d notes -tAc 'SELECT count(*) FROM notes'

Expected (number may differ):

1

The backup and its entry in .gitignore:

grep -c 'CREATE TABLE public.notes' backup/notes.sql
grep -x 'backup/' .gitignore

Expected:

1
backup/

Challenge

Protect the database against an accidental docker compose down -v: the volume should be named notes-pgdata-keep (without the project prefix), contain your note and survive down -v.

Hint: The top-level volumes: supports external: true and name:; you create an external volume yourself – with the label lab=compose-lab so you find it when cleaning up.

Solution

cp compose.yaml compose.yaml.bak
docker compose down

Replace the volumes: block at the end of the file (whole file: code/tag03/challenge/compose.yaml):

volumes:
  pgdata:
    external: true          # Compose never creates or deletes it
    name: notes-pgdata-keep # name without project prefix
docker compose up -d db
docker volume create --label lab=compose-lab notes-pgdata-keep
docker compose up -d db
until docker compose exec db pg_isready -h 127.0.0.1 -U notes -d notes >/dev/null; do
  sleep 1
done
docker compose exec -T db psql -q -U notes -d notes < backup/notes.sql >/dev/null
docker compose down -v
docker volume ls --filter name=notes
docker compose up -d db
sleep 3
docker compose exec db psql -U notes -d notes -tAc 'SELECT text FROM notes'

Expected (shortened):

external volume "notes-pgdata-keep" not found
notes-pgdata-keep
 Container notes-db-1 Started
 Container notes-db-1 Removed
 Network notes_backend Removed
DRIVER    VOLUME NAME
local     notes-pgdata-keep
local     notes_pgdata
 Container notes-db-1 Started
Tag 3 liegt im Volume

The first start fails on purpose: Compose never creates external volumes itself. down -v removes only the container and network. notes_pgdata stays as well – it is currently no longer in the file, so Compose doesn’t touch it. Back to the state of the day:

docker compose down
mv compose.yaml.bak compose.yaml
docker volume rm notes-pgdata-keep

Expected (shortened):

 Container notes-db-1 Removed
 Network notes_backend Removed
notes-pgdata-keep

The volume notes_pgdata with your note stays for Day 4.

Save your progress? Sign up for free

Enterprise Note

  • pg_dump is a logical backup for small databases. For large databases and point-in-time recovery there is WAL archiving with pgBackRest or WAL-G.
  • Backups leave the host, are encrypted and regularly restored into an empty instance – automated, with a check like in Drill 7.
  • The local driver ties data to one host. Network storage via driver_opts (NFS) is risky for Postgres (locking, latency); in production usually a dedicated disk or a managed database.
  • Major upgrades (18 → 19) go through pg_upgrade or dump and restore. The layout /var/lib/postgresql/<version>/docker is designed exactly for that.
  • External volumes are a contract like external networks (Day 2): who creates, backs up and deletes them must be documented.