Day 3: Quadlet
Goal: Describe the pod notes as systemd services that come up on their own after every reboot, recover from crashes and pick up image updates automatically. · Duration: approx. 75 min · Prerequisite: Day 2
Briefing
- Quadlet translates small INI files
(
.container,.pod,.volume,.network) into systemd services. It is a systemd generator and part of Podman. - Your files live in
~/.config/containers/systemd/. After every change:systemctl --user daemon-reload. notes-api.containerbecomesnotes-api.service,notes.podbecomesnotes-pod.service. Starting, stopping and logging are done withsystemctlandjournalctl.- Without linger, your user systemd dies when you log out – and the containers die with it.
AutoUpdate=pluspodman auto-updatepulls new images and restarts services.
~/.config/containers/systemd/ systemd --user
notes.network ──quadlet──► notes-network.service
notes-pgdata.volume ──────────► notes-pgdata-volume.service
notes.pod ──────────► notes-pod.service ─┬─► notes-db.service (healthy)
notes-db.container ──────────► ├─► notes-migrate.service (1x)
notes-migrate/-api/-web.container ─► ├─► notes-api.service
└─► notes-web.service
Drill 1: Quadlet – profile and first service
| Profile | Quadlet |
|---|---|
| Purpose | Run containers, pods, volumes and networks as systemd units |
| Special feature | no service of its own: a generator at daemon-reload,
the result is plain systemd |
| Project | docs.podman.io, manual page
podman-systemd.unit(5) |
| Version in the book | same as Podman: 5.7.0 |
Quadlet comes with the podman package – there is nothing
extra to install. Check where it lives and how systemd calls it:
dpkg -S /usr/libexec/podman/quadlet
ls -l /usr/lib/systemd/user-generators/podman-user-generator
/usr/libexec/podman/quadlet -versionExpected:
podman: /usr/libexec/podman/quadlet
lrwxrwxrwx 1 root root 31 Feb 23 2026 /usr/lib/systemd/user-generators/podman-user-g…
5.7.0
How Quadlet works: On every
daemon-reload, systemd calls all generators. Quadlet reads
your files, writes finished .service units to
/run/user/1000/systemd/generator/ and is gone again. The
service itself is a podman run under systemd.
Start of the day and the first service – nginx on port 8080:
source ~/ct-lab/setup/env.sh
mkdir -p ~/ct-lab/tag03 && cd ~/ct-lab/tag03
cp -R "$LAB/code/notes/." . && cp -R "$LAB/code/tag02/notes/." .
mkdir -p ~/.config/containers/systemd
cat > ~/.config/containers/systemd/web.container <<'EOF'
# ~/.config/containers/systemd/web.container – nur Tag 3, Drill 1–2
[Unit]
Description=nginx als Quadlet (Tag 3)
[Container]
ContainerName=web
Image=docker.io/library/nginx:1.30.6-alpine
PublishPort=127.0.0.1:8080:80
[Install]
WantedBy=default.target
EOF
systemctl --user daemon-reload
systemctl --user start web.service
systemctl --user status web.service --no-pager | head -5
curl -sI 127.0.0.1:8080 | head -2Expected (abridged, IDs and times will differ on your machine):
● web.service - nginx als Quadlet (Tag 3)
Loaded: loaded (/home/ubuntu/.config/containers/systemd/web.container; generated)
Active: active (running) since Fri 2026-10-30 19:04:17 UTC; 106ms ago
Invocation: ac94eca29c47418482807f2ecf11d601
Main PID: 32732 (conmon)
HTTP/1.1 200 OK
Server: nginx/1.30.6
Loaded: … generated – the unit comes from the generator,
the source is your .container file. Main PID
is conmon, the watchdog from Day 1.
Save your progress? Sign up for free
Drill 2: What Quadlet generates
The dry run shows the generated units without changing anything – the most important tool when something goes wrong:
/usr/libexec/podman/quadlet -dryrun -userExpected (abridged):
quadlet-generator[32657]: Loading source unit file /home/ubuntu/.config/containers/sy…
---web.service---
# ~/.config/containers/systemd/web.container – nur Tag 3, Drill 1–2
[Unit]
Wants=podman-user-wait-network-online.service
After=podman-user-wait-network-online.service
Description=nginx als Quadlet (Tag 3)
SourcePath=/home/ubuntu/.config/containers/systemd/web.container
RequiresMountsFor=%t/containers
[X-Container]
ContainerName=web
…
[Service]
Environment=PODMAN_SYSTEMD_UNIT=%n
KillMode=mixed
ExecStop=/usr/bin/podman rm -v -f -i web
ExecStopPost=-/usr/bin/podman rm -v -f -i web
Delegate=yes
Type=notify
NotifyAccess=all
SyslogIdentifier=%N
ExecStart=/usr/bin/podman run --name web --replace --rm --cgroups=split --sdnotify=c…
[Container]becomes a longpodman run:--replacereplaces a container with the same name,--rmdeletes it on stop. State therefore belongs in volumes.Type=notifywith--sdnotify=conmon: systemd reports “started” as soon as the container is running.
And [Install]? You cannot enable generated
units – Quadlet creates the link itself:
systemctl --user enable web.service; echo "Exit: $?"
ls /run/user/1000/systemd/generator/default.target.wants/Expected:
Failed to enable unit: Unit /run/user/1000/systemd/generator/web.service is transient o…
Exit: 1
web.service
The dry run also reports typos. QUADLET_UNIT_DIRS makes
it read only one directory:
mkdir -p /tmp/qtest
printf '[Container]\nImage=%s\nPublishport=8081:80\n' \
docker.io/library/nginx:1.30.6-alpine > /tmp/qtest/kaputt.container
QUADLET_UNIT_DIRS=/tmp/qtest /usr/libexec/podman/quadlet -dryrun -user; echo "Exit: $?"
rm -rf /tmp/qtestExpected:
quadlet-generator[40035]: Loading source unit file /tmp/qtest/kaputt.container
quadlet-generator[40035]: converting "kaputt.container": unsupported key 'Publishport' …
quadlet-generator[40035]: processing encountered some errors
Exit: 1
With a real daemon-reload, the unit is then simply
missing: Unit kaputt.service not found. The nginx service
has served its purpose:
systemctl --user stop web.service
rm ~/.config/containers/systemd/web.container
systemctl --user daemon-reloadSave your progress? Sign up for free
Drill 3: Network, volume and pod
Three small units. NetworkName=,
VolumeName= and PodName= fix the names;
without them, the objects would be called systemd-notes and
so on.
cd ~/.config/containers/systemd
cat > notes.network <<'EOF'
# ~/.config/containers/systemd/notes.network – Stand Ende Tag 3
[Unit]
Description=Netz für den Pod notes
[Network]
NetworkName=notes
EOF
cat > notes-pgdata.volume <<'EOF'
# ~/.config/containers/systemd/notes-pgdata.volume – Stand Ende Tag 3
[Unit]
Description=Postgres-Daten für notes
[Volume]
VolumeName=notes-pgdata
EOF
cat > notes.pod <<'EOF'
# ~/.config/containers/systemd/notes.pod – Stand Ende Tag 3
[Unit]
Description=Pod notes: db, migrate, api, web
[Pod]
PodName=notes
Network=notes.network
PublishPort=127.0.0.1:8080:8080
PublishPort=127.0.0.1:3000:3000
[Install]
WantedBy=default.target
EOFVolumeName=notes-pgdata is deliberate: the volume from
Day 2 already exists, and Quadlet creates it with
podman volume create --ignore – existing data stays. And
Network=notes.network refers to the unit, not to the
network name. That is how Quadlet recognizes the dependency.
Save your progress? Sign up for free
Drill 4: The pod’s containers
Postgres only reports itself as started once the healthcheck is green
(Notify=healthy):
cat > notes-db.container <<'EOF'
# ~/.config/containers/systemd/notes-db.container – Stand Ende Tag 3
[Unit]
Description=Postgres für notes
[Container]
ContainerName=notes-db
Image=docker.io/library/postgres:18.6-alpine
Pod=notes.pod
Volume=notes-pgdata.volume:/var/lib/postgresql
Environment=POSTGRES_USER=notes POSTGRES_DB=notes
Environment=POSTGRES_PASSWORD=lab-db-pw-123
HealthCmd=pg_isready -h 127.0.0.1 -U notes -d notes
HealthInterval=5s
Notify=healthy
AutoUpdate=registry
[Service]
Restart=on-failure
TimeoutStartSec=300
EOFThe migration is a one-time job: Type=oneshot waits
until the container ends, RemainAfterExit=yes leaves the
unit “active” afterwards.
cat > notes-migrate.container <<'EOF'
# ~/.config/containers/systemd/notes-migrate.container – Stand Ende Tag 3
[Unit]
Description=Schema für notes anlegen (einmalig pro Start)
Requires=notes-db.container
After=notes-db.container
[Container]
ContainerName=notes-migrate
Image=localhost/notes-api:0.1.0
Pod=notes.pod
Exec=migrate
Environment=NOTES_DB_URL=postgres://notes:[email protected]:5432/notes
[Service]
Type=oneshot
RemainAfterExit=yes
EOF
cat > notes-api.container <<'EOF'
# ~/.config/containers/systemd/notes-api.container – Stand Ende Tag 3
[Unit]
Description=notes-api
Requires=notes-migrate.container
After=notes-migrate.container
[Container]
ContainerName=notes-api
Image=localhost/notes-api:0.1.0
Pod=notes.pod
Environment=NOTES_API_KEY=lab-api-key-123
Environment=NOTES_DB_URL=postgres://notes:[email protected]:5432/notes
Notify=healthy
AutoUpdate=local
[Service]
Restart=on-failure
EOF
cat > notes-web.container <<'EOF'
# ~/.config/containers/systemd/notes-web.container – Stand Ende Tag 3
[Unit]
Description=notes-web
Wants=notes-api.container
After=notes-api.container
[Container]
ContainerName=notes-web
Image=localhost/notes-web:0.1.0
Pod=notes.pod
Environment=NOTES_API_URL=http://127.0.0.1:8080
Environment=NOTES_API_KEY=lab-api-key-123
AutoUpdate=local
[Service]
Restart=on-failure
EOF- In
Requires=/After=you may name the Quadlet file (notes-db.container); Quadlet translates it intonotes-db.service. Pod=notes.podbinds every container tonotes-pod.serviceviaBindsTo=: if the pod stops, all of them stop.- notes-api has its healthcheck in the image (Day 1), so
Notify=healthyis enough.
Check, load, start:
/usr/libexec/podman/quadlet -dryrun -user 2>&1 | grep -E '^---|rror'
systemctl --user daemon-reload
systemctl --user list-unit-files 'notes*' --no-pager
systemctl --user start notes-pod.service
until curl -sf 127.0.0.1:3000 >/dev/null; do sleep 1; done
systemctl --user list-units 'notes*' --all --no-pager --legend=no
podman ps -a --pod --format '{{.Names}}\t{{.Status}}'
curl -s -X POST 127.0.0.1:3000/api/notes -H 'Content-Type: application/json' \
-d '{"text":"Tag 3 mit Quadlet"}'; echo
curl -s 127.0.0.1:8080/notes; echoExpected (abridged):
---notes-pgdata-volume.service---
---notes-network.service---
---notes-api.service---
---notes-db.service---
---notes-migrate.service---
---notes-web.service---
---notes-pod.service---
UNIT FILE STATE PRESET
notes-api.service generated -
notes-db.service generated -
notes-migrate.service generated -
notes-network.service generated -
notes-pgdata-volume.service generated -
notes-pod.service generated -
notes-web.service generated -
7 unit files listed.
notes-api.service loaded active running notes-api
notes-db.service loaded active running Postgres für notes
notes-migrate.service loaded active exited Schema für notes anlegen (einmalig…
notes-network.service loaded active exited Netz für den Pod notes
notes-pgdata-volume.service loaded active exited Postgres-Daten für notes
notes-pod.service loaded active running Pod notes: db, migrate, api, web
notes-web.service loaded active running notes-web
notes-infra Up 18 seconds
notes-db Up 17 seconds (healthy)
notes-api Up 10 seconds (healthy)
notes-web Up 10 seconds
{"id":"ec7fd844-…","text":"Tag 3 mit Quadlet"}
[{"id":"1ac607e9-…","text":"Tag 2 im Pod"},{"id":"ec7fd844-…","text":"Tag 3 mit Quadlet…
systemctl start notes-pod.service returns after a
second; the containers follow via Wants=, in the order
given by After=. The note from Day 2 is there: the same
volume. notes-migrate is no longer a running container, just a unit in
the state exited.
Save your progress? Sign up for free
Drill 5: journalctl and restart after a crash
Every container logs to the journal, with the unit name as identifier
(-t). Under -u you also find the messages from
systemd and podman:
journalctl --user -t notes-migrate -o cat --no-pager -n 1
journalctl --user -u notes-db.service --no-pager -n 2 -o cat
podman kill notes-api
sleep 8
journalctl --user -u notes-api.service --no-pager -o cat \
| grep -E 'Main process|Failed|Scheduled|Started' | tail -4
podman ps --format '{{.Names}}\t{{.Status}}'Expected (abridged):
time=2026-10-30T19:04:50.700Z level=INFO msg="migration complete" table=notes
3cd7df25729bb0003c5862423ccf9b25bce91ee643feaf4837178d97ae57197c
Started notes-db.service - Postgres für notes.
notes-api
notes-api.service: Main process exited, code=exited, status=137/n/a
notes-api.service: Failed with result 'exit-code'.
notes-api.service: Scheduled restart job, restart counter is at 1.
Started notes-api.service - notes-api.
notes-infra Up 37 seconds
notes-db Up 36 seconds (healthy)
notes-web Up 29 seconds
notes-api Up 7 seconds (healthy)
Exit 137 = SIGKILL. Restart=on-failure restarts the
service, and podman logs still works too – both read the
same journal. A clean systemctl --user stop does not count
as a failure and does not trigger a restart.
Save your progress? Sign up for free
Drill 6: Linger – services without a login
Your systemd only runs as long as you are logged in – unless linger is on:
loginctl show-user "$USER" -p Linger
sudo loginctl enable-linger "$USER"
loginctl show-user "$USER" -p Linger -p StateExpected:
Linger=no
Linger=yes
State=active
With linger, systemd starts your user manager at boot, and it starts
everything that hangs off default.target – here
notes-pod.service from [Install] in
notes.pod. The real test is a reboot of the VM:
sudo rebootAfter half a minute, run multipass shell ct-lab on the
Mac, then:
systemctl --user list-units 'notes*' --no-pager --legend=no | awk '{print $1, $3, $4}'
curl -s 127.0.0.1:8080/notes | jq lengthExpected (abridged):
notes-api.service active running
notes-db.service active running
notes-migrate.service active exited
notes-network.service active exited
notes-pgdata-volume.service active exited
notes-pod.service active running
notes-web.service active running
2
Warning: Without linger, systemd terminates your user manager including its containers when you log out – often only seconds later, which makes troubleshooting confusing.
Save your progress? Sign up for free
Drill 7: AutoUpdate and
podman auto-update
AutoUpdate=registry (notes-db) compares the digest in
the registry with the local image, AutoUpdate=local (api,
web) compares with the local image of the same name. First a dry
run:
podman auto-update --dry-runExpected:
UNIT CONTAINER IMAGE …
notes-pod.service e35b3fb550f5 (notes-db) docker.io/library/postgres:1…
notes-pod.service d6f8c01025dc (notes-api) localhost/notes-api:0.1.0 …
notes-pod.service 3a69eb4a82bc (notes-web) localhost/notes-web:0.1.0 …
The last columns (cut off) are called POLICY and
UPDATED: registry false,
local false, local false. Now a new local
image under the same tag:
cd ~/ct-lab/tag03
podman build -q --format docker -t localhost/notes-api:0.1.0 \
--build-arg VERSION=0.1.0 --build-arg REVISION=tag03 notes-api
podman auto-update --dry-run --format '{{.Container}} {{.Policy}} {{.Updated}}'
podman auto-update --format '{{.Container}} {{.Policy}} {{.Updated}}'
sleep 15
podman inspect notes-api \
--format '{{index .Config.Labels "org.opencontainers.image.revision"}}'Expected (abridged):
f0ad4bdd6bbb…
e35b3fb550f5 (notes-db) registry false
d6f8c01025dc (notes-api) local pending
3a69eb4a82bc (notes-web) local false
e35b3fb550f5 (notes-db) registry true
d6f8c01025dc (notes-api) local true
3a69eb4a82bc (notes-web) local true
tag03
pending means: a restart would bring a different image.
Afterwards all three show true: containers
in a pod belong to the unit notes-pod.service, and
auto-update restarts the whole unit. A timer does this automatically,
daily:
systemctl --user enable --now podman-auto-update.timer
systemctl --user list-timers podman-auto-update.timer --no-pager | head -2Expected (abridged):
Created symlink '/home/ubuntu/.config/systemd/user/timers.target.wants/podman-auto-up…
NEXT LEFT LAST PASSED UNIT ACTIVATES
Sat 2026-10-31 00:12:21 UTC 5h 5min - - podman-auto-update.timer podman-auto-…
Warning:
AutoUpdate=registryrequires a fully qualified name. WithImage=nginx:…the dry run already warns, andpodman runwith the label fails:auto updates require fully-qualified image reference.
Save your progress? Sign up for free
Checkpoint
The seven Quadlet files and an error-free dry run:
cd ~/ct-lab/tag03
ls ~/.config/containers/systemd/
/usr/libexec/podman/quadlet -dryrun -user >/dev/null 2>&1; echo "Exit: $?"Expected:
notes-api.container
notes-db.container
notes-migrate.container
notes-pgdata.volume
notes-web.container
notes.network
notes.pod
Exit: 0
The units are running:
systemctl --user is-active notes-pod notes-db notes-api notes-webExpected:
active
active
active
active
Linger and auto-update timer:
loginctl show-user "$USER" -p Linger
systemctl --user is-enabled podman-auto-update.timerExpected:
Linger=yes
enabled
Notes via the Quadlet pod:
curl -s 127.0.0.1:8080/notes | jq lengthExpected:
3
Challenge
Limit notes-web to 128 MB of memory via Quadlet and prove with
podman that the limit takes effect.
Hint: podman-systemd.unit(5) knows the
key Memory= in the [Container] section.
Solution
Add one line to the [Container] section of
notes-web.container (complete file:
code/tag03/challenge/notes-web.container):
AutoUpdate=local
Memory=128mq=~/.config/containers/systemd
cp "$q/notes-web.container" /tmp/notes-web.container.bak
sed -i 's/^AutoUpdate=local$/AutoUpdate=local\nMemory=128m/' "$q/notes-web.container"
/usr/libexec/podman/quadlet -dryrun -user 2>/dev/null | grep -o -- '--memory [^ ]*'
systemctl --user daemon-reload
systemctl --user restart notes-web.service
sleep 3
podman inspect notes-web --format '{{.HostConfig.Memory}}'
podman stats --no-stream --format '{{.Name}} {{.MemUsage}}' notes-webExpected (abridged):
--memory 128m
134217728
notes-web 26.26MB / 134.2MB
In rootless mode, the limit only takes effect because systemd
delegates the cgroup controller memory to your user
(cpu memory pids in cgroup.subtree_control of
your [email protected]). Back to the state of the day:
cp /tmp/notes-web.container.bak ~/.config/containers/systemd/notes-web.container
systemctl --user daemon-reload && systemctl --user restart notes-web.serviceSave your progress? Sign up for free
Enterprise Note
- Quadlet files belong in Git and are distributed via Ansible or
similar. For system services they live in
/etc/containers/systemd/and run rootful – or rootless under a dedicated service user with linger. - No passwords in
Environment=: useSecret=withpodman secret createorEnvironmentFile=with mode 0600, or better, a secret store. - Use
AutoUpdate=registryonly with tags you control yourselves, and with rollback:podman auto-updaterestores the old image if the new service does not start – provided thatNotify=healthyreally reports “healthy”. - If you have an existing Kubernetes YAML, you can run it unchanged
with a
.kubeunit;.containerfiles are more readable and allow finer control. - Resource limits (
Memory=,PodmanArgs=--cpus …) and logs shipped to the central system (journald → Fluent Bit, Vector) belong to every production service.