Day 2: Rootless and pods
Goal: Understand how rootless Podman maps UIDs, file permissions and ports, run notes as a pod with Postgres, and export the pod as Kubernetes YAML and start it again. · Duration: approx. 75 min · Prerequisite: Day 1
Briefing
- In rootless mode, every container runs in a user
namespace: UID 0 in the container is your user, all other UIDs
come from your range in
/etc/subuid. - Files a container creates therefore often belong to “foreign” UIDs
on the host, such as 100069.
podman unshareopens a shell in exactly this namespace. - Networking is done by pasta: it forwards ports from the host into the container. A normal user may not open ports below 1024.
- A pod is a group of containers with a shared
network: they all reach each other via
127.0.0.1. The concept comes from Kubernetes –podman kubespeaks its YAML.
Pod notes (infra container holds network namespace) Host
┌────────────────────────────────────────────────┐
│ notes-db :5432 ◄─ notes-api :8080 ◄─ notes-web :3000 ◄──── 127.0.0.1:3000
│ │ (127.0.0.1) │ ◄───────── 127.0.0.1:8080
│ └── Volume notes-pgdata │
└────────────────────────────────────────────────┘
Drill 1: subuid, subgid and newuidmap
Start of the day with the state of Day 1:
source ~/ct-lab/setup/env.sh
mkdir -p ~/ct-lab/tag02 && cd ~/ct-lab/tag02
cp -R "$LAB/code/notes/." . && cp -R "$LAB/code/tag01/notes/." .
id
grep "^$USER:" /etc/subuid /etc/subgid
ls -l /usr/bin/newuidmap /usr/bin/newgidmapExpected (abridged):
uid=1000(ubuntu) gid=1000(ubuntu) groups=1000(ubuntu),4(adm),24(cdrom),27(sudo),…
/etc/subuid:ubuntu:100000:65536
/etc/subgid:ubuntu:100000:65536
-rwsr-xr-x 1 root root 41944 Feb 2 2026 /usr/bin/newgidmap
-rwsr-xr-x 1 root root 41944 Feb 2 2026 /usr/bin/newuidmap
ubuntu:100000:65536means: your user may use the 65,536 UIDs starting at 100000.useraddenters such ranges automatically for every new user.newuidmapandnewgidmapare setuid root (sin the mode). They write the mapping into the kernel – the only piece of root that rootless Podman needs.
Save your progress? Sign up for free
Drill 2: podman unshare and the UID
mapping
cat /proc/self/uid_map
podman unshare cat /proc/self/uid_map
podman unshare id
podman run --rm docker.io/library/nginx:1.30.6-alpine cat /proc/self/uid_mapExpected:
0 0 4294967295
0 1000 1
1 100000 65536
uid=0(root) gid=0(root) groups=0(root)
0 1000 1
1 100000 65536
The three columns: UID inside, UID outside, count. Outside, your process sees all UIDs 1:1. In the namespace, UID 0 = 1000 (you), UID 1 = 100000, so UID 70 is 100000 + 70 − 1 = 100069. Check it with Postgres, which runs as UID 70:
podman run --rm docker.io/library/postgres:18.6-alpine id postgres
podman run -d --name pg-test -e POSTGRES_PASSWORD=lab-db-pw-123 \
docker.io/library/postgres:18.6-alpine
sleep 3
ps -o user,pid,args -u 100069 | head -3
podman rm -f -t 0 pg-testExpected (abridged, PIDs will differ):
Trying to pull docker.io/library/postgres:18.6-alpine...
…
uid=70(postgres) gid=70(postgres) groups=70(postgres),70(postgres)
1f0c…
USER PID COMMAND
100069 73958 bash /usr/local/bin/docker-entrypoint.sh postgres
100069 73988 initdb --username=postgres --pwfile=/dev/fd/63
pg-test
Postgres is initializing its database right now – as 100069. For
ps this is a UID without a name – it belongs to nobody on
the host. If a process breaks out of the container, it does not even
have your permissions.
Save your progress? Sign up for free
Drill 3: Volumes and permissions,
:U
A bind mount is a directory of yours. Postgres starts as root and
sets up the permissions itself; many images, however, start directly as
a normal user. You simulate that with --user 70:70:
mkdir pgdata-bind && ls -ldn pgdata-bind
podman run --name pg-bind --user 70:70 -e POSTGRES_PASSWORD=lab-db-pw-123 \
-v ./pgdata-bind:/var/lib/postgresql docker.io/library/postgres:18.6-alpine \
2>&1 | tail -1
podman rm pg-bindExpected:
drwxr-xr-x 2 1000 1000 4096 Oct 30 19:00 pgdata-bind
mkdir: can't create directory '/var/lib/postgresql/18/': Permission denied
pg-bind
The directory belongs to you (1000) – so to root inside the
container. UID 70 may not create anything there. The option
:U adjusts the owner to the container UID
before the start:
podman run -d --name pg-bind --user 70:70 -e POSTGRES_PASSWORD=lab-db-pw-123 \
-v ./pgdata-bind:/var/lib/postgresql:U docker.io/library/postgres:18.6-alpine
sleep 5
podman ps -a --filter name=pg-bind --format '{{.Names}} {{.Status}}'
ls -ldn pgdata-bind
podman unshare ls -ldn pgdata-bindExpected (abridged):
915f7541…
pg-bind Up 5 seconds
drwxr-xr-x 3 100069 100069 4096 Oct 30 19:01 pgdata-bind
drwxr-xr-x 3 70 70 4096 Oct 30 19:01 pgdata-bind
The same file, two views: 100069 outside, 70 in the namespace. The downside: you yourself can no longer get at the files.
podman rm -f pg-bind
rm -rf pgdata-bind; echo "Exit: $?"
podman unshare rm -rf pgdata-bind && ls -d pgdata-bindExpected (abridged):
pg-bind
rm: cannot remove 'pgdata-bind/18/docker': Permission denied
Exit: 1
ls: cannot access 'pgdata-bind': No such file or directory
In the namespace you are root and may do anything with what lies in
your range. Named volumes
(-v notes-pgdata:/…) do not have this problem: they live
under ~/.local/share/containers/storage/volumes, and Podman
sets up the permissions itself on the first mount.
Save your progress? Sign up for free
Drill 4: Networking with pasta, ports below 1024
podman run --rm docker.io/library/nginx:1.30.6-alpine ip -4 addr show eth0
podman run --rm -p 127.0.0.1:80:80 docker.io/library/nginx:1.30.6-alpine true
echo "Exit: $?"
sysctl net.ipv4.ip_unprivileged_port_startExpected (abridged, addresses will differ):
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65520 qdisc fq_codel state UNKNOWN …
inet 192.168.64.…/24 brd 192.168.64.255 scope global eth0
valid_lft forever preferred_lft forever
Error: pasta failed with exit code 1:
Listen failed for HOST TCP port 127.0.0.1/80: Permission denied
Couldn't listen on requested TCP ports
Exit: 126
net.ipv4.ip_unprivileged_port_start = 1024
- pasta gives the container a copy of your VM’s address and translates connections in user space. No bridge interface, no iptables rules, no root.
- Ports below 1024 are privileged. You could unlock them system-wide
with the sysctl
net.ipv4.ip_unprivileged_port_start; in the lab we stick with 8080 and 3000.
Warning: Ubuntu 26.04 protects pasta with an AppArmor profile that rejects signals from Podman. If you remove the last container of a custom network in rootless mode, Podman reports
rootless netns: kill network process: permission denied. The container is gone anyway, and you can ignore the message. Pods in the default network are not affected.
Save your progress? Sign up for free
Drill 5: The pod notes
podman pod create --name notes -p 127.0.0.1:8080:8080 -p 127.0.0.1:3000:3000
podman pod ps
podman ps -a --pod --format '{{.Names}}\t{{.PodName}}\t{{.Status}}'Expected (abridged, IDs and times will differ on your machine):
16f6dcaf5f98…
POD ID NAME STATUS CREATED INFRA ID # OF CON…
16f6dcaf5f98 notes Created Less than a second ago fedd3f40d7cc 1
16f6dcaf5f98-infra notes Created
The ports belong to the pod, not to a container. The infra container does nothing except hold the network namespace. Now the database with a named volume:
podman volume create notes-pgdata
podman run -d --pod notes --name notes-db \
-e POSTGRES_USER=notes -e POSTGRES_DB=notes -e POSTGRES_PASSWORD=lab-db-pw-123 \
-v notes-pgdata:/var/lib/postgresql docker.io/library/postgres:18.6-alpine
until podman exec notes-db pg_isready -h 127.0.0.1 -U notes -d notes; do sleep 1; doneExpected (abridged):
notes-pgdata
c2f1c1c9…
127.0.0.1:5432 - no response
127.0.0.1:5432 - accepting connections
api and web find the database and each other at
127.0.0.1 – like processes on the same machine:
DB=postgres://notes:[email protected]:5432/notes
podman run --rm --pod notes -e NOTES_DB_URL=$DB localhost/notes-api:0.1.0 migrate
podman run -d --pod notes --name notes-api \
-e NOTES_API_KEY=lab-api-key-123 -e NOTES_DB_URL=$DB localhost/notes-api:0.1.0
podman run -d --pod notes --name notes-web -e NOTES_API_URL=http://127.0.0.1:8080 \
-e NOTES_API_KEY=lab-api-key-123 localhost/notes-web:0.1.0
sleep 4
podman ps --pod --format '{{.Names}}\t{{.PodName}}\t{{.Status}}'
curl -s -X POST 127.0.0.1:3000/api/notes -H 'Content-Type: application/json' \
-d '{"text":"Tag 2 im Pod"}'; echo
curl -s 127.0.0.1:8080/notes; echo
podman exec notes-db netstat -ltnExpected (abridged):
time=… level=INFO msg="migration complete" table=notes
beea6244…
35374…
16f6dcaf5f98-infra notes Up 11 seconds
notes-db notes Up 11 seconds
notes-api notes Up 5 seconds (healthy)
notes-web notes Up 4 seconds
{"id":"1ac607e9-…","text":"Tag 2 im Pod"}
[{"id":"1ac607e9-…","text":"Tag 2 im Pod"}]
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:5432 0.0.0.0:* LISTEN
tcp 0 0 :::3000 :::* LISTEN
tcp 0 0 :::5432 :::* LISTEN
tcp 0 0 :::8080 :::* LISTEN
netstat in the db container shows all three services:
one network namespace for the whole pod. Postgres listens on 5432 but is
not published – from the host you only get to 8080 and 3000.
Save your progress? Sign up for free
Drill 6: podman kube generate
podman kube generate notes -f notes-pod.yaml
grep -v -E '^#|SandboxID|creationTimestamp' notes-pod.yamlExpected (abridged):
apiVersion: v1
kind: Pod
metadata:
annotations:
labels:
app: notes
name: notes
spec:
containers:
- args:
- postgres
env:
- name: POSTGRES_DB
value: notes
- name: POSTGRES_USER
value: notes
- name: POSTGRES_PASSWORD
value: lab-db-pw-123
image: docker.io/library/postgres:18.6-alpine
name: notes-db
ports:
- containerPort: 3000
hostIP: 127.0.0.1
hostPort: 3000
- containerPort: 8080
hostIP: 127.0.0.1
hostPort: 8080
volumeMounts:
- mountPath: /var/lib/postgresql
name: notes-pgdata-pvc
- env:
- name: NOTES_API_KEY
value: lab-api-key-123
- name: NOTES_DB_URL
value: postgres://notes:[email protected]:5432/notes
image: localhost/notes-api:0.1.0
name: notes-api
securityContext: {}
- args:
- node
- server.js
env:
- name: NOTES_API_URL
value: http://127.0.0.1:8080
- name: NOTES_API_KEY
value: lab-api-key-123
image: localhost/notes-web:0.1.0
name: notes-web
securityContext: {}
volumes:
- name: notes-pgdata-pvc
persistentVolumeClaim:
claimName: notes-pgdata- The pod ports are attached to the first container
(
hostPort); Kubernetes has no infra container to put them on. - The named volume becomes a PersistentVolumeClaim
with
claimName: notes-pgdata. - The passwords are in plain text – fine for lab values, in production
they belong in a
Secret(Enterprise Note).
Save your progress? Sign up for free
Drill 7: podman kube play and
kube down
Remove the pod, play the YAML again:
podman pod rm -f notes
podman volume ls
podman kube play notes-pod.yaml
sleep 5
podman ps --pod --format '{{.Names}}\t{{.PodName}}\t{{.Status}}'
curl -s 127.0.0.1:8080/notes; echoExpected (abridged):
16f6dcaf5f98…
DRIVER VOLUME NAME
local notes-pgdata
Pod:
9ac3b313…
Containers:
480e459f…
4cab69ee…
4e5da64e…
9ac3b3130d8f-infra notes Up 6 seconds
notes-notes-db notes Up 5 seconds
notes-notes-api notes Up 5 seconds (starting)
notes-notes-web notes Up 6 seconds
[{"id":"1ac607e9-…","text":"Tag 2 im Pod"}]
pod rmdeletes containers, but no named volumes.kube playfinds the claimnotes-pgdataand mounts the existing volume – the note is still there.- The containers are now named
<pod>-<container>, sonotes-notes-api. kube playattaches pods to the networkpodman-default-kube-network, not to the default network.
Tearing down works with the same file:
podman kube down notes-pod.yaml
podman volume ls -q
podman kube play notes-pod.yaml >/dev/nullExpected (abridged):
Pods stopped:
Error: stopping container 4eae3488…: removing container 4eae3488… network: 1 error
occurred:
* rootless netns: kill network process: permission denied
Pods removed:
9ac3b3130d8f…
Secrets removed:
Volumes removed:
notes-pgdata
The error message is the AppArmor trap from Drill 4 (custom network)
– the pod is removed anyway. kube down deletes volumes only
with --force. For the checkpoint the pod is running
again.
Save your progress? Sign up for free
Checkpoint
Your user has a subuid range, unshare makes you root in
the namespace:
cd ~/ct-lab/tag02
grep "^$USER:" /etc/subuid
podman unshare id -uExpected:
ubuntu:100000:65536
0
Volume, pod definition and running pod:
podman volume ls --format '{{.Name}}'
grep 'claimName' notes-pod.yaml
podman pod ps --filter name=notes --format '{{.Name}} {{.Status}}'Expected (the first line is an anonymous volume, its ID will differ):
07d0786783aa…
notes-pgdata
claimName: notes-pgdata
notes Running
The note comes from the volume, the bind directory is gone:
curl -s 127.0.0.1:8080/notes | jq -r '.[].text'
ls -d pgdata-bindExpected:
Tag 2 im Pod
ls: cannot access 'pgdata-bind': No such file or directory
Challenge
Add notes-worker to the pod – only via the YAML, not with
podman run. The worker should summarize every 5
seconds.
Hint: The worker reads NOTES_API_URL,
NOTES_API_KEY and INTERVAL;
kube play --replace replaces a running pod.
Solution
Create a copy and insert a fourth container before
volumes: (complete file:
code/tag02/challenge/notes-pod-worker.yaml):
- env:
- name: NOTES_API_URL
value: http://127.0.0.1:8080
- name: NOTES_API_KEY
value: lab-api-key-123
- name: INTERVAL
value: "5"
image: localhost/notes-worker:0.1.0
name: notes-workerpodman kube play --replace notes-pod-worker.yaml >/dev/null 2>&1
sleep 12
podman ps --pod --format '{{.Names}}\t{{.Status}}'
podman logs notes-notes-worker | tail -3Expected (abridged):
0cb01d682f13-infra Up 13 seconds
notes-notes-db Up 13 seconds
notes-notes-api Up 12 seconds (healthy)
notes-notes-web Up 12 seconds
notes-notes-worker Up 12 seconds
worker started, api=http://127.0.0.1:8080, interval=5s
summary: 1 notes, 4 words
summary: 1 notes, 4 words
"5" must be in quotes: Kubernetes environment variables
are strings. To finish the day, tear down the pod – the volume stays for
Day 3:
podman kube down notes-pod-worker.yaml 2>/dev/null | tail -1
podman pod ps -q | wc -lExpected:
Volumes removed:
0
Save your progress? Sign up for free
Enterprise Note
- Assign subuid ranges centrally (LDAP/SSSD or configuration management), so that the ranges of different users never overlap.
- Bind mounts with
:Uchange owners on the host – dangerous on shared directories. For service data, named volumes or a separate directory per service are the better choice. - For ports 80/443 in rootless mode, prefer a rootful reverse proxy or a socket-activated service over opening the sysctl for all users.
- Generated kube YAML is a starting point, not a finished manifest: secrets instead of plain text, resource limits, probes and a Deployment instead of a bare pod are added in “Kubernetes hands-on”.
podman kube playunderstands only part of the Kubernetes API (Pod, Deployment, PVC, ConfigMap, Secret). Some of what it does not know it ignores without a warning.