Day 6: notes-api as an Appliance
Goal: Build notes-api with Postgres from two templates: a database VM with a persistent disk in the internal network and an API VM that connects to it via the context. · Duration: approx. 90 min · Prerequisite: Day 5
Briefing
- An appliance is a VM that offers a service after start without any manual work. Here it is built from the lean Alpine plus a startup script that runs on every start.
- The split:
| VM | Networks | Disks | Service |
|---|---|---|---|
db |
vnet (only for packages), intern with
fixed IP 10.10.10.20 |
Alpine + notes-daten (persistent) |
Postgres 16 |
notes |
vnet (port 8080), intern |
Alpine | notes-api as a container |
- Postgres listens only on the internal network. In
vnet,dbgets the security groupwebfrom Day 5, and it does not let port 5432 in. - Secrets such as the database password come as
USER_INPUTS of type
password. Sunstone shows them masked; inside the VM they end up in the context.
Browser/curl ──► notes (192.168.150.x:8080) ── intern ──► db (10.10.10.20:5432)
Podman: notes-api Postgres 16
/var/lib/postgresql ⇐ notes-daten
Drill 1: The data disk
oneimage create --name notes-daten --type DATABLOCK --size 1024 --persistent \
--datastore defaultExpected:
ID: 4
Save your progress? Sign up for free
Drill 2: The database script
mkdir -p ~/one-lab/tag06
cd ~/one-lab/tag06
cat > db.sh <<'EOF'
#!/bin/sh
# Postgres-Appliance: Datenplatte einhängen, Datenbank anlegen, im internen Netz anbieten
. /run/one-context/one_env
apk add -q postgresql16 e2fsprogs
blkid /dev/sda >/dev/null || mkfs.ext4 -q /dev/sda
mkdir -p /var/lib/postgresql
grep -q /var/lib/postgresql /etc/fstab || echo "/dev/sda /var/lib/postgresql ext4 defaults 0 2" >> /etc/fstab
mount -a
chown postgres:postgres /var/lib/postgresql
# Konfiguration bei den Daten lassen, nicht auf der flüchtigen Systemplatte
echo conf_dir=/var/lib/postgresql/16/data >> /etc/conf.d/postgresql
if [ ! -f /var/lib/postgresql/16/data/PG_VERSION ]; then
rc-service postgresql setup
echo "listen_addresses = '*'" >> /var/lib/postgresql/16/data/postgresql.conf
echo "host notes notes 10.10.10.0/24 scram-sha-256" >> /var/lib/postgresql/16/data/pg_hba.conf
rc-service postgresql start
su postgres -c "psql -c \"CREATE USER notes PASSWORD '$DB_PASSWORD'\""
su postgres -c "createdb -O notes notes"
fi
rc-update add postgresql
rc-service postgresql start
EOF
sh -n db.sh && echo okExpected:
ok
The script runs on every start, so it has to handle both cases:
- First start: The disk is empty.
blkidfinds no file system, somkfs.ext4. Thensetupcreates the database. - Every further start, even in a new VM: File system and database are there. The script only mounts and starts.
Tried it: Without the
conf_dirline, Alpine’spostgresql setupmoves the configuration to/etc/postgresql16, that is, onto the non-persistent system disk, and leaves only links behind on the data disk. The first VM runs. Every new VM with the same data disk fails withcould not access the server configuration file. Everything an appliance is supposed to survive belongs on the persistent disk.
Save your progress? Sign up for free
Drill 3: The API script
cat > notes.sh <<'EOF'
#!/bin/sh
# notes-api-Appliance: auf Postgres warten, Schema anlegen, Container starten
. /run/one-context/one_env
apk add -q podman
rc-update add cgroups
rc-service cgroups start
until nc -z -w2 "$DB_HOST" 5432; do sleep 5; done
URL="postgres://notes:$DB_PASSWORD@$DB_HOST:5432/notes?sslmode=disable"
IMG="ghcr.io/thomaszachmann/notes-api:$NOTES_VERSION"
podman run --rm --network host -e NOTES_DB_URL="$URL" "$IMG" migrate
podman rm -f notes
podman run -d --name notes --network host -e NOTES_API_KEY="$API_KEY" -e NOTES_DB_URL="$URL" "$IMG"
EOF
sh -n notes.sh && echo okExpected:
ok
rc-service cgroups: Without this service, Podman on Alpine fails withcrun: invalid file system type on /sys/fs/cgroup.- The loop with
nc -zwaits until Postgres answers. The two VMs start independently of each other, and OpenNebula does not guarantee any order. Only OneFlow can do that (Day 9). migratecreates the schema. That is harmless on every start.
Save your progress? Sign up for free
Drill 4: The templates
cat > db.tpl <<'EOF'
NAME = "notes-db"
CPU = "0.5"
VCPU = "1"
MEMORY = "512"
DISK = [ IMAGE = "Alpine Linux 3.20", IMAGE_UNAME = "oneadmin" ]
DISK = [ IMAGE = "notes-daten", IMAGE_UNAME = "oneadmin" ]
NIC = [ NETWORK = "vnet", NETWORK_UNAME = "oneadmin" ]
NIC = [ NETWORK = "intern", NETWORK_UNAME = "oneadmin", IP = "10.10.10.20" ]
GRAPHICS = [ TYPE = "vnc", LISTEN = "0.0.0.0" ]
USER_INPUTS = [ DB_PASSWORD = "M|password|Passwort für den Datenbank-Benutzer notes" ]
CONTEXT = [
NETWORK = "YES",
SSH_PUBLIC_KEY = "$USER[SSH_PUBLIC_KEY]",
SET_HOSTNAME = "$NAME",
DB_PASSWORD = "$DB_PASSWORD",
START_SCRIPT_BASE64 = "SKRIPT" ]
EOF
sed -i "s|SKRIPT|$(base64 -w0 db.sh)|" db.tpl
cat > notes.tpl <<'EOF'
NAME = "notes"
CPU = "0.5"
VCPU = "1"
MEMORY = "384"
DISK = [ IMAGE = "Alpine Linux 3.20", IMAGE_UNAME = "oneadmin" ]
NIC = [ NETWORK = "vnet", NETWORK_UNAME = "oneadmin" ]
NIC = [ NETWORK = "intern", NETWORK_UNAME = "oneadmin" ]
GRAPHICS = [ TYPE = "vnc", LISTEN = "0.0.0.0" ]
USER_INPUTS = [
DB_PASSWORD = "M|password|Passwort für den Datenbank-Benutzer notes",
API_KEY = "M|password|API-Key für notes-api",
NOTES_VERSION = "O|text|Version von notes-api||0.7.1" ]
CONTEXT = [
NETWORK = "YES",
SSH_PUBLIC_KEY = "$USER[SSH_PUBLIC_KEY]",
SET_HOSTNAME = "$NAME",
DB_HOST = "10.10.10.20",
DB_PASSWORD = "$DB_PASSWORD",
API_KEY = "$API_KEY",
NOTES_VERSION = "$NOTES_VERSION",
START_SCRIPT_BASE64 = "SKRIPT" ]
EOF
sed -i "s|SKRIPT|$(base64 -w0 notes.sh)|" notes.tpl
onetemplate create db.tpl
onetemplate create notes.tplExpected:
ID: 4
ID: 5
M before password means mandatory:
no value, no start. The order of the NICs in the template determines
which card becomes eth0.
Save your progress? Sign up for free
Drill 5: Start
onetemplate instantiate notes-db --name db --user-inputs DB_PASSWORD=lab-db-pass-123
onetemplate instantiate notes --name notes \
--user-inputs DB_PASSWORD=lab-db-pass-123,API_KEY=lab-api-key-123,NOTES_VERSION=0.7.1Expected (IDs will differ):
VM ID: 20
VM ID: 21
Downloading packages, creating the database, pulling the container: this takes two to three minutes. Then:
IP=$(onevm show notes | grep -o 'ETH0_IP="[0-9.]*' | cut -d'"' -f2)
curl -s $IP:8080/version; echo
curl -s -H "X-API-Key: lab-api-key-123" -d '{"text":"Hallo OpenNebula"}' $IP:8080/notes; echoExpected (the ID will differ):
0.7.1
{"id":"602722bb-05ab-4559-ac6e-47609ba9ab2b","text":"Hallo OpenNebula"}
If curl does not answer, check where the script is.
one-context logs along, and ps shows whether the loop is
still waiting for Postgres:
ssh root@$IP "ps | grep -E 'nc -z|podman' | grep -v grep"Save your progress? Sign up for free
Drill 6: The data survives
The database VM is deleted and recreated; the disk remains:
onevm terminate dbWait half a minute until onevm list no longer shows the
VM, then:
onetemplate instantiate notes-db --name db --user-inputs DB_PASSWORD=lab-db-pass-123After two minutes:
curl -s $IP:8080/notes; echoExpected:
[{"id":"602722bb-05ab-4559-ac6e-47609ba9ab2b","text":"Hallo OpenNebula"}]
The note is there. The new db has the same internal IP
(fixed in the template), the same data disk, and the same configuration.
notes-api re-established the connection on its own.
Save your progress? Sign up for free
Checkpoint
onevm listExpected: db and notes, both
runn.
onevm show db | grep -A4 '^VM NICS'Expected (shortened):
0 vnet minionebr 192.168.150.22 02:00:c0:a8:96:16
1 intern intern0 10.10.10.20 02:00:0a:0a:0a:14
diff ~/one-lab/tag06/db.sh ~/one-lab/code/loesung/tag06/db.shExpected: no output.
Challenge
Is Postgres really reachable only internally? Check from a VM in
vnet whether the database’s port 5432 is open via its
vnet address, and via the internal address.
Note: Day 5 showed why you must not use the host for
this. notes is attached to both networks.
Solution
IP_DB=$(onevm show db | grep -o 'ETH0_IP="[0-9.]*' | cut -d'"' -f2)
ssh root@$IP "nc -w2 $IP_DB 5432 </dev/null && echo vnet offen || echo vnet zu; \
nc -w2 10.10.10.20 5432 </dev/null && echo intern offen || echo intern zu"Expected:
vnet zu
intern offen
Postgres listens on all cards (listen_addresses = '*').
The security group web on vnet blocks 5432
anyway; in the intern network only the default
group applies. Belt and suspenders: pg_hba.conf only allows
the user notes from 10.10.10.0/24 anyway.
Save your progress? Sign up for free
Enterprise Note
- Ready-made appliances: The Marketplace has
appliances that are configured with context variables, for example for
WordPress or the Virtual Router. The pattern is the same as here:
ONEAPP_*variables in the context, a script inside the VM. - Startup time: Installing packages on every start costs minutes and needs internet. In production, you build the image in advance (Day 10) or with Packer, and the startup script only configures.
- Databases as a service: If you need many databases, you build your own appliance from this with backups (Day 7) and monitoring (Day 11). That gives you a small “Database as a Service” on OpenNebula.