Day 2: Architecture and CLI
Goal: Know the parts of OpenNebula, read their logs, and understand how the scheduler finds a host for a VM, and why it sometimes finds none. · Duration: approx. 60 min · Prerequisite: Day 1
Briefing
onedis the core. It holds the state of all objects in the database and offers an XML-RPC API on port 2633. CLI, Sunstone, Terraform, everything talks to this API (Day 13).- Drivers do the actual work: the VMM driver starts
VMs via libvirt, the Transfer Manager (TM) copies disks, the IM driver
monitors hosts. These are scripts under
/var/lib/one/remotes. - The scheduler distributes waiting VMs to hosts. It calculates with what the templates request (allocated), not with what is actually consumed (real).
CLI / Sunstone / Terraform ── XML-RPC :2633 ──► oned ── MariaDB
│
┌──────────────┬──────────────┼──────────────┐
Scheduler VMM (kvm) TM (qcow2) IM (kvm)
wählt Host libvirt kopiert Disks überwacht Host
| Object | CLI | In short |
|---|---|---|
| Host | onehost |
a hypervisor |
| Datastore | onedatastore |
storage location for images or running VMs |
| Image | oneimage |
a disk |
| Template | onetemplate |
blueprint of a VM |
| VM | onevm |
a running or stopped instance |
| Virtual Network | onevnet |
a network with addresses |
| User, Group, ACL | oneuser, onegroup,
oneacl |
permissions (Day 8) |
Drill 1: Services and version
systemctl list-units 'opennebula*' --no-legend | cut -c1-80Expected (shortened):
opennebula-fireedge.service loaded active running OpenNebula FireEdge Ser
opennebula-flow.service loaded active running OpenNebula Flow Service
opennebula-gate.service loaded active running OpenNebula Gate Service
opennebula-hem.service loaded active running OpenNebula Hook Executi
...
opennebula.service loaded active running OpenNebula Cloud Contro
| Service | Job |
|---|---|
opennebula |
oned with scheduler |
opennebula-fireedge |
Sunstone, the web UI |
opennebula-flow |
OneFlow, services made of multiple VMs (Day 9) |
opennebula-gate |
OneGate, back channel from the VMs (Day 11) |
opennebula-hem |
Hook Execution Manager, runs hooks (Day 11) |
cat /var/lib/one/remotes/VERSION
onezone listExpected:
7.4.1
C ID NAME ENDPOINT FED_INDEX STAT
* 0 OpenNebula http://localhost:2633/RPC2 -1 on
The zone is one OpenNebula installation. Multiple zones can be joined into a federation. The endpoint shows the XML-RPC API.
Save your progress? Sign up for free
Drill 2: Where the logs are
sudo ls /var/log/one/Expected (shortened):
0.log 1.log 2.log 3.log fireedge.error fireedge.log monitor.log oned.log oneflow.log
onegate.log onehem.log rank_sched.log …
oned.log: everything the core does.<ID>.log: one file per VM, with every driver step (Day 1, Challenge).rank_sched.log: the scheduler’s decisions.monitor.log: host monitoring.
Save your progress? Sign up for free
Drill 3: Allocated and real
onehost show localhost | sed -n '/HOST SHARES/,/USED (ALLOCATED)/p'Expected (values will differ):
HOST SHARES
RUNNING VMS : 0
MEMORY
TOTAL : 3.8G
TOTAL +/- RESERVED : 3.8G
USED (REAL) : 1.6G
USED (ALLOCATED) : 0K
Below it follows the same block for the CPU.
- REAL measures what the host actually consumes, including OpenNebula itself and MariaDB.
- ALLOCATED only counts what the VMs get according to their templates. The scheduler decides based on this.
So to the scheduler, a host with 3.8 GB and zero allocated VMs looks
empty, even though 1.6 GB is already in use. RESERVED (Day
14) brings order here.
Save your progress? Sign up for free
Drill 4: A VM that does not fit
instantiate can override individual values of the
template. Ask for 4 GB:
onetemplate instantiate 1 --name gross --memory 4096Expected (your ID will differ):
VM ID: 4
Half a minute later:
onevm list
onevm show gross | grep -E '^STATE|SCHED_MESSAGE'Expected (shortened):
ID USER GROUP NAME STAT CPU MEM HOST TIME
4 oneadmin oneadmin gross pend 1 4G 0d 00h00
STATE : PENDING
SCHED_MESSAGE="Thu Nov 12 19:04:01 2026: [HOST] Not enough MEMORY on hosts: 0"
The VM waits in PENDING, and the scheduler writes the
reason into the VM itself. The host has 3.8 GB, the VM wants 4 GB. It
waits until a host becomes free, even for days.
Save your progress? Sign up for free
Drill 5: Output for scripts
The CLI tables are for humans. For scripts there are two ways: selected columns or JSON.
onevm list --list ID,NAME,STAT,MEMExpected:
ID NAME STAT MEM
4 gross pend 4G
onevm show gross --json | jq -r '.VM.USER_TEMPLATE.SCHED_MESSAGE'Expected:
Thu Nov 12 19:04:01 2026: [HOST] Not enough MEMORY on hosts: 0
miniONE installed jq along with everything else. With
-x instead of --json, you get the same data as
XML, the way the API delivers it.
Save your progress? Sign up for free
Checkpoint
onevm show gross | grep '^STATE'Expected: STATE : PENDING.
sudo grep -c 'Deploying VM' /var/log/one/oned.logExpected: a number greater than 0, one line per started VM.
Challenge
Get gross running without deleting and recreating
it.
Note: onevm has a command for changing
sizes. Look in onevm --help.
Solution
onevm resize gross --memory 512Half a minute later:
onevm listExpected:
ID USER GROUP NAME STAT CPU MEM HOST TIME
4 oneadmin oneadmin gross runn 1 512M localhost 0d 00h01
resize works on waiting and powered-off VMs. On running
VMs only if the template allows hotplug (Day 7). The oned
log shows when the scheduler deployed the VM:
sudo grep 'VM 4' /var/log/one/oned.log | tail -2Expected (shortened):
… [Z0][DiM][D]: Deploying VM 4
Clean up:
onevm terminate --hard gross--hard switches the VM off immediately without giving
the operating system time to shut down. For a throwaway VM, that is
fine.
Save your progress? Sign up for free
Enterprise Note
- Overcommitting: In templates,
CPU(share) andVCPU(visible cores) let you overcommit CPU deliberately, for exampleCPU=0.25withVCPU=2. Better not to overcommit RAM, otherwise the host’s OOM killer steps in. - Scheduler rules:
SCHED_REQUIREMENTSandSCHED_RANKin the template control where a VM may go and which host is preferred. This lets you separate production and test onto different hosts, for example. - Central logs: In production,
oned.logand the VM logs go to a central system. The VM logs contain every driver step and are the first source during incidents.