Day 3: libvirt Takes Over
Goal: Install libvirt, describe the VM from Day 1 as a libvirt domain, and start, inspect and shut it down with
virsh. · Duration: approx. 60 min · Prerequisite: Day 2
Briefing
- libvirtd is a service that starts and monitors QEMU processes for you. There a VM is called a domain and is described in an XML file.
virshis the matching command-line tool. It talks to libvirtd through a connection URI:qemu:///systemfor the system’s VMs,qemu:///sessionfor your user’s private VMs.- The QEMU processes no longer run under your name but as the user
libvirt-qemu, protected by AppArmor. This has consequences for where disks are allowed to live.
virsh ──► libvirtd ──► qemu-system-x86_64 (user libvirt-qemu)
│ └─ vda: /var/lib/libvirt/images/erste.qcow2
├─ /etc/libvirt/qemu/erste.xml definition
└─ network default: virbr0 192.168.122.1 (NAT + DHCP)
Drill 1: Install libvirt
sudo apt-get install -y libvirt-daemon-system libvirt-clients virtinst
sudo usermod -aG libvirt $USER
virsh --versionExpected:
10.0.0
Log out and back in so that the libvirt group applies.
Then:
virsh uriExpected:
qemu:///session
As a regular user, virsh lands in your private session.
You do not see the system’s VMs there. Change the default:
echo 'export LIBVIRT_DEFAULT_URI=qemu:///system' >> ~/.bashrc
source ~/.bashrc
virsh uriExpected:
qemu:///system
Without the libvirt group, the same command ends with
Failed to connect socket to '/var/run/libvirt/libvirt-sock': Permission denied.
Save your progress? Sign up for free
Drill 2: Can the host do it?
virt-host-validate qemuExpected (shortened):
QEMU: Checking for hardware virtualization : PASS
QEMU: Checking if device /dev/kvm exists : PASS
QEMU: Checking if device /dev/kvm is accessible : PASS
QEMU: Checking if device /dev/vhost-net exists : PASS
…
QEMU: Checking for cgroup 'devices' controller support : WARN (Enable 'devices' …)
QEMU: Checking for device assignment IOMMU support : WARN (No ACPI DMAR table …)
Everything with PASS counts. The warnings concern cgroup
v2, which no longer has its own devices controller, and the
IOMMU, which you only need for passing through real devices. You can
ignore both here.
Save your progress? Sign up for free
Drill 3: The default network
virsh net-list --all
ip -br a show virbr0Expected:
Name State Autostart Persistent
--------------------------------------------
default active yes yes
virbr0 UP 192.168.122.1/24
libvirt ships a network called default: the bridge
virbr0, NAT to the internet and a DHCP server (dnsmasq) for
192.168.122.0/24. On Day 6 you build your own networks.
Save your progress? Sign up for free
Drill 4: The first domain
mkdir -p ~/kvm-lab/tag03 && cd ~/kvm-lab/tag03
cat > erste.xml <<'EOF'
<domain type="kvm">
<name>erste</name>
<memory unit="MiB">1024</memory>
<vcpu>2</vcpu>
<os>
<type arch="x86_64" machine="q35">hvm</type>
</os>
<cpu mode="host-passthrough"/>
<devices>
<disk type="file" device="disk">
<driver name="qemu" type="qcow2"/>
<source file="/home/ubuntu/kvm-lab/tag01/erste.qcow2"/>
<target dev="vda" bus="virtio"/>
</disk>
<interface type="network">
<source network="default"/>
<model type="virtio"/>
</interface>
<serial type="pty"/>
<console type="pty"/>
</devices>
</domain>
EOFAlmost every line has a counterpart in the QEMU invocation from Day
1: memory is -m, vcpu is
-smp, cpu mode="host-passthrough" is
-cpu host, the disk is -drive.
New are the machine type q35 and
interface type="network": the VM is attached to the
default network instead of the user-mode network.
virsh define erste.xml
virsh start ersteExpected:
Domain 'erste' defined from erste.xml
error: Failed to start domain 'erste'
error: Cannot access storage file '/home/ubuntu/kvm-lab/tag01/erste.qcow2' (as uid:64055,
gid:993): Permission denied
define worked, start did not. Who is uid
64055?
getent passwd 64055Expected:
libvirt-qemu:x:64055:993:Libvirt Qemu,,,:/var/lib/libvirt:/usr/sbin/nologin
QEMU runs as libvirt-qemu and may not enter your home
directory. That is intentional: a hijacked QEMU process should see as
little as possible. Disks belong in
/var/lib/libvirt/images.
Save your progress? Sign up for free
Drill 5: The disk in its place
sudo mv ~/kvm-lab/tag01/erste.qcow2 /var/lib/libvirt/images/
virsh undefine ersteExpected:
Domain 'erste' has been undefined
undefine only deletes the definition, not the disk.
Tried it: Without
undefine, a secondvirsh define erste.xmlfails withdomain 'erste' already exists with uuid …. Your file contains no UUID, so libvirt considers it a new domain with a name that is already taken. In daily work you change domains withvirsh edit erste, which opens the complete XML in the editor.
Rewrite the file, with the correct path and a fixed MAC address:
cat > erste.xml <<'EOF'
<domain type="kvm">
<name>erste</name>
<memory unit="MiB">1024</memory>
<vcpu>2</vcpu>
<os>
<type arch="x86_64" machine="q35">hvm</type>
</os>
<cpu mode="host-passthrough"/>
<devices>
<disk type="file" device="disk">
<driver name="qemu" type="qcow2"/>
<source file="/var/lib/libvirt/images/erste.qcow2"/>
<target dev="vda" bus="virtio"/>
</disk>
<interface type="network">
<source network="default"/>
<mac address="52:54:00:12:34:56"/>
<model type="virtio"/>
</interface>
<serial type="pty"/>
<console type="pty"/>
</devices>
</domain>
EOF
virsh define erste.xml
virsh start ersteExpected:
Domain 'erste' defined from erste.xml
Domain 'erste' started
Why the fixed MAC 52:54:00:12:34:56? That is the MAC
QEMU assigned on Day 1 without libvirt. On first boot, cloud-init bound
the network configuration to exactly this address:
sudo apt-get install -y libguestfs-tools
virsh destroy erste
sudo virt-cat -a /var/lib/libvirt/images/erste.qcow2 /etc/netplan/50-cloud-init.yaml
virsh start erstedestroy powers the VM off hard, but deletes nothing
(more in Drill 8).
Expected (after the packages have downloaded):
network:
version: 2
ethernets:
ens3:
match:
macaddress: "52:54:00:12:34:56"
dhcp4: true
…
Tried it: Without
<mac>, libvirt assigns a random MAC, and netplan in the guest does not consider itself responsible for the new card. The VM runs but never gets an IP address. This happens to everyone who takes a disk from one VM into another.virt-catreads files directly from a disk without starting the VM. It is only reliable with the VM powered off; otherwise you see half-written data.
Save your progress? Sign up for free
Drill 6: Where is the VM?
Give the guest a minute, then:
virsh list --all
virsh domifaddr ersteExpected (your IPs will differ):
Id Name State
-----------------------
3 erste running
Name MAC address Protocol Address
-------------------------------------------------------------------------------
vnet1 52:54:00:12:34:56 ipv4 192.168.122.76/24
domifaddr asks the network’s DHCP server. If the list
stays empty, the guest is not ready yet or has not received an IP. The
full list of all leases is shown by:
virsh net-dhcp-leases defaultExpected (shortened):
Expiry Time MAC address Protocol IP address Hostname …
--------------------------------------------------------------------------------------
2026-11-13 14:17:48 52:54:00:12:34:56 ipv4 192.168.122.76/24 erste …
Now you can reach the guest directly, without port forwarding. Use
the IP from domifaddr:
ssh -o StrictHostKeyChecking=no [email protected] lspciExpected (shortened):
…
01:00.0 Ethernet controller: Red Hat, Inc. Virtio 1.0 network device (rev 01)
03:00.0 SCSI storage controller: Red Hat, Inc. Virtio 1.0 block device (rev 01)
04:00.0 Unclassified device [00ff]: Red Hat, Inc. Virtio 1.0 memory balloon (rev 01)
libvirt added the memory balloon on its own. More on that in the Challenge.
Save your progress? Sign up for free
Drill 7: What libvirt makes of it
virsh dumpxml erste | head -18
virsh dumpxml erste | wc -lExpected (shortened):
<domain type='kvm' id='3'>
<name>erste</name>
<uuid>387e1a17-29a0-4d2d-a2e6-5da8a969ed39</uuid>
<memory unit='KiB'>1048576</memory>
<currentMemory unit='KiB'>1048576</currentMemory>
<vcpu placement='static'>2</vcpu>
…
<os>
<type arch='x86_64' machine='pc-q35-noble'>hvm</type>
<boot dev='hd'/>
</os>
<cpu mode='host-passthrough' check='none' migratable='on'/>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
114
23 lines became 114. libvirt fills in everything missing with
defaults: UUID, machine type pc-q35-noble, clock, USB
controller, PCI addresses. The stored definition lives here:
sudo ls /etc/libvirt/qemu/Expected:
erste.xml
networks
Warning: Never edit these files directly. libvirtd only reads them at startup. Always make changes through
virsh defineorvirsh edit.
And the actual QEMU command line is in the domain’s log:
sudo head -12 /var/log/libvirt/qemu/erste.logExpected (shortened):
…
/usr/bin/qemu-system-x86_64 \
-name guest=erste,debug-threads=on \
-S \
-object '{"qom-type":"secret","id":"masterKey0",…
-machine pc-q35-noble,usb=off,dump-guest-core=off,memory-backend=pc.ram,… \
-accel kvm \
-cpu host,migratable=on \
…
That is your QEMU invocation from Day 1, just much more verbose.
Save your progress? Sign up for free
Drill 8: Console and shutdown
virsh console erstePress Enter, log in as ubuntu with
lab-console-pw-123, and log out again. You leave the
console with Ctrl+]. This works thanks to
<serial> and <console> in the XML,
even when the network in the guest is broken.
virsh shutdown ersteExpected:
Domain 'erste' is being shutdown
Wait a few seconds, then:
virsh domstate --reason ersteExpected:
shut off (shutdown)
| Command | Effect |
|---|---|
virsh shutdown |
asks the guest via ACPI to shut down |
virsh destroy |
ends QEMU immediately, like pulling the plug |
virsh reboot |
asks the guest to reboot |
virsh reset |
hard reboot without shutting down |
destroy sounds dramatic but deletes nothing. The
definition and the disk remain.
Save your progress? Sign up for free
Checkpoint
virsh uriExpected: qemu:///system.
virsh list --allExpected: erste with the state
shut off.
virsh domblklist ersteExpected: vda points to
/var/lib/libvirt/images/erste.qcow2.
Challenge
Give erste 2 GB RAM and 4 vCPUs permanently, without
touching the XML file, and check the vCPUs in the guest.
Hint: virsh setmaxmem,
setmem and setvcpus know the
--config option for the stored definition. The maximum has
to go up first.
Solution
virsh setmaxmem erste 2G --config
virsh setmem erste 2G --config
virsh setvcpus erste 4 --config --maximum
virsh setvcpus erste 4 --config
virsh dominfo erste | grep -E 'CPU|memory'Expected:
CPU(s): 4
Max memory: 2097152 KiB
Used memory: 2097152 KiB
virsh start erste
ssh [email protected] nprocExpected (after a minute):
4
With --live, setmem even works on a running
VM: the balloon driver in the guest then returns memory to the host or
takes it back, up to maxmem at most.
Shut erste down with virsh shutdown erste,
but do not delete it. You clear it away on Day 5.
Save your progress? Sign up for free
Enterprise Note
- Session only for desktops:
qemu:///sessionis meant for developer laptops. Server VMs run inqemu:///system, because only there do VMs start when the host boots, and libvirt networks and bridges work without detours. - The libvirt group is almost root: Whoever may
control libvirt can start a VM with the host’s
/as a disk. Grant the group as carefully as sudo. - Back up definitions: The
virsh dumpxmlof every domain belongs in the backup or in Git. - Fixed MACs: Platforms like Proxmox and OpenStack store the MAC of every card so that guests keep their network after moves and reboots.