Skip to content
Hands-on IT IT Drills

Day 3: libvirt Takes Over

Goal: Install libvirt, describe the VM from Day 1 as a libvirt domain, and start, inspect and shut it down with virsh. · Duration: approx. 60 min · Prerequisite: Day 2

Briefing

  • libvirtd is a service that starts and monitors QEMU processes for you. There a VM is called a domain and is described in an XML file.
  • virsh is the matching command-line tool. It talks to libvirtd through a connection URI: qemu:///system for the system’s VMs, qemu:///session for your user’s private VMs.
  • The QEMU processes no longer run under your name but as the user libvirt-qemu, protected by AppArmor. This has consequences for where disks are allowed to live.
 virsh ──► libvirtd ──► qemu-system-x86_64 (user libvirt-qemu)
            │            └─ vda: /var/lib/libvirt/images/erste.qcow2
            ├─ /etc/libvirt/qemu/erste.xml        definition
            └─ network default: virbr0 192.168.122.1 (NAT + DHCP)

Drill 1: Install libvirt

sudo apt-get install -y libvirt-daemon-system libvirt-clients virtinst
sudo usermod -aG libvirt $USER
virsh --version

Expected:

10.0.0

Log out and back in so that the libvirt group applies. Then:

virsh uri

Expected:

qemu:///session

As a regular user, virsh lands in your private session. You do not see the system’s VMs there. Change the default:

echo 'export LIBVIRT_DEFAULT_URI=qemu:///system' >> ~/.bashrc
source ~/.bashrc
virsh uri

Expected:

qemu:///system

Without the libvirt group, the same command ends with Failed to connect socket to '/var/run/libvirt/libvirt-sock': Permission denied.

Save your progress? Sign up for free

Drill 2: Can the host do it?

virt-host-validate qemu

Expected (shortened):

  QEMU: Checking for hardware virtualization                 : PASS
  QEMU: Checking if device /dev/kvm exists                   : PASS
  QEMU: Checking if device /dev/kvm is accessible            : PASS
  QEMU: Checking if device /dev/vhost-net exists             : PASS
  …
  QEMU: Checking for cgroup 'devices' controller support     : WARN (Enable 'devices' …)
  QEMU: Checking for device assignment IOMMU support         : WARN (No ACPI DMAR table …)

Everything with PASS counts. The warnings concern cgroup v2, which no longer has its own devices controller, and the IOMMU, which you only need for passing through real devices. You can ignore both here.

Save your progress? Sign up for free

Drill 3: The default network

virsh net-list --all
ip -br a show virbr0

Expected:

 Name      State    Autostart   Persistent
--------------------------------------------
 default   active   yes         yes

virbr0           UP             192.168.122.1/24 

libvirt ships a network called default: the bridge virbr0, NAT to the internet and a DHCP server (dnsmasq) for 192.168.122.0/24. On Day 6 you build your own networks.

Save your progress? Sign up for free

Drill 4: The first domain

mkdir -p ~/kvm-lab/tag03 && cd ~/kvm-lab/tag03
cat > erste.xml <<'EOF'
<domain type="kvm">
  <name>erste</name>
  <memory unit="MiB">1024</memory>
  <vcpu>2</vcpu>
  <os>
    <type arch="x86_64" machine="q35">hvm</type>
  </os>
  <cpu mode="host-passthrough"/>
  <devices>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2"/>
      <source file="/home/ubuntu/kvm-lab/tag01/erste.qcow2"/>
      <target dev="vda" bus="virtio"/>
    </disk>
    <interface type="network">
      <source network="default"/>
      <model type="virtio"/>
    </interface>
    <serial type="pty"/>
    <console type="pty"/>
  </devices>
</domain>
EOF

Almost every line has a counterpart in the QEMU invocation from Day 1: memory is -m, vcpu is -smp, cpu mode="host-passthrough" is -cpu host, the disk is -drive. New are the machine type q35 and interface type="network": the VM is attached to the default network instead of the user-mode network.

virsh define erste.xml
virsh start erste

Expected:

Domain 'erste' defined from erste.xml

error: Failed to start domain 'erste'
error: Cannot access storage file '/home/ubuntu/kvm-lab/tag01/erste.qcow2' (as uid:64055,
gid:993): Permission denied

define worked, start did not. Who is uid 64055?

getent passwd 64055

Expected:

libvirt-qemu:x:64055:993:Libvirt Qemu,,,:/var/lib/libvirt:/usr/sbin/nologin

QEMU runs as libvirt-qemu and may not enter your home directory. That is intentional: a hijacked QEMU process should see as little as possible. Disks belong in /var/lib/libvirt/images.

Save your progress? Sign up for free

Drill 5: The disk in its place

sudo mv ~/kvm-lab/tag01/erste.qcow2 /var/lib/libvirt/images/
virsh undefine erste

Expected:

Domain 'erste' has been undefined

undefine only deletes the definition, not the disk.

Tried it: Without undefine, a second virsh define erste.xml fails with domain 'erste' already exists with uuid …. Your file contains no UUID, so libvirt considers it a new domain with a name that is already taken. In daily work you change domains with virsh edit erste, which opens the complete XML in the editor.

Rewrite the file, with the correct path and a fixed MAC address:

cat > erste.xml <<'EOF'
<domain type="kvm">
  <name>erste</name>
  <memory unit="MiB">1024</memory>
  <vcpu>2</vcpu>
  <os>
    <type arch="x86_64" machine="q35">hvm</type>
  </os>
  <cpu mode="host-passthrough"/>
  <devices>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2"/>
      <source file="/var/lib/libvirt/images/erste.qcow2"/>
      <target dev="vda" bus="virtio"/>
    </disk>
    <interface type="network">
      <source network="default"/>
      <mac address="52:54:00:12:34:56"/>
      <model type="virtio"/>
    </interface>
    <serial type="pty"/>
    <console type="pty"/>
  </devices>
</domain>
EOF
virsh define erste.xml
virsh start erste

Expected:

Domain 'erste' defined from erste.xml

Domain 'erste' started

Why the fixed MAC 52:54:00:12:34:56? That is the MAC QEMU assigned on Day 1 without libvirt. On first boot, cloud-init bound the network configuration to exactly this address:

sudo apt-get install -y libguestfs-tools
virsh destroy erste
sudo virt-cat -a /var/lib/libvirt/images/erste.qcow2 /etc/netplan/50-cloud-init.yaml
virsh start erste

destroy powers the VM off hard, but deletes nothing (more in Drill 8).

Expected (after the packages have downloaded):

network:
  version: 2
  ethernets:
    ens3:
      match:
        macaddress: "52:54:00:12:34:56"
      dhcp4: true
…

Tried it: Without <mac>, libvirt assigns a random MAC, and netplan in the guest does not consider itself responsible for the new card. The VM runs but never gets an IP address. This happens to everyone who takes a disk from one VM into another. virt-cat reads files directly from a disk without starting the VM. It is only reliable with the VM powered off; otherwise you see half-written data.

Save your progress? Sign up for free

Drill 6: Where is the VM?

Give the guest a minute, then:

virsh list --all
virsh domifaddr erste

Expected (your IPs will differ):

 Id   Name    State
-----------------------
 3    erste   running

 Name       MAC address          Protocol     Address
-------------------------------------------------------------------------------
 vnet1      52:54:00:12:34:56    ipv4         192.168.122.76/24

domifaddr asks the network’s DHCP server. If the list stays empty, the guest is not ready yet or has not received an IP. The full list of all leases is shown by:

virsh net-dhcp-leases default

Expected (shortened):

 Expiry Time           MAC address         Protocol   IP address          Hostname …
--------------------------------------------------------------------------------------
 2026-11-13 14:17:48   52:54:00:12:34:56   ipv4       192.168.122.76/24   erste    …

Now you can reach the guest directly, without port forwarding. Use the IP from domifaddr:

ssh -o StrictHostKeyChecking=no [email protected] lspci

Expected (shortened):

…
01:00.0 Ethernet controller: Red Hat, Inc. Virtio 1.0 network device (rev 01)
03:00.0 SCSI storage controller: Red Hat, Inc. Virtio 1.0 block device (rev 01)
04:00.0 Unclassified device [00ff]: Red Hat, Inc. Virtio 1.0 memory balloon (rev 01)

libvirt added the memory balloon on its own. More on that in the Challenge.

Save your progress? Sign up for free

Drill 7: What libvirt makes of it

virsh dumpxml erste | head -18
virsh dumpxml erste | wc -l

Expected (shortened):

<domain type='kvm' id='3'>
  <name>erste</name>
  <uuid>387e1a17-29a0-4d2d-a2e6-5da8a969ed39</uuid>
  <memory unit='KiB'>1048576</memory>
  <currentMemory unit='KiB'>1048576</currentMemory>
  <vcpu placement='static'>2</vcpu>
  …
  <os>
    <type arch='x86_64' machine='pc-q35-noble'>hvm</type>
    <boot dev='hd'/>
  </os>
  <cpu mode='host-passthrough' check='none' migratable='on'/>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
114

23 lines became 114. libvirt fills in everything missing with defaults: UUID, machine type pc-q35-noble, clock, USB controller, PCI addresses. The stored definition lives here:

sudo ls /etc/libvirt/qemu/

Expected:

erste.xml
networks

Warning: Never edit these files directly. libvirtd only reads them at startup. Always make changes through virsh define or virsh edit.

And the actual QEMU command line is in the domain’s log:

sudo head -12 /var/log/libvirt/qemu/erste.log

Expected (shortened):

…
/usr/bin/qemu-system-x86_64 \
-name guest=erste,debug-threads=on \
-S \
-object '{"qom-type":"secret","id":"masterKey0",…
-machine pc-q35-noble,usb=off,dump-guest-core=off,memory-backend=pc.ram,… \
-accel kvm \
-cpu host,migratable=on \
…

That is your QEMU invocation from Day 1, just much more verbose.

Save your progress? Sign up for free

Drill 8: Console and shutdown

virsh console erste

Press Enter, log in as ubuntu with lab-console-pw-123, and log out again. You leave the console with Ctrl+]. This works thanks to <serial> and <console> in the XML, even when the network in the guest is broken.

virsh shutdown erste

Expected:

Domain 'erste' is being shutdown

Wait a few seconds, then:

virsh domstate --reason erste

Expected:

shut off (shutdown)
Command Effect
virsh shutdown asks the guest via ACPI to shut down
virsh destroy ends QEMU immediately, like pulling the plug
virsh reboot asks the guest to reboot
virsh reset hard reboot without shutting down

destroy sounds dramatic but deletes nothing. The definition and the disk remain.

Save your progress? Sign up for free

Checkpoint

virsh uri

Expected: qemu:///system.

virsh list --all

Expected: erste with the state shut off.

virsh domblklist erste

Expected: vda points to /var/lib/libvirt/images/erste.qcow2.

Challenge

Give erste 2 GB RAM and 4 vCPUs permanently, without touching the XML file, and check the vCPUs in the guest.

Hint: virsh setmaxmem, setmem and setvcpus know the --config option for the stored definition. The maximum has to go up first.

Solution

virsh setmaxmem erste 2G --config
virsh setmem erste 2G --config
virsh setvcpus erste 4 --config --maximum
virsh setvcpus erste 4 --config
virsh dominfo erste | grep -E 'CPU|memory'

Expected:

CPU(s):         4
Max memory:     2097152 KiB
Used memory:    2097152 KiB
virsh start erste
ssh [email protected] nproc

Expected (after a minute):

4

With --live, setmem even works on a running VM: the balloon driver in the guest then returns memory to the host or takes it back, up to maxmem at most.

Shut erste down with virsh shutdown erste, but do not delete it. You clear it away on Day 5.

Save your progress? Sign up for free

Enterprise Note

  • Session only for desktops: qemu:///session is meant for developer laptops. Server VMs run in qemu:///system, because only there do VMs start when the host boots, and libvirt networks and bridges work without detours.
  • The libvirt group is almost root: Whoever may control libvirt can start a VM with the host’s / as a disk. Grant the group as carefully as sudo.
  • Back up definitions: The virsh dumpxml of every domain belongs in the backup or in Git.
  • Fixed MACs: Platforms like Proxmox and OpenStack store the MAC of every card so that guests keep their network after moves and reboots.