Day 9: Guest System Integration
Goal: Query and control the guest through the agent, adjust memory while running, share a host directory via virtiofs, and get to the graphical console. · Duration: approx. 75 min · Prerequisite: Day 8
Briefing
- virtio means: the guest knows it is virtual and uses simple paravirtualized devices instead of emulated hardware. Network, disk, console, randomness and memory balloon are all virtio in our VMs.
- The QEMU guest agent is a service in the guest. It talks to libvirt over a virtio channel and can provide information, freeze file systems, set passwords and run commands. It does not need a network.
- virtiofs mounts a host directory in the guest,
without NFS and without a network. For that, one
virtiofsdper share runs on the host.
Host guest web1
libvirt ── virtio-serial (agent) ──────── qemu-guest-agent
/srv/teilen ── virtiofsd ── vhost-user ── /mnt/teilen (type virtiofs)
QEMU ── virtio-balloon ────────────────── returns RAM
Drill 1: The virtio devices
ssh ubuntu@web1 "lspci | grep -i virtio"Expected:
01:00.0 Ethernet controller: Red Hat, Inc. Virtio 1.0 network device (rev 01)
03:00.0 Communication controller: Red Hat, Inc. Virtio 1.0 console (rev 01)
04:00.0 SCSI storage controller: Red Hat, Inc. Virtio 1.0 block device (rev 01)
05:00.0 Unclassified device [00ff]: Red Hat, Inc. Virtio 1.0 memory balloon (rev 01)
06:00.0 Unclassified device [00ff]: Red Hat, Inc. Virtio 1.0 RNG (rev 01)
07:00.0 SCSI storage controller: Red Hat, Inc. Virtio 1.0 block device (rev 01)
| Device | Task |
|---|---|
| network device | network card (enp1s0) |
| console | the channel for the QEMU guest agent |
| block device | vda and vdb |
| memory balloon | moving memory between host and guest |
| RNG | randomness from the host’s /dev/urandom, important for
SSH keys on first boot |
All of them come from “Red Hat, Inc.”: the virtio devices use the PCI vendor ID that belongs to Red Hat today.
Save your progress? Sign up for free
Drill 2: The QEMU guest agent provides information
web1 has had the agent since Day 4:
virsh guestinfo web1 --os --hostname --timezone
virsh domfsinfo web1Expected:
os.id : ubuntu
os.name : Ubuntu
os.pretty-name : Ubuntu 24.04.5 LTS
os.version : 24.04.5 LTS (Noble Numbat)
os.version-id : 24.04
os.machine : x86_64
os.kernel-release : 6.8.0-147-generic
os.kernel-version : #147-Ubuntu SMP PREEMPT_DYNAMIC Thu Sep 24 11:33:16 UTC 2026
timezone.name : UTC
timezone.offset : 0
hostname : web1
Mountpoint Name Type Target
-------------------------------------
/ vda1 ext4 vda
/boot vda16 ext4 vda
/boot/efi vda15 vfat vda
No SSH, no network: the answers come over the virtio channel. The agent can also write, for example set a password:
virsh set-user-password web1 ubuntu lab-console-pw-123Expected:
Password set successfully for ubuntu in web1
This lets you get back in via virsh console web1, even
when SSH is broken.
Tried it:
virsh domtime web1 --syncfails withguest-set-time: this feature or command is not currently supported. Ubuntu’s agent does not allow setting the clock. In the guest,systemd-timesyncdkeeps the time current.virsh domtime web1without--synconly reads it.
Save your progress? Sign up for free
Drill 3: Commands through the agent
For everything else there is the raw agent protocol.
guest-exec starts a program in the guest and returns a
process ID:
virsh qemu-agent-command web1 \
'{"execute":"guest-exec","arguments":{"path":"/usr/bin/uptime","capture-output":true}}'Expected (the PID will differ):
{"return":{"pid":826}}
With the PID you fetch the result:
virsh qemu-agent-command web1 '{"execute":"guest-exec-status","arguments":{"pid":826}}'Expected:
{"return":{"exitcode":0,"out-data":"IDE2OjM5OjE4IHVwIDEgbWluLCAgMCB1c2VyLCAgbG9hZCBhdmVyYWdlOiAxLjA5LCAwLjU4LCAwLjIyCg==","exited":true}}
The output is base64-encoded:
echo IDE2OjM5OjE4IHVwIDEgbWluLCAgMCB1c2VyLCAgbG9hZCBhdmVyYWdlOiAxLjA5LCAwLjU4LCAwLjIyCg== | base64 -dExpected:
16:39:18 up 1 min, 0 user, load average: 1.09, 0.58, 0.22
Warning: The result can only be fetched once. A second
guest-exec-statuswith the same PID ends withunable to execute QEMU agent command 'guest-exec-status': PID ld does not exist. Theldis a bug in the agent’s message. It means the PID.The agent runs everything as root. Whoever may use
virshon the host is thus root in every guest with an agent.
Save your progress? Sign up for free
Drill 4: Memory while running
virsh dommemstat web1Expected (shortened):
actual 1048576
…
unused 854232
available 975288
usable 783832
…
rss 519252
actual is what the guest has been given,
rss what the QEMU process actually occupies on the host.
The balloon driver in the guest can return memory:
ssh ubuntu@web1 "free -m | head -2"
virsh setmem web1 768M --live
sleep 5
ssh ubuntu@web1 "free -m | head -2"Expected:
total used free shared buff/cache available
Mem: 961 291 625 1 186 669
total used free shared buff/cache available
Mem: 705 286 374 1 186 419
The guest sees 256 MB less, without a reboot. You can go back up to the maximum, but not beyond:
virsh setmem web1 1024M --live
virsh setmem web1 2G --liveExpected:
error: invalid argument: cannot set memory higher than max memory
The maximum (setmaxmem) only changes with the VM powered
off, as in the Day 3 Challenge.
Save your progress? Sign up for free
Drill 5: Sharing a directory with virtiofs
sudo apt-get install -y virtiofsd
sudo mkdir -p /srv/teilen
echo "Hallo vom Host" | sudo tee /srv/teilen/gruss.txtvirtiofs needs memory that QEMU and virtiofsd use
together (shared memory). That and the share itself change the
definition, so do it with the VM powered off:
virsh shutdown web1virt-xml from the virtinst package (Day 3)
changes individual parts of the domain XML using options like those of
virt-install.
When virsh domstate web1 shows
shut off:
virt-xml web1 --edit --memorybacking source.type=memfd,access.mode=shared
virt-xml web1 --add-device --filesystem \
driver.type=virtiofs,source.dir=/srv/teilen,target.dir=teilen
virsh dumpxml web1 | grep -A3 -E "<memoryBacking|<filesystem"Expected:
Domain 'web1' defined successfully.
Domain 'web1' defined successfully.
<memoryBacking>
<source type='memfd'/>
<access mode='shared'/>
</memoryBacking>
--
<filesystem type='mount' accessmode='passthrough'>
<driver type='virtiofs'/>
<source dir='/srv/teilen'/>
<target dir='teilen'/>
target dir is not a path but a name under which the
guest finds the share.
virsh start web1
ps -o user,args -C virtiofsdExpected:
Domain 'web1' started
USER COMMAND
root /usr/libexec/virtiofsd --fd=44 -o source=/srv/teilen
root /usr/libexec/virtiofsd --fd=44 -o source=/srv/teilen
After booting, in the guest:
ssh ubuntu@web1sudo mkdir -p /mnt/teilen
sudo mount -t virtiofs teilen /mnt/teilen
cat /mnt/teilen/gruss.txt
echo "Hallo vom Gast" | sudo tee /mnt/teilen/antwort.txt
echo "teilen /mnt/teilen virtiofs defaults,nofail 0 0" | sudo tee -a /etc/fstab
exitExpected:
Hallo vom Host
Hallo vom Gast
teilen /mnt/teilen virtiofs defaults,nofail 0 0
And on the host:
ls -l /srv/teilenExpected:
total 8
-rw-r--r-- 1 root root 15 Nov 13 16:41 antwort.txt
-rw-r--r-- 1 root root 15 Nov 13 16:40 gruss.txt
The guest’s file is owned by root on the host.
accessmode='passthrough' passes the user IDs through
unchanged. root in the guest is root in /srv/teilen.
Tried it: With virtiofs,
virsh snapshot-create-as web1 testfails withcannot migrate domain: Migration disabled: vhost-user backend lacks VHOST_USER_PROTOCOL_F_LOG_SHMFD feature. A snapshot with RAM internally uses the migration machinery, andvirtiofsdcannot hand over its state. The same applies to live migration on Day 12.
Save your progress? Sign up for free
Drill 6: The graphical console
Since Day 4, web1 has no graphics
(--graphics none). notes and db
have the virt-install default:
virsh dumpxml notes | grep -A3 "<graphics"
virsh domdisplay notesExpected:
<graphics type='spice' port='5901' autoport='yes' listen='127.0.0.1'>
<listen type='address' address='127.0.0.1'/>
<image compression='off'/>
</graphics>
spice://127.0.0.1:5901
SPICE only listens on 127.0.0.1 of kvm1. Nobody gets at
it directly from outside, and that is a good thing: the console has no
password. You can still get a picture, without a viewer:
virsh screenshot notes /tmp/notes.png
file /tmp/notes.pngExpected:
Screenshot saved to /tmp/notes.png, with type of image/png
/tmp/notes.png: PNG image data, 1280 x 800, 8-bit/color RGB, non-interlaced
The image shows notes login:, the text screen of
tty1.
For a real viewer, you build an SSH tunnel from your machine and open
remote-viewer from the virt-viewer package
(Linux) or a SPICE app:
ssh -L 5901:127.0.0.1:5901 [email protected]As long as the connection is open,
remote-viewer spice://127.0.0.1:5901 on your machine shows
the console of notes. On Day 11, virt-manager
handles the tunneling by itself.
Save your progress? Sign up for free
Checkpoint
virsh guestinfo web1 --hostnameExpected: hostname : web1.
ssh ubuntu@web1 findmnt /mnt/teilenExpected: /mnt/teilen teilen virtiofs rw,relatime.
ls /srv/teilenExpected: antwort.txt and gruss.txt.
Challenge
db is only attached to the intern network.
Find out how full the Postgres data disk is, without
SSH.
Hint: guest-exec takes arguments as a
list under arg.
Solution
virsh qemu-agent-command db \
'{"execute":"guest-exec","arguments":{"path":"/usr/bin/df","arg":["-h","/var/lib/postgresql"],"capture-output":true}}'Expected (the PID will differ):
{"return":{"pid":921}}
virsh qemu-agent-command db '{"execute":"guest-exec-status","arguments":{"pid":921}}'Expected:
{"return":{"exitcode":0,"out-data":"RmlsZXN5c3RlbSAgICAgIFNpemUgIFVzZWQgQXZhaWwgVXNlJSBNb3VudGVkIG9uCi9kZXYvdmRiICAgICAgICA0LjlHICAgNDZNICA0LjZHICAgMSUgL3Zhci9saWIvcG9zdGdyZXNxbAo=","exited":true}}
echo RmlsZXN5c3RlbSAgICAgIFNpemUgIFVzZWQgQXZhaWwgVXNlJSBNb3VudGVkIG9uCi9kZXYvdmRiICAgICAgICA0LjlHICAgNDZNICA0LjZHICAgMSUgL3Zhci9saWIvcG9zdGdyZXNxbAo= | base64 -dExpected:
Filesystem Size Used Avail Use% Mounted on
/dev/vdb 4.9G 46M 4.6G 1% /var/lib/postgresql
Save your progress? Sign up for free
Enterprise Note
- Secure the agent: With
block-rpcsorallow-rpcsin/etc/qemu/qemu-ga.confyou restrict what the agent may do. In multi-tenant environments,guest-execshould be blocked. - Windows guests: Need the virtio-win drivers (ISO from Fedora); otherwise they see neither virtio disks nor virtio network. The agent is included there.
- Use virtiofs with care: For tenant VMs, an ID
mapping (
<idmap>) belongs with it, so that root in the guest is not root on the host. And virtiofs prevents live migration. - Console in production: Proxmox and OpenStack ship noVNC, a console in the browser that tunnels through the management server.