Skip to content
Hands-on IT IT Drills

Day 13: VMs as Code with OpenTofu

Goal: Describe, create, change and tear down VMs, disks and cloud-init with OpenTofu and the libvirt provider. And know the provider’s limits. · Duration: approx. 75 min · Prerequisite: Day 12

Briefing

  • OpenTofu is the open fork of Terraform. You describe the desired state in .tf files. tofu plan shows the difference from the actual state, tofu apply establishes it.
  • The dmacvicar/libvirt provider translates resources into libvirt objects: libvirt_volume becomes a volume, libvirt_domain a domain, libvirt_cloudinit_disk a seed ISO.
  • Since version 0.9 the provider maps the domain XML almost one to one: devices.disks, devices.interfaces and so on. If you know the XML from Day 3, you will find your way around.
  • OpenTofu remembers what it has created in the state (terraform.tfstate). What is not in the state, it does not touch.
 main.tf ── tofu apply ──► libvirt on kvm1
   for_each namen = ["tofu1", "tofu2"]
     libvirt_volume.platte     tofu1.qcow2 (overlay on basis/noble.qcow2)
     libvirt_cloudinit_disk    build seed ISO
     libvirt_volume.seed       tofu1-seed.iso in pool default
     libvirt_domain.vm         domain tofu1

Drill 1: Install OpenTofu

On kvm1, as a package from the OpenTofu releases:

cd /tmp
wget -q \
  https://github.com/opentofu/opentofu/releases/download/v1.13.9/tofu_1.13.9_amd64.deb
sudo apt-get install -y ./tofu_1.13.9_amd64.deb
tofu version

Expected (shortened):

OpenTofu v1.13.9
on linux_amd64

Save your progress? Sign up for free

Drill 2: The configuration

mkdir -p ~/kvm-lab/tag13
cd ~/kvm-lab/tag13
cat > main.tf <<'EOF'
terraform {
  required_providers {
    libvirt = {
      source  = "dmacvicar/libvirt"
      version = "0.9.9"
    }
  }
}

provider "libvirt" {
  uri = "qemu:///system"
}

variable "namen" {
  type    = set(string)
  default = ["tofu1", "tofu2"]
}

resource "libvirt_volume" "platte" {
  for_each = var.namen
  name     = "${each.key}.qcow2"
  pool     = "default"
  capacity = 10737418240
  target = {
    format = { type = "qcow2" }
  }
  backing_store = {
    path   = "/var/lib/libvirt/images/basis/noble.qcow2"
    format = { type = "qcow2" }
  }
}

resource "libvirt_cloudinit_disk" "seed" {
  for_each  = var.namen
  name      = "${each.key}-seed"
  user_data = <<-EOT
    #cloud-config
    users:
      - name: ubuntu
        sudo: ALL=(ALL) NOPASSWD:ALL
        shell: /bin/bash
        ssh_authorized_keys:
          - ${trimspace(file(pathexpand("~/.ssh/id_ed25519.pub")))}
  EOT
  meta_data = <<-EOT
    instance-id: ${each.key}-001
    local-hostname: ${each.key}
  EOT
}

resource "libvirt_volume" "seed" {
  for_each = var.namen
  name     = "${each.key}-seed.iso"
  pool     = "default"
  create = {
    content = { url = libvirt_cloudinit_disk.seed[each.key].path }
  }
}

resource "libvirt_domain" "vm" {
  for_each    = var.namen
  name        = each.key
  type        = "kvm"
  memory      = 768
  memory_unit = "MiB"
  vcpu        = 1
  running     = true

  os = {
    type         = "hvm"
    type_arch    = "x86_64"
    type_machine = "q35"
  }

  cpu = {
    mode = "host-passthrough"
  }

  devices = {
    disks = [
      {
        source = {
          file = { file = libvirt_volume.platte[each.key].path }
        }
        target = { bus = "virtio", dev = "vda" }
        driver = { type = "qcow2" }
      },
      {
        device = "cdrom"
        source = {
          file = { file = libvirt_volume.seed[each.key].path }
        }
        target = { bus = "sata", dev = "sda" }
      }
    ]
    interfaces = [
      {
        type   = "network"
        model  = { type = "virtio" }
        source = { network = { network = "default" } }
      }
    ]
    consoles = [
      {
        type   = "pty"
        target = { type = "serial", port = 0 }
      }
    ]
  }
}
EOF
  • for_each = var.namen creates each resource once per name. each.key is the name.
  • capacity is in bytes, memory with memory_unit = "MiB" in MiB.
  • libvirt_cloudinit_disk builds the ISO locally, libvirt_volume.seed uploads it into the pool, just like virt-install on Day 11.

A second file queries the addresses:

cat > ausgabe.tf <<'EOF'
data "libvirt_domain_interface_addresses" "ip" {
  for_each = libvirt_domain.vm
  domain   = each.value.name
  source   = "lease"
}

output "adressen" {
  value = {
    for name, d in data.libvirt_domain_interface_addresses.ip :
    name => try(d.interfaces[0].addrs[0].addr, "noch keine")
  }
}
EOF

Save your progress? Sign up for free

Drill 3: init, plan, apply

tofu init

Expected (shortened):

Initializing the backend...

Initializing provider plugins...
- Finding dmacvicar/libvirt versions matching "0.9.9"...
- Installing dmacvicar/libvirt v0.9.9...
- Installed dmacvicar/libvirt v0.9.9. Signature validation was skipped due to the registry not containing GPG keys for this provider
…

Warning: The provider’s signature is not verified. The pinned version 0.9.9 and the .terraform.lock.hcl file with the checksums make sure that every run gets exactly this build. The file belongs in Git.

tofu plan | grep -E "^Plan:|will be created"

Expected:

  # libvirt_cloudinit_disk.seed["tofu1"] will be created
  # libvirt_cloudinit_disk.seed["tofu2"] will be created
  # libvirt_domain.vm["tofu1"] will be created
  # libvirt_domain.vm["tofu2"] will be created
  # libvirt_volume.platte["tofu1"] will be created
  # libvirt_volume.platte["tofu2"] will be created
  # libvirt_volume.seed["tofu1"] will be created
  # libvirt_volume.seed["tofu2"] will be created
Plan: 8 to add, 0 to change, 0 to destroy.
tofu apply -auto-approve

Expected (shortened):

…
Outputs:

adressen = {
  "tofu1" = "noch keine"
  "tofu2" = "noch keine"
}

Tried it: With the example from the provider docs, that is source = { volume = { pool = …, volume = … } }, the start failed with Could not open '/var/lib/libvirt/images/basis/noble.qcow2': Permission denied, even though the file is readable by everyone. The domain then gets disks of type volume. After switching to type='file' with virt-xml, the same disk started, and the error moved on to the seed ISO, which was still volume. With the file path (source = { file = … }) you get type='file' as with virt-install, and both VMs start.

The VMs are running but do not have a lease yet. A minute later, a second apply fetches the addresses without changing anything:

tofu apply -auto-approve
tofu output adressen

Expected (your addresses will differ):

…
adressen = {
  "tofu1" = "192.168.122.114"
  "tofu2" = "192.168.122.152"
}
ssh [email protected] "hostname; cloud-init status --wait | tail -1"

Expected (after confirming the host key):

tofu1
status: done

Save your progress? Sign up for free

Drill 4: Drift

What if someone makes changes behind OpenTofu’s back?

virsh setmaxmem tofu1 1G --config
tofu plan | tail -1

Expected:

No changes. Your infrastructure matches the configuration.

Tried it: Provider 0.9.9 does not notice the changed memory. It does not compare every field of the domain XML with the desired value. In the test it only detected that the VM was no longer running.

virsh shutdown tofu1

When tofu1 is off:

tofu plan | grep -E "^\s+~ |^Plan:"

Expected:

  ~ resource "libvirt_domain" "vm" {
      ~ running     = false -> true
Plan: 0 to add, 1 to change, 0 to destroy.

apply would start tofu1, but with a 1 GiB maximum instead of 768 MiB. A replacement enforces the desired state:

tofu apply -auto-approve \
  -replace='libvirt_domain.vm["tofu1"]' | grep -E "replace|^Apply"
virsh dumpxml tofu1 --inactive | grep "<memory"

Expected:

-/+ destroy and then create replacement
  # libvirt_domain.vm["tofu1"] will be replaced, as requested
Apply complete! Resources: 1 added, 0 changed, 1 destroyed.
  <memory unit='KiB'>786432</memory>

Only the domain is replaced. The disk is a separate resource and keeps its contents.

Save your progress? Sign up for free

Checkpoint

tofu state list | wc -l

Expected: 10 (8 resources and 2 entries of the ip data source).

virsh list | grep tofu

Expected: tofu1 and tofu2, both running.

tofu plan | tail -1

Expected: No changes. Your infrastructure matches the configuration.

Challenge

tofu2 is no longer needed. Remove it without changing main.tf and without touching tofu1. Then tear everything down and check that the base image is still there.

Hint: Variables can be overridden with -var.

Solution

tofu plan -var 'namen=["tofu1"]' | grep -E "will be destroyed|^Plan:"

Expected:

  # libvirt_cloudinit_disk.seed["tofu2"] will be destroyed
  # libvirt_domain.vm["tofu2"] will be destroyed
  # libvirt_volume.platte["tofu2"] will be destroyed
  # libvirt_volume.seed["tofu2"] will be destroyed
Plan: 0 to add, 0 to change, 4 to destroy.
tofu apply -auto-approve -var 'namen=["tofu1"]' | grep "^Apply"
virsh list --all | grep tofu

Expected:

Apply complete! Resources: 0 added, 0 changed, 4 destroyed.
 34   tofu1   running

And everything gone:

tofu destroy -auto-approve | grep "^Destroy"
virsh list --all | grep -c tofu
sudo ls /var/lib/libvirt/images/basis/

Expected:

Destroy complete! Resources: 4 destroyed.
0
noble.qcow2

The base image was never part of the state. OpenTofu only deletes what it created itself.

Save your progress? Sign up for free

Enterprise Note

  • State does not belong on the laptop: In a team it lives in a backend with locking (S3 with a lock file, GitLab’s HTTP backend). OpenTofu can encrypt the state (encryption block).
  • Provider maturity: The libvirt provider is a community project with one main author. Version 0.9 changed the schema completely. If you build on it, pin the version and test updates in a lab like this one.
  • Larger environments: For Proxmox there is the bpg/proxmox provider; for OpenStack and OpenNebula there are official providers. There, OpenTofu pays off more than for individual libvirt hosts.
  • Multiple hosts: A provider block with alias and uri = "qemu+ssh://ubuntu@kvm2/system" controls kvm2 from the same configuration.