Skip to content
Hands-on IT IT Drills

Day 4: count, for_each, and Data Sources

Goal: Create several similar resources with count and for_each, generate blocks with dynamic, read information from the outside with data sources, and so build the complete notes stack in Docker. · Duration: approx. 90 min · Prerequisite: Day 3

Briefing

  • count = 3 creates a resource three times: addresses x[0], x[1], x[2].
  • for_each takes a map or a set (set) and creates one resource per key: x["backend"]. Keys are stable, numbers are not – removing an item from the middle of a count list shifts all the following ones.
  • dynamic "block" generates nested blocks from a list or map.
  • A data source (data) reads something OpenTofu doesn’t manage – here the exact digest of an image in the registry.
 data docker_registry_image ─► digest ─► docker_image (pinned to the digest)
 docker_network for_each {backend, frontend}
   db (Postgres, health check) ◄─ migrate (one-off) ◄─ notes-api[a|b] ◄─ proxy :8080

Drill 1: Data Sources and for_each

mkdir -p ~/tf-lab/tag04 && cd ~/tf-lab/tag04
cat > versions.tf <<'EOF'
terraform {
  required_providers {
    docker = {
      source  = "kreuzwerker/docker"
      version = "~> 4.6"
    }
  }
}

provider "docker" {}
EOF
cat > netze.tf <<'EOF'
resource "docker_network" "notes" {
  for_each = toset(["backend", "frontend"])
  name     = "notes-${each.key}"
}
EOF
cat > images.tf <<'EOF'
data "docker_registry_image" "postgres" {
  name = "postgres:18.6-alpine"
}

resource "docker_image" "postgres" {
  name          = data.docker_registry_image.postgres.name
  pull_triggers = [data.docker_registry_image.postgres.sha256_digest]
  keep_locally  = true
}

resource "docker_image" "nginx" {
  name         = "nginx:1.31.8-alpine"
  keep_locally = true
}

# notes-api:0.4.0 was built locally on Day 0 – read only
data "docker_image" "notes_api" {
  name = "notes-api:0.4.0"
}
EOF
tofu init -no-color >/dev/null
tofu apply -auto-approve -no-color | grep -E 'Apply complete'
tofu state list
tofu console <<< 'substr(data.docker_registry_image.postgres.sha256_digest, 0, 15)'

Expected:

Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
data.docker_image.notes_api
data.docker_registry_image.postgres
docker_image.nginx
docker_image.postgres
docker_network.notes["backend"]
docker_network.notes["frontend"]
"sha256:10a5c69a"
  • for_each = toset([...]) creates one network per entry. each.key is the entry; the addresses carry it in square brackets.
  • data.docker_registry_image asks the registry for the digest. If the image behind the tag changes, the digest changes – and pull_triggers makes sure it is pulled again.
  • data "docker_image" manages nothing, it only reads. If the image is missing, the plan fails with a clear message.

Save your progress? Sign up for free

Drill 2: Database and Migration

cat > db.tf <<'EOF'
resource "docker_volume" "db" {
  name = "notes-db"
}

resource "docker_container" "db" {
  name  = "notes-db"
  image = docker_image.postgres.image_id
  env = [
    "POSTGRES_USER=notes",
    "POSTGRES_PASSWORD=lab-db-pw-123",
    "POSTGRES_DB=notes",
  ]
  networks_advanced {
    name = docker_network.notes["backend"].id
  }
  volumes {
    volume_name    = docker_volume.db.name
    container_path = "/var/lib/postgresql"
  }
  healthcheck {
    test     = ["CMD-SHELL", "pg_isready -U notes -d notes"]
    interval = "2s"
    retries  = 15
  }
  wait = true
}

resource "docker_container" "migrate" {
  name     = "notes-migrate"
  image    = data.docker_image.notes_api.id
  command  = ["migrate"]
  env      = ["NOTES_DB_URL=postgres://notes:lab-db-pw-123@notes-db:5432/notes"]
  must_run = false
  attach   = true
  logs     = true
  networks_advanced {
    name = docker_network.notes["backend"].id
  }
  depends_on = [docker_container.db]
}
EOF
tofu apply -auto-approve -no-color | grep -E 'Creation complete|Apply complete'
docker logs notes-migrate 2>&1 | grep -o 'msg="migration complete".*'

Expected (IDs and times differ):

docker_volume.db: Creation complete after 0s [id=notes-db]
docker_container.db: Creation complete after 5s [id=2c7e…]
docker_container.migrate: Creation complete after 1s [id=9a41…]
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
msg="migration complete" table=notes
  • wait = true waits until the health check reports “healthy”. Only then is the resource done – and only then does the migration start.
  • depends_on: the migration refers to the database only through a hostname inside a string. OpenTofu can’t detect a dependency there, so it is stated explicitly.
  • attach = true waits until the container has finished; must_run = false allows it to be stopped afterwards.

Save your progress? Sign up for free

Drill 3: Multiple notes-api Instances and the Proxy

cat > api.tf <<'EOF'
variable "instanzen" {
  description = "Namen der notes-api-Instanzen"
  type        = set(string)
  default     = ["a", "b"]
}

resource "docker_container" "api" {
  for_each = var.instanzen
  name     = "notes-api-${each.key}"
  image    = data.docker_image.notes_api.id
  env = [
    "NOTES_API_KEY=lab-api-key-123",
    "NOTES_DB_URL=postgres://notes:lab-db-pw-123@notes-db:5432/notes",
  ]
  dynamic "networks_advanced" {
    for_each = docker_network.notes
    content {
      name = networks_advanced.value.id
    }
  }
  depends_on = [docker_container.migrate]
}
EOF
cat > nginx.conf.tftpl <<'EOF'
upstream notes_api {
%{ for name in instanzen ~}
  server ${name}:8080;
%{ endfor ~}
}
server {
  listen 80;
  location / {
    proxy_pass http://notes_api;
  }
}
EOF
cat > proxy.tf <<'EOF'
resource "docker_container" "proxy" {
  name  = "notes-proxy"
  image = docker_image.nginx.image_id
  ports {
    internal = 80
    external = 8080
    ip       = "127.0.0.1"
  }
  networks_advanced {
    name = docker_network.notes["frontend"].id
  }
  upload {
    file = "/etc/nginx/conf.d/default.conf"
    content = templatefile("${path.module}/nginx.conf.tftpl", {
      instanzen = [for c in docker_container.api : c.name]
    })
  }
}

output "url" {
  value = "http://127.0.0.1:8080"
}
EOF
tofu apply -auto-approve -no-color | grep 'Apply complete'
U=$(tofu output -raw url)
curl -s -X POST -H "X-API-Key: $NOTES_API_KEY" -H 'Content-Type: application/json' \
  -d '{"text":"aus OpenTofu"}' "$U/notes" | jq -r .text
curl -s "$U/notes" | jq length
docker ps --filter name=notes- --format '{{.Names}}' | sort

Expected:

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
aus OpenTofu
1
notes-api-a
notes-api-b
notes-db
notes-db-tag03
notes-proxy
notes-web-dev

notes-db-tag03 and notes-web-dev come from Days 2 and 3 – they keep running in their own states.

  • dynamic "networks_advanced" attaches each API instance to both networks: to backend for the database, to frontend for the proxy. The proxy doesn’t see the database.
  • [for c in docker_container.api : c.name] turns all instances into a list of names for the nginx configuration – if an instance is added, the configuration changes by itself.

Save your progress? Sign up for free

Drill 4: Scaling – and the count Trap

tofu plan -no-color -var 'instanzen=["a","b","c"]' | grep -E '^  # |Plan:'
tofu plan -no-color -var 'instanzen=["a","c"]' | grep -E '^  # |Plan:'

Expected:

  # docker_container.api["c"] will be created
  # docker_container.proxy must be replaced
Plan: 2 to add, 0 to change, 1 to destroy.
  # docker_container.api["b"] will be destroyed
  # (because key ["b"] is not in for_each map)
  # docker_container.api["c"] will be created
  # docker_container.proxy must be replaced
Plan: 2 to add, 0 to change, 2 to destroy.

With for_each, a change affects exactly the instance with that key. With count it would be different – had the list ["a","b","c"] served as index 0, 1, 2, removing b would turn “c at position 2” into “c at position 1”: OpenTofu would replace [1] and delete [2]. That’s why you use count only for truly identical copies or as a switch: count = var.mit_proxy ? 1 : 0.

Save your progress? Sign up for free

Drill 5: Cleaning Up with a Safeguard

The database data should survive a destroy – a protection right in the code:

printf '  lifecycle {\n    prevent_destroy = true\n  }\n' > schutz.txt
sed -i '/^resource "docker_volume" "db" {$/r schutz.txt' db.tf
tofu destroy -auto-approve -no-color 2>&1 | grep -E '^Error|prevent_destroy set'
docker ps --filter name=notes- --format '{{.Names}}' | wc -l

Expected:

Error: Resource instance cannot be destroyed
Resource instance docker_volume.db has prevent_destroy set, but the plan
6

OpenTofu refuses the whole run – nothing was deleted, not even the containers. The stack stays up; Day 5 turns it into a module.

Save your progress? Sign up for free

Checkpoint

curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8080/healthz

Expected:

200
tofu state list | grep -c 'docker_container.api\['

Expected:

2
tofu plan -detailed-exitcode -no-color >/dev/null; echo "Exit: $?"

Expected:

Exit: 0

Challenge

The API instances should get additional environment variables from a map, e.g. { LOG_LEVEL = "debug" }. Without the map, nothing should change.

Hint: concat(liste1, liste2) and [for k, v in map : "${k}=${v}"].

Solution

cat > api.tf <<'EOF'
variable "instanzen" {
  description = "Namen der notes-api-Instanzen"
  type        = set(string)
  default     = ["a", "b"]
}

variable "extra_env" {
  type    = map(string)
  default = {}
}

resource "docker_container" "api" {
  for_each = var.instanzen
  name     = "notes-api-${each.key}"
  image    = data.docker_image.notes_api.id
  env = concat([for k, v in var.extra_env : "${k}=${v}"], [
    "NOTES_API_KEY=lab-api-key-123",
    "NOTES_DB_URL=postgres://notes:lab-db-pw-123@notes-db:5432/notes",
  ])
  dynamic "networks_advanced" {
    for_each = docker_network.notes
    content {
      name = networks_advanced.value.id
    }
  }
  depends_on = [docker_container.migrate]
}
EOF
tofu validate -no-color
tofu plan -no-color -detailed-exitcode >/dev/null; echo "ohne Map: Exit $?"
tofu plan -no-color -var 'extra_env={LOG_LEVEL="debug"}' | grep -E 'LOG_LEVEL|Plan:'

Expected:

Success! The configuration is valid.
ohne Map: Exit 0
          + "LOG_LEVEL=debug",
          + "LOG_LEVEL=debug",
Plan: 2 to add, 0 to change, 2 to destroy.

Save your progress? Sign up for free

Enterprise Note

  • Images by digest: A tag like 18.6-alpine can change, a digest can’t. If you want reproducible rollouts, pin digests – and let Renovate update them.
  • for_each instead of count is the rule. count remains for “0 or 1” and for truly equivalent copies.
  • prevent_destroy protects against accidents, not against intent: whoever removes the block can delete. Production databases additionally get backups outside of OpenTofu.