Day 8: Collections with forEach
Goal: The team specifies a list of paths, and kro creates one CronJob per path that checks every minute whether notes-api responds. Each check gets its own condition. · Duration: approx. 50 min · Prerequisite: Day 7
Briefing
forEachturns one object into a collection: one object per element of a list. The list comes from an expression, usually fromschema.spec.- The loop variable (here
check) is available in the template. It belongs in the name; otherwise all objects get the same name. - Several iterators result in the Cartesian product: 2 regions × 3 tiers = 6 objects. If a list is empty, zero objects are created.
- Other objects see the collection as a list:
size(checkJobs),checkJobs.map(j, …). - If the list gets shorter, kro removes the surplus objects.
spec.checks: [health, version]
│ forEach
├─► CronJob notes-check-health (curl http://notes/healthz)
└─► CronJob notes-check-version (curl http://notes/version)
Drill 1: A Type and a List
mkdir -p ~/kro-lab/tag08
cd ~/kro-lab/tag08
cp ../tag07/notesapp-rgd.yaml .Add a custom type above spec: and the list below
networkPolicy:
group: lab.example.com
types:
Check:
name: string | required=true pattern="^[a-z0-9-]+$"
path: string | required=true pattern="^/"
spec:
...
networkPolicy: boolean | default=false
checks: "[]Check"
name ends up in the name of a CronJob. The pattern
therefore enforces lowercase letters, digits and hyphens, as Kubernetes
requires in names.
Save your progress? Sign up for free
Drill 2: The Collection
Append to the end of the file:
- id: checkJobs
forEach:
- check: ${schema.spec.?checks.orValue([])}
template:
apiVersion: batch/v1
kind: CronJob
metadata:
name: ${schema.metadata.name + '-check-' + check.name}
labels:
check: ${check.name}
spec:
schedule: "* * * * *"
successfulJobsHistoryLimit: 1
failedJobsHistoryLimit: 1
jobTemplate:
spec:
backoffLimit: 0
template:
spec:
restartPolicy: Never
containers:
- name: curl
image: curlimages/curl:8.23.0
args:
- -fsS
- http://${service.metadata.name}${check.path}
forEachis a list of entries with exactly one key: the name of the loop variable.?checks.orValue([]): Without checks the list is empty, and no CronJob is created.-fmakescurlexit with an error code on an HTTP error. Then the job fails.${service.metadata.name}makes the checks depend on the Service. So they only start once notes-api exists.
Save your progress? Sign up for free
Drill 3: Status from the Collection
Add a counter field below storage::
storage: ${dbData.spec.resources.requests.storage}
checks: ${size(checkJobs)}
And append a third entry to the conditions list:
- |-
${checkJobs.map(j, runtime.newCondition({
type: 'Check-' + j.metadata.labels.check,
status: j.status.?lastSuccessfulTime.hasValue() ? 'True' : 'Unknown',
reason: 'CronJob',
message: j.spec.jobTemplate.spec.template.spec.containers[0].args[1]
}))}
One entry that returns a list of conditions: the
number of conditions grows with the number of checks. A CronJob has
lastSuccessfulTime as soon as a run has succeeded.
kubectl apply -f notesapp-rgd.yaml
kubectl get rgd notesapp \
-o jsonpath='{.status.state}: {.status.conditions[?(@.type=="GraphAccepted")].message}{"\n"}'Expected:
resourcegraphdefinition.kro.run/notesapp configured
Active: resource graph and schema are valid
If notes then jumps to ERROR with
schema not found for field checks, you know the remedy from
Day 6:
kubectl -n kro-system rollout restart deploy/kro.
Save your progress? Sign up for free
Drill 4: Two Checks
cat > notes.yaml <<'EOF'
apiVersion: lab.example.com/v1alpha1
kind: NotesApp
metadata:
name: notes
namespace: app
spec:
apiKey: lab-api-key-123
nodePort: 30080
networkPolicy: true
checks:
- name: health
path: /healthz
- name: version
path: /version
EOF
kubectl apply -f notes.yaml
kubectl -n app get cronjobExpected:
notesapp.lab.example.com/notes configured
NAME SCHEDULE TIMEZONE SUSPEND ACTIVE LAST SCHEDULE AGE
notes-check-health * * * * * <none> False 0 <none> 16s
notes-check-version * * * * * <none> False 0 <none> 16s
From now on, notes.yaml is the file that describes the
instance. After a minute at most, the first jobs run:
kubectl -n app get jobs
kubectl -n app get notesapp notes \
-o jsonpath='{.status.checks}{"\n"}{range .status.conditions[*]}{.type}={.status} {.message}{"\n"}{end}'Expected:
NAME STATUS COMPLETIONS DURATION AGE
notes-check-health-29906997 Complete 1/1 8s 58s
notes-check-version-29906997 Complete 1/1 8s 58s
2
Ready=True all resources are created and ready
ApiAvailable=True 1 von 1 Pods bereit
Check-health=True http://notes/healthz
Check-version=True http://notes/version
Save your progress? Sign up for free
Drill 5: A Broken Check
Swap version for a path that doesn’t exist. Change the
second check in notes.yaml:
- name: kaputt
path: /gibtsnicht
kubectl apply -f notes.yaml
kubectl -n app get cronjobExpected:
notesapp.lab.example.com/notes configured
NAME SCHEDULE TIMEZONE SUSPEND ACTIVE LAST SCHEDULE AGE
notes-check-health * * * * * <none> False 0 24s 3m3s
notes-check-kaputt * * * * * <none> False 0 <none> 5s
notes-check-version is gone: the list got shorter, and
kro cleaned up. After a minute:
kubectl -n app get jobs
kubectl -n app logs -l job-name --prefix --tail=1Expected:
NAME STATUS COMPLETIONS DURATION AGE
notes-check-health-29906999 Complete 1/1 4s 24s
notes-check-kaputt-29906999 Failed 0/1 24s 24s
[pod/notes-check-health-29906999-6mq67/curl] ok
[pod/notes-check-kaputt-29906999-r49kn/curl] curl: (22) The requested URL returned
error: 404
kubectl -n app get notesapp notes \
-o jsonpath='{range .status.conditions[*]}{.type}={.status} {.message}{"\n"}{end}'Expected:
Ready=True all resources are created and ready
ApiAvailable=True 1 von 1 Pods bereit
Check-health=True http://notes/healthz
Check-kaputt=Unknown http://notes/gibtsnicht
Check-kaputt stays Unknown because there
was never a successful run. Ready stays True,
because for kro the CronJob is created and thus done. A failing check
doesn’t hold up the app; it only draws attention to itself.
Set notes.yaml back to version and
/version and apply.
Save your progress? Sign up for free
Checkpoint
kubectl -n app get cronjobExpected: notes-check-health and
notes-check-version.
kubectl -n app get notesapp notes -o jsonpath='{.status.checks}{"\n"}'Expected: 2.
diff notesapp-rgd.yaml ~/kro-lab/code/loesung/tag08/notesapp-rgd.yamlExpected: no output.
Challenge
The check should create two CronJobs for each check:
one against the Service notes and one against the Service
notes-db (port 5432 isn’t HTTP, so the run fails, but
that’s not the point here). What does forEach look like,
and what are the objects called?
Note: Two iterators result in the Cartesian product. Both variables belong in the name.
Solution
forEach:
- check: ${schema.spec.?checks.orValue([])}
- target: ${["notes", "notes-db"]}
template:
...
metadata:
name: ${schema.metadata.name + '-' + target + '-' + check.name}
With two checks, four CronJobs are created:
notes-notes-health, notes-notes-version,
notes-notes-db-health and
notes-notes-db-version. The first list is the outer loop.
Don’t apply this; the NotesApp keeps its two checks.
Save your progress? Sign up for free
Enterprise Note
- Limits: kro limits a collection to 1000 objects and
to 10 iterators (
config.rgd.maxCollectionSizeandmaxCollectionDimensionSizein the chart). Also limit lists in the schema withmaxItems, so nobody accidentally creates a hundred CronJobs. - Sort maps: If you iterate over the keys of a map,
sort them (
.map(k, k).sort()). Maps have no fixed order, and a changing order causes unnecessary changes. - Synthetic checks: For real monitoring there is the Prometheus Blackbox Exporter. The CronJobs here show the pattern: a list in the spec, one object per entry, one condition per object.