Skip to content
Hands-on IT IT Drills

Day 9: External References

Goal: The platform centrally defines which image all NotesApps use and publishes announcements. kro reads both from foreign ConfigMaps and reacts immediately to changes. · Duration: approx. 45 min · Prerequisite: Day 8

Briefing

  • externalRef instead of template: kro reads an existing object, but never creates it, never changes it and never deletes it.
  • Two forms: with name, exactly one object; with selector, a list of all matching objects.
  • External objects are nodes in the graph like all others. If the object is missing, kro waits.
  • kro watches external objects. If one changes, kro immediately reconciles all affected instances.
  • For ConfigMaps and Secrets, kro doesn’t know the keys under data. You read them with ? and orValue.
 Namespace platform                     Namespace app
 ConfigMap notes-platform ─(image)───►  NotesApp/notes ─► Deployment
 ConfigMaps with label    ─(list)────►  status.announcements
   notes.lab/announcement=true

Drill 1: Two External Objects

mkdir -p ~/kro-lab/tag09
cd ~/kro-lab/tag09
cp ../tag08/notesapp-rgd.yaml .

Insert at the very top below resources: (before - id: secret):

  resources:
    - id: platform
      externalRef:
        apiVersion: v1
        kind: ConfigMap
        metadata:
          name: notes-platform
          namespace: platform
    - id: announcements
      externalRef:
        apiVersion: v1
        kind: ConfigMap
        metadata:
          namespace: platform
          selector:
            matchLabels:
              notes.lab/announcement: "true"
    - id: secret

In the Deployment, replace both lines image: ${schema.spec.image} (init container and container) with:

                  image: ${platform.data.?image.orValue(schema.spec.image)}

And add below checks: in the status:

      checks: ${size(checkJobs)}
      announcements: ${announcements.map(a, a.metadata.name).join(",")}
  • If the platform ConfigMap contains an image, it wins. Otherwise the instance’s field applies. This lets the platform enforce a version for everyone.
  • announcements is a list, so map and join work.
kubectl apply -f notesapp-rgd.yaml
kubectl get rgd notesapp -o jsonpath='{.status.topologicalOrder}{"\n"}'
kubectl -n app get notesapp

Expected:

resourcegraphdefinition.kro.run/notesapp configured
["platform","announcements","secret","dbSecret","dbData","database","dbService",
"deployment","service","dbPolicy","budget","checkJobs"]
NAME    REPLICAS   AVAILABLE   PORT    STATE
notes   1          1           30080   ERROR

A new status field, so the quirk from Day 6 again:

kubectl -n kro-system rollout restart deploy/kro
kubectl -n kro-system rollout status deploy/kro

Save your progress? Sign up for free

Drill 2: kro Waits

kubectl -n app get notesapp
kubectl -n app get notesapp notes \
  -o jsonpath='{.status.conditions[?(@.type=="Ready")].message}{"\n"}'

Expected:

NAME    REPLICAS   AVAILABLE   PORT   STATE
notes   1                             IN_PROGRESS
waiting for unresolved resource: apply "platform" (ref): external ref ConfigMap
"platform/notes-platform" not found: executor: node not ready

The ConfigMap doesn’t exist yet, so kro halts the entire graph. The running pods stay untouched; kro just doesn’t change anything anymore. AVAILABLE and PORT are empty because kro only writes the status fields again once the graph is complete.

Save your progress? Sign up for free

Drill 3: The Platform Delivers

The platform ConfigMap belongs to the platform team, not to kro:

kubectl create namespace platform
cat > platform.yaml <<'EOF'
apiVersion: v1
kind: ConfigMap
metadata:
  name: notes-platform
  namespace: platform
data:
  image: ghcr.io/thomaszachmann/notes-api:0.5.1
EOF
kubectl apply -f platform.yaml
kubectl -n app get notesapp
kubectl -n app get notesapp notes -o jsonpath='{.status.image} [{.status.announcements}]{"\n"}'

Expected:

namespace/platform created
configmap/notes-platform created
NAME    REPLICAS   AVAILABLE   PORT    STATE
notes   1          1           30080   ACTIVE
ghcr.io/thomaszachmann/notes-api:0.5.1 []

An announcement:

cat > wartung.yaml <<'EOF'
apiVersion: v1
kind: ConfigMap
metadata:
  name: wartung
  namespace: platform
  labels:
    notes.lab/announcement: "true"
data:
  text: Samstag 10 Uhr Wartung
EOF
kubectl apply -f wartung.yaml
kubectl -n app get notesapp notes -o jsonpath='{.status.image} [{.status.announcements}]{"\n"}'

Expected:

configmap/wartung created
ghcr.io/thomaszachmann/notes-api:0.5.1 [wartung]

A new ConfigMap with the right label, and the collection is one element longer. kro noticed that without a restart.

Save your progress? Sign up for free

Drill 4: A New Version for Everyone

The platform team rolls out a new version of notes-api:

kubectl -n platform patch configmap notes-platform --type merge \
  -p '{"data":{"image":"ghcr.io/thomaszachmann/notes-api:0.6.0-rc.1"}}'
kubectl -n app rollout status deploy/notes
curl -s localhost:30080/version

Expected:

configmap/notes-platform patched
Waiting for deployment "notes" rollout to finish: 1 old replicas are pending termination...
deployment "notes" successfully rolled out
0.6.0-rc.1

No change to the instance, no change to the RGD. The platform changed one line in its ConfigMap, and every NotesApp in the cluster rolls out. The notes are still in Postgres:

curl -s localhost:30080/notes

Expected:

[{"id":"5953a797-216a-45be-a2f9-fe1e5782a1f4","text":"bleibt erhalten"}]

Back to the tested version:

kubectl apply -f platform.yaml
kubectl -n app rollout status deploy/notes

Save your progress? Sign up for free

Checkpoint

kubectl -n app get notesapp notes

Expected: STATE ACTIVE.

curl -s localhost:30080/version

Expected: 0.5.1.

kubectl -n platform get configmap notes-platform --show-labels

Expected:

NAME             DATA   AGE   LABELS
notes-platform   1      10m   <none>

No kro.run labels: kro only read the object; it didn’t adopt it.

diff notesapp-rgd.yaml ~/kro-lab/code/loesung/tag09/notesapp-rgd.yaml

Expected: no output.

Challenge

Delete the announcement wartung. What is in status.announcements afterwards, and what happens to the notes-api Deployment?

Note: An empty list is a list too.

Solution

kubectl -n platform delete configmap wartung
kubectl -n app get notesapp notes -o jsonpath='[{.status.announcements}]{"\n"}'
kubectl -n app get notesapp notes

Expected:

configmap "wartung" deleted from platform namespace
[]
NAME    REPLICAS   AVAILABLE   PORT    STATE
notes   1          1           30080   ACTIVE

An empty collection is not an error. Unlike with notes-platform, kro doesn’t wait, because a selector that finds nothing simply results in an empty list. The Deployment stays as it is.

Save your progress? Sign up for free

Enterprise Note

  • Permissions: kro reads external objects with its own permissions. In aggregation mode (Day 12), kro must have get, list and watch on these kinds.
  • Central defaults: Image versions, storage classes, domains or resource limits belong in a platform ConfigMap. The teams don’t see them and don’t have to worry about them.
  • Beware of the big bang: One line in the platform ConfigMap rolls out all instances at the same time. For real rollouts, this should happen in stages (first dev, then prod), for example with one ConfigMap per environment or with GitOps (Day 14).