Skip to content
Hands-on IT IT Drills

Day 10: RGD Chaining: The Fleet

Goal: A second API NotesFleet builds on NotesApp: a team says “gelb, environments dev and prod”, and kro creates a namespace and a complete NotesApp per environment. · Duration: approx. 45 min · Prerequisite: Day 9

Briefing

  • Every RGD generates a CRD. This CRD can use another RGD as an object, like a Deployment or a Service. kro calls this RGD chaining.
  • That’s how building blocks emerge: small RGDs with a clear spec (input) and status (output), from which larger ones are composed.
  • scope: Cluster makes the instances cluster-scoped. Every namespaced object in it then needs an explicit namespace.
  • Troubleshooting goes from the outside in: first the fleet, then the NotesApp, then its Deployment.
 NotesFleet/gelb  (cluster-scoped)
   ├─ namespaces[0]  Namespace gelb-dev
   ├─ namespaces[1]  Namespace gelb-prod
   ├─ apps[0]        NotesApp gelb-dev/notes   (31000) ─► 9 objects
   └─ apps[1]        NotesApp gelb-prod/notes  (31001) ─► 9 objects

Drill 1: The Fleet’s RGD

mkdir -p ~/kro-lab/tag10
cd ~/kro-lab/tag10
cat > fleet-rgd.yaml <<'EOF'
apiVersion: kro.run/v1alpha1
kind: ResourceGraphDefinition
metadata:
  name: notesfleet
spec:
  schema:
    apiVersion: v1alpha1
    kind: NotesFleet
    group: lab.example.com
    scope: Cluster
    spec:
      team: string | required=true pattern="^[a-z][a-z0-9-]*$"
      environments: "[]string | required=true minItems=1 maxItems=3"
      basePort: integer | default=31000 minimum=30000 maximum=32700
    status:
      apps: ${apps.map(a, a.metadata.namespace + "=" + a.status.?state.orValue("?")).join(", ")}
  resources:
    - id: namespaces
      forEach:
        - env: ${schema.spec.environments}
      template:
        apiVersion: v1
        kind: Namespace
        metadata:
          name: ${schema.spec.team + "-" + env}
    - id: apps
      forEach:
        - idx: ${lists.range(size(schema.spec.environments))}
      template:
        apiVersion: lab.example.com/v1alpha1
        kind: NotesApp
        metadata:
          name: notes
          namespace: ${namespaces[idx].metadata.name}
        spec:
          apiKey: ${random.seededString(20, schema.metadata.name + "/" + schema.spec.environments[idx])}
          nodePort: ${schema.spec.basePort + idx}
      readyWhen:
        - ${each.status.?state.orValue("") == "ACTIVE"}
EOF
  • lists.range(n) returns [0, 1, …, n-1]. With the index idx you access the environment and the matching namespace of the first collection.
  • ${namespaces[idx].metadata.name} makes apps depend on namespaces. kro creates all namespaces first.
  • nodePort: ${schema.spec.basePort + idx}: dev gets 31000, prod 31001.
  • readyWhen with each applies to each element individually. The collection is ready when all NotesApps are ACTIVE.
  • maxItems=3 protects the cluster: each environment brings its own database.
kubectl apply -f fleet-rgd.yaml
kubectl get rgd notesfleet -o wide

Expected:

resourcegraphdefinition.kro.run/notesfleet created
NAME         APIVERSION   KIND         STATE    READY   TOPOLOGICALORDER        AGE
notesfleet   v1alpha1     NotesFleet   Active   True    ["namespaces","apps"]   8s

During validation, kro read the schema of the CRD notesapps.lab.example.com, just like that of a Deployment. A typo in apiKey or a text in nodePort would already have been caught here.

Save your progress? Sign up for free

Drill 2: A Fleet

cat > team-gelb.yaml <<'EOF'
apiVersion: lab.example.com/v1alpha1
kind: NotesFleet
metadata:
  name: gelb
spec:
  team: gelb
  environments:
    - dev
    - prod
EOF
kubectl apply -f team-gelb.yaml
kubectl get notesfleet -w

Expected (stop with Ctrl+C):

notesfleet.lab.example.com/gelb created
NAME   STATE         READY   AGE
gelb   IN_PROGRESS   False   10s
gelb   IN_PROGRESS   False   21s
gelb   ACTIVE        True    31s

No namespace in the metadata: the fleet is cluster-scoped. After half a minute, two environments are up, each with its own database:

kubectl get notesfleet gelb -o jsonpath='{.status.apps}{"\n"}'
kubectl get notesapp -A
curl -s localhost:31000/version
curl -s localhost:31001/version

Expected:

gelb-dev=ACTIVE, gelb-prod=ACTIVE
NAMESPACE   NAME    REPLICAS   AVAILABLE   PORT    STATE
app         notes   1          1           30080   ACTIVE
gelb-dev    notes   1          1           31000   ACTIVE
gelb-prod   notes   1          1           31001   ACTIVE
0.5.1
0.5.1

One instance with six lines of spec created 2 namespaces, 2 NotesApps and, beneath them, 18 objects. Each NotesApp follows everything you built on Days 4 to 9: readiness, database, checks, the image from the platform ConfigMap.

Save your progress? Sign up for free

Drill 3: From the Outside In

The labels show who created whom:

kubectl get namespace -l kro.run/instance-name=gelb
kubectl -n gelb-dev get notesapp notes \
  -o jsonpath='{.metadata.labels.kro\.run/instance-kind}{"\n"}'
kubectl -n gelb-dev get deploy -l kro.run/instance-kind=NotesApp

Expected:

NAME        STATUS   AGE
gelb-dev    Active   2m
gelb-prod   Active   2m
NotesFleet
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
notes   1/1     1            1           2m
  • The namespaces belong to the instance gelb.
  • The NotesApp in gelb-dev carries instance-kind=NotesFleet: it was created by the fleet.
  • The Deployment in it carries instance-kind=NotesApp: it was created by the NotesApp.

If a fleet is stuck, you proceed in this order: kubectl get notesfleet, then the conditions of the NotesApps, then their objects.

Save your progress? Sign up for free

Drill 4: Dissolving the Fleet

kubectl delete -f team-gelb.yaml
kubectl get namespace -l kro.run/instance-name=gelb

Expected (deletion takes just under a minute):

notesfleet.lab.example.com "gelb" deleted
No resources found

kubectl delete waits until the instance’s finalizer is gone, and kro removes it only once everything has been cleaned up. kro deletes in reverse order: first the NotesApps, which remove their nine objects, then the namespaces. The PVCs with Orphaned don’t survive this, because a namespace takes everything in it along when it is deleted. The deletion policy protects against kro, not against a deleted namespace.

Save your progress? Sign up for free

Checkpoint

kubectl get rgd notesfleet

Expected: STATE Active.

kubectl apply -f team-gelb.yaml
kubectl wait notesfleet/gelb --for=condition=Ready --timeout=180s

Expected:

notesfleet.lab.example.com/gelb created
notesfleet.lab.example.com/gelb condition met
diff fleet-rgd.yaml ~/kro-lab/code/loesung/tag10/fleet-rgd.yaml

Expected: no output.

Challenge

On gelb, replace the environment prod with test. What happens to gelb-prod, and which port does gelb-test get?

Note: The index determines the port.

Solution

kubectl patch notesfleet gelb --type merge -p '{"spec":{"environments":["dev","test"]}}'
kubectl wait notesfleet/gelb --for=condition=Ready --timeout=180s
kubectl get notesfleet gelb -o jsonpath='{.status.apps}{"\n"}'
kubectl get notesapp -A

Expected:

notesfleet.lab.example.com/gelb patched
notesfleet.lab.example.com/gelb condition met
gelb-dev=ACTIVE, gelb-test=ACTIVE
NAMESPACE   NAME    REPLICAS   AVAILABLE   PORT    STATE
app         notes   1          1           30080   ACTIVE
gelb-dev    notes   1          1           31000   ACTIVE
gelb-test   notes   1          1           31001   ACTIVE

gelb-prod is deleted along with its database; gelb-test takes over index 1 and thus port 31001. If a port depends on the index, every change in the middle of the list shifts the ports of all following environments. For real platforms, the port therefore belongs in the list itself (environments: [{name: dev, port: 31000}]).

Save your progress? Sign up for free

Enterprise Note

  • Building blocks instead of a monolith: An RGD that does everything quickly gets confusing. Small RGDs (database, app, environment) with clear inputs and outputs can be tested and combined individually.
  • Limit the depth: Each level is its own controller run with its own readiness. Two to three levels are easy to keep track of; with more, troubleshooting becomes tedious.
  • Creating namespaces is power: Whoever may create a NotesFleet creates namespaces. In a real platform, quotas, roles and NetworkPolicies belong with them (see graph decorator on Day 14).