Skip to content
Hands-on IT IT Drills

Day 4: Several Resources, One Graph

Goal: The API NotesApp creates Secret, Deployment and Service for notes-api. kro derives the order itself from the expressions. · Duration: approx. 45 min · Prerequisite: Day 3

Briefing

  • If one object refers to another via ${…}, an edge is created in the graph: ${secret.metadata.name} in the Deployment means “Secret first”.
  • From all edges, kro calculates the topological order. Objects are created in this order and deleted in the reverse order. The order in the file doesn’t matter.
  • A cycle (A needs B, B needs A) is forbidden. kro rejects the RGD.
  • One instance of NotesApp replaces the five commands from Day 0.
 NotesApp/notes
   ├─ secret      notes-api        (apiKey)
   ├─ deployment  notes            ◄─ secret.metadata.name
   └─ service     notes (30080)    ◄─ deployment.spec.selector.matchLabels

Drill 1: The RGD

mkdir -p ~/kro-lab/tag04
cd ~/kro-lab/tag04
cat > notesapp-rgd.yaml <<'EOF'
apiVersion: kro.run/v1alpha1
kind: ResourceGraphDefinition
metadata:
  name: notesapp
spec:
  schema:
    apiVersion: v1alpha1
    kind: NotesApp
    group: lab.example.com
    spec:
      image: string | default="ghcr.io/thomaszachmann/notes-api:0.5.1"
      replicas: integer | default=1 minimum=1 maximum=5
      apiKey: string | required=true minLength=12
      nodePort: integer | required=true minimum=30000 maximum=32767
  resources:
    - id: secret
      template:
        apiVersion: v1
        kind: Secret
        metadata:
          name: ${schema.metadata.name}-api
        stringData:
          apiKey: ${schema.spec.apiKey}
    - id: deployment
      template:
        apiVersion: apps/v1
        kind: Deployment
        metadata:
          name: ${schema.metadata.name}
        spec:
          replicas: ${schema.spec.replicas}
          selector:
            matchLabels:
              app: ${schema.metadata.name}
          template:
            metadata:
              labels:
                app: ${schema.metadata.name}
            spec:
              containers:
                - name: api
                  image: ${schema.spec.image}
                  ports:
                    - containerPort: 8080
                  env:
                    - name: NOTES_API_KEY
                      valueFrom:
                        secretKeyRef:
                          name: ${secret.metadata.name}
                          key: apiKey
    - id: service
      template:
        apiVersion: v1
        kind: Service
        metadata:
          name: ${schema.metadata.name}
        spec:
          type: NodePort
          selector: ${deployment.spec.selector.matchLabels}
          ports:
            - port: 80
              targetPort: 8080
              nodePort: ${schema.spec.nodePort}
EOF
  • The team sets only four fields, and two of them have a default.
  • ${secret.metadata.name} takes the name of the Secret from the object secret. The name is the same as ${schema.metadata.name}-api, but the reference makes the dependency visible.
  • selector: ${deployment.spec.selector.matchLabels} takes over a whole map. The type matches, because both fields are map[string]string.
kubectl apply -f notesapp-rgd.yaml
kubectl get rgd notesapp -o wide

Expected:

resourcegraphdefinition.kro.run/notesapp created
NAME       APIVERSION   KIND       STATE    READY   TOPOLOGICALORDER                    AGE
notesapp   v1alpha1     NotesApp   Active   True    ["secret","deployment","service"]   6s

kro calculated the order from the references.

Save your progress? Sign up for free

Drill 2: The Instance

cat > notes.yaml <<'EOF'
apiVersion: lab.example.com/v1alpha1
kind: NotesApp
metadata:
  name: notes
  namespace: app
spec:
  apiKey: lab-api-key-123
  nodePort: 30080
EOF
kubectl apply -f notes.yaml
kubectl -n app get notesapp -w

Expected (stop with Ctrl+C):

notesapp.lab.example.com/notes created
NAME    STATE   READY   AGE
notes                   1s
notes   ACTIVE   True    1s

Nine lines of YAML instead of five commands. What was created:

kubectl -n app get deploy,svc,secret -l kro.run/instance-name=notes

Expected:

NAME                    READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/notes   1/1     1            1           25s

NAME            TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
service/notes   NodePort   10.43.48.122   <none>        80:30080/TCP   25s

NAME               TYPE     DATA   AGE
secret/notes-api   Opaque   1      25s
curl -s localhost:30080/version
curl -s -H "X-API-Key: lab-api-key-123" -d '{"text":"erste Notiz"}' localhost:30080/notes
curl -s localhost:30080/notes

Expected (the ID will differ):

0.5.1
{"id":"2378d513-f86b-4e0e-a514-edc8c1809aae","text":"erste Notiz"}
[{"id":"2378d513-f86b-4e0e-a514-edc8c1809aae","text":"erste Notiz"}]

Save your progress? Sign up for free

Drill 3: ACTIVE Is Not the Same as Ready

Look at the times in the output of Drill 2: the instance was ACTIVE after one second, but at that point the Deployment didn’t have a finished pod yet. Without further information, kro considers an object ready as soon as it has been created.

Try it with an image that doesn’t exist:

kubectl -n app patch notesapp notes --type merge \
  -p '{"spec":{"image":"ghcr.io/thomaszachmann/notes-api:9.9.9"}}'
kubectl -n app get notesapp notes
kubectl -n app get pods

Expected:

notesapp.lab.example.com/notes patched
NAME    STATE    READY   AGE
notes   ACTIVE   True    3m
NAME                     READY   STATUS         RESTARTS   AGE
notes-6d8b7f5c9-2kq4x    0/1     ErrImagePull   0          8s
notes-86b5dd7468-lsqd9   1/1     Running        0          3m

The instance reports ACTIVE even though the new pod never starts. You’ll fix that on Day 5. For now, revert the image:

kubectl -n app patch notesapp notes --type json \
  -p '[{"op":"remove","path":"/spec/image"}]'

Without image, the default from the schema applies again.

Save your progress? Sign up for free

Drill 4: A Cycle

What happens if the Deployment wants to know the name of the Service? Create a second RGD. Instead of typing the whole file, you copy it and change three places in the editor:

cp notesapp-rgd.yaml zyklus.yaml
  • name: notesapp → name: notesapp-zyklus
  • kind: NotesApp → kind: NotesAppZyklus
  • in the container, insert two lines above - name: NOTES_API_KEY:
                  env:
                    - name: NOTES_SVC
                      value: ${service.metadata.name}
                    - name: NOTES_API_KEY
kubectl apply -f zyklus.yaml
kubectl get rgd notesapp-zyklus \
  -o jsonpath='{.status.state}: {.status.conditions[?(@.type=="GraphAccepted")].message}{"\n"}'

Expected:

resourcegraphdefinition.kro.run/notesapp-zyklus created
Inactive: failed to build dependency graph: graph contains a cycle: service ->
deployment -> service

The Service needs the Deployment’s selector, the Deployment needs the Service’s name. There is no possible order. If you really need the name, you write it as text (${schema.metadata.name}), which depends on no object.

kubectl delete -f zyklus.yaml

Save your progress? Sign up for free

Checkpoint

kubectl get rgd notesapp -o wide

Expected: Active, TOPOLOGICALORDER ["secret","deployment","service"].

kubectl -n app get notesapp

Expected:

NAME    STATE    READY   AGE
notes   ACTIVE   True    6m
curl -s localhost:30080/version

Expected: 0.5.1.

diff notesapp-rgd.yaml ~/kro-lab/code/loesung/tag04/notesapp-rgd.yaml

Expected: no output.

Challenge

Scale notes to three replicas, not on the Deployment but on the instance. What happens if you then set the Deployment to one replica by hand?

Note: Day 1, Drill 5.

Solution

kubectl -n app patch notesapp notes --type merge -p '{"spec":{"replicas":3}}'
kubectl -n app get deploy notes
kubectl -n app scale deploy notes --replicas=1
kubectl -n app get deploy notes

Expected:

notesapp.lab.example.com/notes patched
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
notes   3/3     3            3           7m
deployment.apps/notes scaled
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
notes   3/3     3            3           7m

kro immediately sets replicas back to the instance’s value. Back to one:

kubectl -n app patch notesapp notes --type merge -p '{"spec":{"replicas":1}}'

Save your progress? Sign up for free

Enterprise Note

  • Lean API: Give the team only the fields it should really decide. Everything else (probes, resources, security context) is set by the platform in the RGD.
  • Secrets in the instance: An API key in the spec of an instance is stored in plain text in etcd and shows up in every kubectl get -o yaml. That’s fine for the lab. In production, secrets come from a secrets manager (see “OpenBao secrets hands-on”) or are generated (Day 6).
  • Who will win? A HorizontalPodAutoscaler and kro would fight over replicas. Omit the field (feature gate CELOmitFunction, Day 12) or don’t offer the autoscaler in the first place.