Day 4: Several Resources, One Graph
Goal: The API
NotesAppcreates Secret, Deployment and Service for notes-api. kro derives the order itself from the expressions. · Duration: approx. 45 min · Prerequisite: Day 3
Briefing
- If one object refers to another via
${…}, an edge is created in the graph:${secret.metadata.name}in the Deployment means “Secret first”. - From all edges, kro calculates the topological order. Objects are created in this order and deleted in the reverse order. The order in the file doesn’t matter.
- A cycle (A needs B, B needs A) is forbidden. kro rejects the RGD.
- One instance of
NotesAppreplaces the five commands from Day 0.
NotesApp/notes
├─ secret notes-api (apiKey)
├─ deployment notes ◄─ secret.metadata.name
└─ service notes (30080) ◄─ deployment.spec.selector.matchLabels
Drill 1: The RGD
mkdir -p ~/kro-lab/tag04
cd ~/kro-lab/tag04
cat > notesapp-rgd.yaml <<'EOF'
apiVersion: kro.run/v1alpha1
kind: ResourceGraphDefinition
metadata:
name: notesapp
spec:
schema:
apiVersion: v1alpha1
kind: NotesApp
group: lab.example.com
spec:
image: string | default="ghcr.io/thomaszachmann/notes-api:0.5.1"
replicas: integer | default=1 minimum=1 maximum=5
apiKey: string | required=true minLength=12
nodePort: integer | required=true minimum=30000 maximum=32767
resources:
- id: secret
template:
apiVersion: v1
kind: Secret
metadata:
name: ${schema.metadata.name}-api
stringData:
apiKey: ${schema.spec.apiKey}
- id: deployment
template:
apiVersion: apps/v1
kind: Deployment
metadata:
name: ${schema.metadata.name}
spec:
replicas: ${schema.spec.replicas}
selector:
matchLabels:
app: ${schema.metadata.name}
template:
metadata:
labels:
app: ${schema.metadata.name}
spec:
containers:
- name: api
image: ${schema.spec.image}
ports:
- containerPort: 8080
env:
- name: NOTES_API_KEY
valueFrom:
secretKeyRef:
name: ${secret.metadata.name}
key: apiKey
- id: service
template:
apiVersion: v1
kind: Service
metadata:
name: ${schema.metadata.name}
spec:
type: NodePort
selector: ${deployment.spec.selector.matchLabels}
ports:
- port: 80
targetPort: 8080
nodePort: ${schema.spec.nodePort}
EOF- The team sets only four fields, and two of them have a default.
${secret.metadata.name}takes the name of the Secret from the objectsecret. The name is the same as${schema.metadata.name}-api, but the reference makes the dependency visible.selector: ${deployment.spec.selector.matchLabels}takes over a whole map. The type matches, because both fields aremap[string]string.
kubectl apply -f notesapp-rgd.yaml
kubectl get rgd notesapp -o wideExpected:
resourcegraphdefinition.kro.run/notesapp created
NAME APIVERSION KIND STATE READY TOPOLOGICALORDER AGE
notesapp v1alpha1 NotesApp Active True ["secret","deployment","service"] 6s
kro calculated the order from the references.
Save your progress? Sign up for free
Drill 2: The Instance
cat > notes.yaml <<'EOF'
apiVersion: lab.example.com/v1alpha1
kind: NotesApp
metadata:
name: notes
namespace: app
spec:
apiKey: lab-api-key-123
nodePort: 30080
EOF
kubectl apply -f notes.yaml
kubectl -n app get notesapp -wExpected (stop with Ctrl+C):
notesapp.lab.example.com/notes created
NAME STATE READY AGE
notes 1s
notes ACTIVE True 1s
Nine lines of YAML instead of five commands. What was created:
kubectl -n app get deploy,svc,secret -l kro.run/instance-name=notesExpected:
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/notes 1/1 1 1 25s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/notes NodePort 10.43.48.122 <none> 80:30080/TCP 25s
NAME TYPE DATA AGE
secret/notes-api Opaque 1 25s
curl -s localhost:30080/version
curl -s -H "X-API-Key: lab-api-key-123" -d '{"text":"erste Notiz"}' localhost:30080/notes
curl -s localhost:30080/notesExpected (the ID will differ):
0.5.1
{"id":"2378d513-f86b-4e0e-a514-edc8c1809aae","text":"erste Notiz"}
[{"id":"2378d513-f86b-4e0e-a514-edc8c1809aae","text":"erste Notiz"}]
Save your progress? Sign up for free
Drill 3: ACTIVE Is Not the Same as Ready
Look at the times in the output of Drill 2: the instance was
ACTIVE after one second, but at that point the Deployment
didn’t have a finished pod yet. Without further information, kro
considers an object ready as soon as it has been created.
Try it with an image that doesn’t exist:
kubectl -n app patch notesapp notes --type merge \
-p '{"spec":{"image":"ghcr.io/thomaszachmann/notes-api:9.9.9"}}'
kubectl -n app get notesapp notes
kubectl -n app get podsExpected:
notesapp.lab.example.com/notes patched
NAME STATE READY AGE
notes ACTIVE True 3m
NAME READY STATUS RESTARTS AGE
notes-6d8b7f5c9-2kq4x 0/1 ErrImagePull 0 8s
notes-86b5dd7468-lsqd9 1/1 Running 0 3m
The instance reports ACTIVE even though the new pod
never starts. You’ll fix that on Day 5. For now, revert the image:
kubectl -n app patch notesapp notes --type json \
-p '[{"op":"remove","path":"/spec/image"}]'Without image, the default from the schema applies
again.
Save your progress? Sign up for free
Drill 4: A Cycle
What happens if the Deployment wants to know the name of the Service? Create a second RGD. Instead of typing the whole file, you copy it and change three places in the editor:
cp notesapp-rgd.yaml zyklus.yamlname: notesapp→name: notesapp-zykluskind: NotesApp→kind: NotesAppZyklus- in the container, insert two lines above
- name: NOTES_API_KEY:
env:
- name: NOTES_SVC
value: ${service.metadata.name}
- name: NOTES_API_KEY
kubectl apply -f zyklus.yaml
kubectl get rgd notesapp-zyklus \
-o jsonpath='{.status.state}: {.status.conditions[?(@.type=="GraphAccepted")].message}{"\n"}'Expected:
resourcegraphdefinition.kro.run/notesapp-zyklus created
Inactive: failed to build dependency graph: graph contains a cycle: service ->
deployment -> service
The Service needs the Deployment’s selector, the Deployment needs the
Service’s name. There is no possible order. If you really need the name,
you write it as text (${schema.metadata.name}), which
depends on no object.
kubectl delete -f zyklus.yamlSave your progress? Sign up for free
Checkpoint
kubectl get rgd notesapp -o wideExpected: Active, TOPOLOGICALORDER
["secret","deployment","service"].
kubectl -n app get notesappExpected:
NAME STATE READY AGE
notes ACTIVE True 6m
curl -s localhost:30080/versionExpected: 0.5.1.
diff notesapp-rgd.yaml ~/kro-lab/code/loesung/tag04/notesapp-rgd.yamlExpected: no output.
Challenge
Scale notes to three replicas, not on the Deployment but
on the instance. What happens if you then set the Deployment to one
replica by hand?
Note: Day 1, Drill 5.
Solution
kubectl -n app patch notesapp notes --type merge -p '{"spec":{"replicas":3}}'
kubectl -n app get deploy notes
kubectl -n app scale deploy notes --replicas=1
kubectl -n app get deploy notesExpected:
notesapp.lab.example.com/notes patched
NAME READY UP-TO-DATE AVAILABLE AGE
notes 3/3 3 3 7m
deployment.apps/notes scaled
NAME READY UP-TO-DATE AVAILABLE AGE
notes 3/3 3 3 7m
kro immediately sets replicas back to the instance’s
value. Back to one:
kubectl -n app patch notesapp notes --type merge -p '{"spec":{"replicas":1}}'Save your progress? Sign up for free
Enterprise Note
- Lean API: Give the team only the fields it should really decide. Everything else (probes, resources, security context) is set by the platform in the RGD.
- Secrets in the instance: An API key in the
specof an instance is stored in plain text in etcd and shows up in everykubectl get -o yaml. That’s fine for the lab. In production, secrets come from a secrets manager (see “OpenBao secrets hands-on”) or are generated (Day 6). - Who will win? A HorizontalPodAutoscaler and kro
would fight over
replicas. Omit the field (feature gateCELOmitFunction, Day 12) or don’t offer the autoscaler in the first place.