Skip to content
Hands-on IT IT Drills

Day 5: Status and Readiness

Goal: The instance shows available pods, image and port. It is only ready once notes-api really responds, and it reports its own conditions. · Duration: approx. 45 min · Prerequisite: Day 4

Briefing

  • Under schema.status go fields that kro reads from the objects. kro derives the types from the expressions and writes them into the CRD.
  • readyWhen on an object says when it counts as ready. In an RGD, all dependent objects wait until it is. readyWhen may only look at its own object.
  • You build your own conditions with runtime.newCondition({type, status, reason, message}). As soon as you specify any, the list status.conditions belongs to you, and kro’s four default conditions disappear from it. They remain readable via runtime.condition(schema, '…').
  • kro always sets status.state (ACTIVE, IN_PROGRESS, ERROR …) itself.

Drill 1: Status Fields

mkdir -p ~/kro-lab/tag05
cd ~/kro-lab/tag05
cp ../tag04/notesapp-rgd.yaml .

In the editor, insert the status block and the columns directly below nodePort: … (same indentation as spec:):

      nodePort: integer | required=true minimum=30000 maximum=32767
    status:
      availableReplicas: ${deployment.status.?availableReplicas.orValue(0)}
      image: ${deployment.spec.template.spec.containers[0].image}
      port: ${service.spec.ports[0].nodePort}
    additionalPrinterColumns:
      - name: Replicas
        type: integer
        jsonPath: .spec.replicas
      - name: Available
        type: integer
        jsonPath: .status.availableReplicas
      - name: Port
        type: integer
        jsonPath: .status.port
      - name: State
        type: string
        jsonPath: .status.state
  • availableReplicas is missing from a Deployment’s status as long as no pod is ready. Hence ? and orValue(0).
  • containers[0] reads the first element of a list.
kubectl apply -f notesapp-rgd.yaml
kubectl get rgd notesapp
kubectl -n app get notesapp

Expected:

resourcegraphdefinition.kro.run/notesapp configured
NAME       APIVERSION   KIND       STATE    READY   AGE
notesapp   v1alpha1     NotesApp   Active   True    94s
NAME    REPLICAS   AVAILABLE   PORT    STATE
notes   1          1           30080   ACTIVE

Save your progress? Sign up for free

Drill 2: readyWhen

notes-api has an endpoint /healthz. A readiness probe tells Kubernetes when a pod may receive requests, and readyWhen tells kro when the whole Deployment is done.

In the container api, add the probe after the env block, and attach the condition to the object deployment (at the level of template:):

                          key: apiKey
                  readinessProbe:
                    httpGet:
                      path: /healthz
                      port: 8080
      readyWhen:
        - ${deployment.status.?availableReplicas.orValue(0) == deployment.spec.replicas}
    - id: service
kubectl apply -f notesapp-rgd.yaml
kubectl get rgd notesapp

Expected: STATE Active.

Now yesterday’s broken instance, as a separate instance:

cat > kaputt.yaml <<'EOF'
apiVersion: lab.example.com/v1alpha1
kind: NotesApp
metadata:
  name: kaputt
  namespace: app
spec:
  image: ghcr.io/thomaszachmann/notes-api:9.9.9
  apiKey: lab-api-key-123
  nodePort: 30081
EOF
kubectl apply -f kaputt.yaml
kubectl -n app get notesapp

Expected (after about 30 seconds):

NAME     REPLICAS   AVAILABLE   PORT    STATE
kaputt   1          0                   IN_PROGRESS
notes    1          1           30080   ACTIVE

IN_PROGRESS instead of ACTIVE. And PORT is empty, because the Service doesn’t even exist yet:

kubectl -n app get deploy,svc,secret -l kro.run/instance-name=kaputt

Expected:

NAME                     READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/kaputt   0/1     1            0           31s

NAME                TYPE     DATA   AGE
secret/kaputt-api   Opaque   1      31s

The Service depends on the Deployment, and the Deployment isn’t ready. So kro waits. No team gets a port to an app that isn’t running.

Save your progress? Sign up for free

Drill 3: Your Own Conditions

Why is kaputt waiting? So far you have to read kro’s conditions to find out. Better is a condition that says in words how many pods are running. Insert below port: ${service…}:

      port: ${service.spec.ports[0].nodePort}
      conditions:
        - |-
          ${runtime.newCondition({
            type: 'Ready',
            status: runtime.condition(schema, 'ResourcesReady').status,
            reason: 'Lifecycle',
            message: runtime.condition(schema, 'ResourcesReady').message
          })}
        - |-
          ${runtime.newCondition({
            type: 'ApiAvailable',
            status: deployment.status.?availableReplicas.orValue(0) > 0 ? 'True' : 'False',
            reason: 'AvailableReplicas',
            message: string(deployment.status.?availableReplicas.orValue(0)) + ' von ' + string(schema.spec.replicas) + ' Pods bereit'
          })}
  • |- is a YAML block: the expression may span several lines.
  • Strings in CEL are in single quotes here. That way they don’t clash with YAML.
  • The first condition Ready passes on kro’s own condition ResourcesReady. Without it, the instance would no longer have any Ready at all, and kubectl wait would wait in vain.
kubectl apply -f notesapp-rgd.yaml
kubectl -n app get notesapp kaputt \
  -o jsonpath='{range .status.conditions[*]}{.type}={.status} {.reason}: {.message}{"\n"}{end}'

Expected:

Ready=False Lifecycle: waiting for unresolved resource: apply "deployment": node
"deployment": readyWhen "deployment.status.?availableReplicas.orValue(0) ==
deployment.spec.replicas" is false: waiting for readiness (executor: node not ready)
ApiAvailable=False AvailableReplicas: 0 von 1 Pods bereit

One line is now enough to see what’s holding things up: the Deployment has no finished pod. With notes, everything is green:

kubectl -n app get notesapp notes \
  -o jsonpath='{range .status.conditions[*]}{.type}={.status} {.reason}: {.message}{"\n"}{end}'

Expected:

Ready=True Lifecycle: all resources are created and ready
ApiAvailable=True AvailableReplicas: 1 von 1 Pods bereit

Save your progress? Sign up for free

Drill 4: Waiting Like a Script

Because the instance has a condition Ready, kubectl wait works. You need that in pipelines:

kubectl -n app wait notesapp/notes --for=condition=Ready --timeout=60s
kubectl -n app wait notesapp/kaputt --for=condition=Ready --timeout=10s

Expected:

notesapp.lab.example.com/notes condition met
error: timed out waiting for the condition on notesapps/kaputt

Save your progress? Sign up for free

Checkpoint

kubectl -n app get notesapp notes

Expected:

NAME    REPLICAS   AVAILABLE   PORT    STATE
notes   1          1           30080   ACTIVE
kubectl -n app get notesapp notes -o jsonpath='{.status.image}{"\n"}'

Expected: ghcr.io/thomaszachmann/notes-api:0.5.1.

diff notesapp-rgd.yaml ~/kro-lab/code/loesung/tag05/notesapp-rgd.yaml

Expected: no output.

Challenge

Repair kaputt without deleting the instance, and watch the Service being created. Delete the instance afterwards.

Note: Without image, the default applies.

Solution

kubectl -n app patch notesapp kaputt --type json \
  -p '[{"op":"remove","path":"/spec/image"}]'
kubectl -n app wait notesapp/kaputt --for=condition=Ready --timeout=90s
kubectl -n app get notesapp kaputt
kubectl -n app delete notesapp kaputt

Expected:

notesapp.lab.example.com/kaputt patched
notesapp.lab.example.com/kaputt condition met
NAME     REPLICAS   AVAILABLE   PORT    STATE
kaputt   1          1           30081   ACTIVE
notesapp.lab.example.com "kaputt" deleted from app namespace

As soon as the Deployment was ready, kro created the Service and wrote the port into the status.

Save your progress? Sign up for free

Enterprise Note

  • Readiness is mandatory: Without readyWhen, a platform API reports “done” just because the API server accepted an object. Pipelines that wait for Ready then wait for nothing. Define readyWhen for everything that takes time.
  • Domain conditions: ApiAvailable tells the team more than ResourcesReady. With the feature gate InstanceConditionEvents (Day 12), kro emits an event on every change, and your monitoring can react to it.
  • Status is an interface: Other RGDs (Day 10) and tools like Backstage read the status. Name fields the way you would name API fields, and don’t change them lightly. A removed status field is a breaking change (Day 11).