Day 13: kro, Helm, Kustomize or Operator?
Goal: The same NotesApp from Day 4, once as a Helm chart and once with Kustomize. Afterwards you know when each tool fits and when your own operator is worth it. · Duration: approx. 50 min · Prerequisite: Day 12
Briefing
- Helm renders text templates into YAML and applies
the result once. After that, Helm only takes care of it again at the
next
helm upgrade. - Kustomize assembles YAML files and patches them. There are no variables, no logic and no running process.
- kro runs as a controller. It checks expressions up front, knows dependencies and readiness and holds the state in place permanently.
- Your own operator in Go can do anything, including logic that no YAML can express. But it is software that you have to write, test and operate.
Drill 1: notes as a Helm Chart
mkdir -p ~/kro-lab/tag13/helm/notes/templates
cd ~/kro-lab/tag13
cat > helm/notes/Chart.yaml <<'EOF'
apiVersion: v2
name: notes
version: 0.1.0
appVersion: "0.5.1"
EOF
cat > helm/notes/values.yaml <<'EOF'
image: ghcr.io/thomaszachmann/notes-api:0.5.1
replicas: 1
apiKey: ""
nodePort: 30090
EOF
cat > helm/notes/templates/secret.yaml <<'EOF'
apiVersion: v1
kind: Secret
metadata:
name: {{ .Release.Name }}-api
stringData:
apiKey: {{ required "apiKey fehlt" .Values.apiKey | quote }}
EOF
cat > helm/notes/templates/deployment.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Release.Name }}
spec:
replicas: {{ .Values.replicas }}
selector:
matchLabels:
app: {{ .Release.Name }}
template:
metadata:
labels:
app: {{ .Release.Name }}
spec:
containers:
- name: api
image: {{ .Values.image }}
ports:
- containerPort: 8080
env:
- name: NOTES_API_KEY
valueFrom:
secretKeyRef:
name: {{ .Release.Name }}-api
key: apiKey
EOF
cat > helm/notes/templates/service.yaml <<'EOF'
apiVersion: v1
kind: Service
metadata:
name: {{ .Release.Name }}
spec:
type: NodePort
selector:
app: {{ .Release.Name }}
ports:
- port: 80
targetPort: 8080
nodePort: {{ .Values.nodePort }}
EOF{{ … }} is Go template syntax. Helm replaces text
without knowing what a Deployment is. A
{{ .Values.replica }} (typo) becomes an empty value, and
you only notice when the API server rejects the finished YAML or a
default kicks in.
helm lint helm/notes --set apiKey=x
helm install notes helm/notes -n helm-demo --create-namespace \
--set apiKey=lab-api-key-123
helm install notes2 helm/notes -n helm-demoExpected (shortened):
1 chart(s) linted, 0 chart(s) failed
NAME: notes
NAMESPACE: helm-demo
STATUS: deployed
REVISION: 1
Error: INSTALLATION FAILED: execution error at (notes/templates/secret.yaml:6:13):
apiKey fehlt
required is Helm’s substitute for a required field.
Limits like minimum or patterns are only available with an
additional file values.schema.json.
curl -s localhost:30090/versionExpected: 0.5.1.
Save your progress? Sign up for free
Drill 2: Drift Under Helm
kubectl -n helm-demo scale deploy notes --replicas=3
kubectl -n helm-demo get deploy notesExpected (after a few seconds):
deployment.apps/notes scaled
NAME READY UP-TO-DATE AVAILABLE AGE
notes 3/3 3 3 4m
Three replicas stay three: Helm doesn’t run in the cluster and doesn’t see the change. At the next upgrade:
helm upgrade notes helm/notes -n helm-demo --set apiKey=lab-api-key-123Expected (shortened):
Error: UPGRADE FAILED: conflict occurred while applying object helm-demo/notes
apps/v1, Kind=Deployment: Apply failed with 1 conflict: conflict with "kubectl" with
subresource "scale" using apps/v1: .spec.replicas
Helm 4 applies via server-side apply, and since the
scale, replicas belongs to the field manager
kubectl. Only with --force-conflicts does Helm
take the field back:
helm upgrade notes helm/notes -n helm-demo --set apiKey=lab-api-key-123 --force-conflicts
kubectl -n helm-demo get deploy notesExpected (shortened):
STATUS: deployed
REVISION: 3
NAME READY UP-TO-DATE AVAILABLE AGE
notes 1/1 1 1 5m
On Day 11, kro reverted the same intervention within a second, without anyone having to do anything.
Save your progress? Sign up for free
Drill 3: notes with Kustomize
mkdir -p kustomize/base kustomize/prod
cat > kustomize/base/kustomization.yaml <<'EOF'
resources:
- deployment.yaml
- service.yaml
secretGenerator:
- name: notes-api
literals:
- apiKey=lab-api-key-123
EOF
cp helm/notes/templates/deployment.yaml kustomize/base/
cp helm/notes/templates/service.yaml kustomize/base/In both copied files, replace every {{ … }} with a fixed
value: {{ .Release.Name }} with notes,
{{ .Values.replicas }} with 1,
{{ .Values.image }} with
ghcr.io/thomaszachmann/notes-api:0.5.1,
{{ .Release.Name }}-api with notes-api and
{{ .Values.nodePort }} with 30091. Kustomize
has no placeholders. Then the variant for prod:
cat > kustomize/prod/kustomization.yaml <<'EOF'
namespace: kust-demo
resources:
- ../base
patches:
- target:
kind: Deployment
name: notes
patch: |-
- op: replace
path: /spec/replicas
value: 2
EOF
kubectl create namespace kust-demo
kubectl kustomize kustomize/prod | head -9Expected:
namespace/kust-demo created
apiVersion: v1
data:
apiKey: bGFiLWFwaS1rZXktMTIz
kind: Secret
metadata:
name: notes-api-mhd7mgtg64
namespace: kust-demo
type: Opaque
---
The secretGenerator appends a hash to the name. If the
content changes, the name changes, and Kustomize puts the new name into
the Deployment. That forces a rollout.
kubectl apply -k kustomize/prod
kubectl -n kust-demo get deploy
curl -s localhost:30091/versionExpected:
secret/notes-api-mhd7mgtg64 created
service/notes created
deployment.apps/notes created
NAME READY UP-TO-DATE AVAILABLE AGE
notes 2/2 2 2 21s
0.5.1
Save your progress? Sign up for free
Drill 4: Side by Side
find helm -type f | xargs wc -l | tail -1
find kustomize -type f | xargs wc -l | tail -1
wc -l ../tag04/notesapp-rgd.yaml ../tag04/notes.yaml | tail -1Expected:
51 total
55 total
71 total
About the same amount of YAML. The difference lies in what happens afterwards:
| Helm | Kustomize | kro | Go operator | |
|---|---|---|---|---|
| Runs in the cluster | no | no | yes | yes |
| Up-front check | lint, optional JSON schema | no | types, fields, cycles | compiler, tests |
| Required fields, limits | required, schema file |
no | in the CRD schema | in the CRD schema |
| Order, readiness | hooks, --wait |
no | from expressions, readyWhen |
in code |
| Drift correction | at the next upgrade | at the next apply | immediately | immediately |
| Own API for teams | no | no | yes | yes |
| Logic (loops, API calls) | templates | no | CEL, forEach |
everything |
| What you operate | a CLI | a CLI | a controller | your software |
Save your progress? Sign up for free
Drill 5: When an Operator?
kro covers what can be described as a graph of Kubernetes objects. You need your own operator as soon as one of these applies:
- You have to talk to external systems for which there is no controller, such as a company API that creates users.
- You need processes over time: first backup, then upgrade, roll back on error. kro only knows “this is how it should look”, not steps.
- You need real logic that CEL can’t express, or performance for thousands of instances with your own caching.
Often the combination is best: an operator (self-written or ready-made, like CloudNativePG or AWS Controllers for Kubernetes) provides individual building blocks, and kro assembles them into an API for the teams. How to write an operator is shown in “Kubernetes operators hands-on”.
Save your progress? Sign up for free
Checkpoint
helm list -n helm-demoExpected: notes with STATUS
deployed.
kubectl -n kust-demo get deploy notesExpected: READY 2/2.
diff -r kustomize ~/kro-lab/code/loesung/tag13/kustomizeExpected: no output.
Challenge
Remove both variants. Which command deletes everything with Helm, and
what is left over with Kustomize if you only call
kubectl delete -k?
Note: Who created the namespace
kust-demo?
Solution
helm uninstall notes -n helm-demo
kubectl delete -k kustomize/prod
kubectl delete namespace helm-demo kust-demoExpected:
release "notes" uninstalled
secret "notes-api-mhd7mgtg64" deleted from kust-demo namespace
service "notes" deleted from kust-demo namespace
deployment.apps "notes" deleted from kust-demo namespace
namespace "helm-demo" deleted
namespace "kust-demo" deleted
helm uninstall deletes everything from the release,
delete -k everything the kustomization generates. The
namespaces stay in place in both cases: Helm created
helm-demo with --create-namespace but didn’t
include it in the release, and you created kust-demo by
hand.
Save your progress? Sign up for free
Enterprise Note
- Not either-or: kro itself comes as a Helm chart. Many platforms distribute RGDs and their ClusterRoles with Helm or Kustomize, and the instances via GitOps.
- Existing charts: You don’t have to rebuild an
existing Helm chart for kro. With Flux’s Helm controller
(
HelmRelease) as an object in an RGD, the chart still gets a simple API. - Decide by operations, not by lines: The amount of YAML is the same everywhere. The question is who makes sure the state in the cluster is correct when nobody is looking.