Skip to content
Hands-on IT IT Drills

Day 2: controller-runtime without Kubebuilder

Goal: restarter restarts deployments as soon as a referenced ConfigMap changes – with manager, reconciler and builder from controller-runtime · Duration: approx. 120 min · Prerequisite: Day 1

Briefing

  • controller-runtime packages Day 1 into building blocks: manager (cache, client, lifecycle), controller (queue, workers) and your reconciler.
  • All you write now is Reconcile(ctx, req): it receives Namespace/Name, reads by itself and reconciles.
  • The builder wires up watches: For (primary object), Owns (your own children, from Day 4 on), Watches (other objects with a mapping).
  • Return values instead of queue calls: err != nil means retry with backoff, nil means Forget.
  • client.Client reads from the cache and writes directly to the API server.
 Day 1 (client-go)                      Day 2 (controller-runtime)
 ------------------------------------   ---------------------------------------
 SharedInformerFactory + Lister         manager cache, mgr.GetClient().Get/List
 AddEventHandler -> queue.Add(key)      Builder: For / Owns / Watches
 Workqueue + Worker + processNext       Controller (internal, same workqueue)
 sync(key) error                        Reconcile(ctx, req) (Result, error)
 AddRateLimited / Forget                return err / return nil
 signal.NotifyContext                   ctrl.SetupSignalHandler()

Drill 1: Create the module

source ~/op-lab/env.sh
mkdir -p ~/op-lab/tag02-restarter && cd ~/op-lab/tag02-restarter
go mod init example.com/restarter
go get sigs.k8s.io/[email protected]

Expected (shortened):

go: creating new go.mod: module example.com/restarter
go: added k8s.io/api v0.37.0
go: added k8s.io/client-go v0.37.0
go: added sigs.k8s.io/controller-runtime v0.25.0

controller-runtime v0.25 pulls in client-go v0.37 – the same combination that Kubebuilder generates on Day 3.

Save your progress? Sign up for free

Drill 2: The reconciler

The contract: a deployment with the annotation restarter.lab.example/configmap: <name> gets a hash of that ConfigMap’s data on its pod template. If the hash changes, the template changes – and Kubernetes rolls out again, just like with kubectl rollout restart.

cat > reconciler.go <<'EOF'
package main

import (
    "context"
    "crypto/sha256"
    "encoding/hex"
    "maps"
    "slices"

    appsv1 "k8s.io/api/apps/v1"
    corev1 "k8s.io/api/core/v1"
    apierrors "k8s.io/apimachinery/pkg/api/errors"
    "k8s.io/apimachinery/pkg/types"
    ctrl "sigs.k8s.io/controller-runtime"
    "sigs.k8s.io/controller-runtime/pkg/builder"
    "sigs.k8s.io/controller-runtime/pkg/client"
    "sigs.k8s.io/controller-runtime/pkg/handler"
    "sigs.k8s.io/controller-runtime/pkg/log"
    "sigs.k8s.io/controller-runtime/pkg/predicate"
    "sigs.k8s.io/controller-runtime/pkg/reconcile"
)

const (
    annoConfigMap = "restarter.lab.example/configmap" // am Deployment
    annoHash      = "restarter.lab.example/hash"      // am Pod-Template
)

// Restarter restarts deployments when their ConfigMap changes.
type Restarter struct {
    client.Client
}

// Reconcile only receives Namespace/Name – it reads the rest itself (level-based).
func (r *Restarter) Reconcile(ctx context.Context, req ctrl.Request) (
    ctrl.Result, error) {
    logger := log.FromContext(ctx)

    var dep appsv1.Deployment
    if err := r.Get(ctx, req.NamespacedName, &dep); err != nil {
        return ctrl.Result{}, client.IgnoreNotFound(err) // gelöscht: fertig
    }
    cmName := dep.Annotations[annoConfigMap]
    if cmName == "" {
        return ctrl.Result{}, nil
    }

    var cm corev1.ConfigMap
    err := r.Get(ctx, types.NamespacedName{Namespace: dep.Namespace,
        Name: cmName}, &cm)
    if apierrors.IsNotFound(err) {
        logger.Info("ConfigMap fehlt, warte", "configmap", cmName)
        return ctrl.Result{}, nil // der ConfigMap-Watch weckt uns
    }
    if err != nil {
        return ctrl.Result{}, err // controller-runtime wiederholt mit Backoff
    }

    hash := dataHash(cm.Data)
    if dep.Spec.Template.Annotations[annoHash] == hash {
        return ctrl.Result{}, nil // aktuell: nichts schreiben
    }
    patch := client.MergeFrom(dep.DeepCopy())
    if dep.Spec.Template.Annotations == nil {
        dep.Spec.Template.Annotations = map[string]string{}
    }
    dep.Spec.Template.Annotations[annoHash] = hash // neues Template = Rollout
    if err := r.Patch(ctx, &dep, patch); err != nil {
        return ctrl.Result{}, err
    }
    logger.Info("Rollout ausgelöst", "configmap", cmName, "hash", hash)
    return ctrl.Result{}, nil
}

// dataHash builds a stable hash over the sorted keys and values.
func dataHash(data map[string]string) string {
    h := sha256.New()
    for _, k := range slices.Sorted(maps.Keys(data)) {
        h.Write([]byte(k + "=" + data[k] + "\n"))
    }
    return hex.EncodeToString(h.Sum(nil))[:16]
}
EOF

Three patterns you’ll see in every reconciler from now on:

Situation Return Why
Object deleted client.IgnoreNotFound(err) Nothing left to do, no retry
Dependency missing Result{}, nil + log A watch wakes us when it shows up
API error Result{}, err Backoff retry like AddRateLimited
Already up to date Result{}, nil without writing Idempotence, no rollout storm

client.MergeFrom creates a merge patch from the difference to the original. That way you write only that one field and don’t overwrite changes made by other controllers.

Save your progress? Sign up for free

Drill 3: Watches and mapping

An event on a ConfigMap has to be translated into requests for deployments – For only knows deployments. A MapFunc takes care of that:

cat >> reconciler.go <<'EOF'

// deploymentsFor translates "ConfigMap changed" into "check these deployments".
func (r *Restarter) deploymentsFor(ctx context.Context,
    obj client.Object) []reconcile.Request {
    var list appsv1.DeploymentList
    if err := r.List(ctx, &list, client.InNamespace(obj.GetNamespace())); err != nil {
        log.FromContext(ctx).Error(err, "Deployments auflisten")
        return nil
    }
    var reqs []reconcile.Request
    for _, d := range list.Items {
        if d.Annotations[annoConfigMap] == obj.GetName() {
            reqs = append(reqs, reconcile.Request{NamespacedName: types.NamespacedName{
                Namespace: d.Namespace, Name: d.Name}})
        }
    }
    return reqs
}

// SetupWithManager wires up watches: deployments directly, ConfigMaps mapped.
func (r *Restarter) SetupWithManager(mgr ctrl.Manager) error {
    annotated := predicate.NewPredicateFuncs(func(o client.Object) bool {
        _, ok := o.GetAnnotations()[annoConfigMap]
        return ok
    })
    return ctrl.NewControllerManagedBy(mgr).
        For(&appsv1.Deployment{}, builder.WithPredicates(annotated)).
        Watches(&corev1.ConfigMap{},
            handler.EnqueueRequestsFromMapFunc(r.deploymentsFor)).
        Named("restarter").
        Complete(r)
}
EOF
go mod tidy && go vet ./...

Expected: no output (except go: downloading lines the first time).

The predicate filters before the queue: deployments without the annotation don’t trigger a reconcile at all. The MapFunc reads from the cache via r.List – no extra call to the API server. On Day 10 you replace the loop with a field index.

Save your progress? Sign up for free

Drill 4: The manager

cat > main.go <<'EOF'
// restarter restarts deployments when a referenced ConfigMap changes.
// Day 2: controller-runtime without Kubebuilder (manager, reconciler, builder).
package main

import (
    "flag"
    "os"

    ctrl "sigs.k8s.io/controller-runtime"
    "sigs.k8s.io/controller-runtime/pkg/log/zap"
    metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server"
)

func main() {
    opts := zap.Options{Development: true}
    opts.BindFlags(flag.CommandLine) // -zap-log-level, -zap-devel, ...
    flag.Parse()                     // -kubeconfig registriert controller-runtime
    ctrl.SetLogger(zap.New(zap.UseFlagOptions(&opts)))
    setupLog := ctrl.Log.WithName("setup")

    // GetConfig: -kubeconfig, $KUBECONFIG, in-cluster, ~/.kube/config.
    if os.Getenv("KUBECONFIG") == "" && flag.Lookup("kubeconfig").Value.String() == "" {
        setupLog.Info("KUBECONFIG nicht gesetzt – Abbruch (Lab-Schutz)")
        os.Exit(1)
    }
    cfg := ctrl.GetConfigOrDie()

    mgr, err := ctrl.NewManager(cfg, ctrl.Options{
        Metrics:                metricsserver.Options{BindAddress: "0"}, // aus
        HealthProbeBindAddress: "0",                                     // aus
    })
    if err != nil {
        setupLog.Error(err, "Manager anlegen")
        os.Exit(1)
    }
    if err := (&Restarter{Client: mgr.GetClient()}).SetupWithManager(mgr); err != nil {
        setupLog.Error(err, "Controller anlegen")
        os.Exit(1)
    }

    setupLog.Info("starte Manager", "host", cfg.Host)
    if err := mgr.Start(ctrl.SetupSignalHandler()); err != nil {
        setupLog.Error(err, "Manager beendet")
        os.Exit(1)
    }
}
EOF
go mod tidy && go build -o restarter . && env -u KUBECONFIG ./restarter
echo "exit=$?"

Expected (without timestamps):

INFO    setup   KUBECONFIG nicht gesetzt – Abbruch (Lab-Schutz)
exit=1

Otherwise, ctrl.GetConfigOrDie() would fall back to ~/.kube/config – just like kubectl. The metrics and probe servers are switched off with "0"; they come on Day 11.

Save your progress? Sign up for free

Drill 5: Live against envtest

Terminal 2: source ~/op-lab/env.sh && envtest-up. Terminal 1:

./restarter > restarter.log 2>&1 &
kubectl create namespace lab
kubectl -n lab create configmap app-config --from-literal=greeting=hallo
kubectl -n lab create deployment web --image=nginxinc/nginx-unprivileged:1.31-alpine
kubectl -n lab annotate deployment web restarter.lab.example/configmap=app-config
sleep 2
kubectl -n lab get deploy web \
  -o jsonpath='{.metadata.generation} {.spec.template.metadata.annotations}{"\n"}'

Expected (hash and generation may differ):

namespace/lab created
configmap/app-config created
deployment.apps/web created
deployment.apps/web annotated
3 {"restarter.lab.example/hash":"7620aab77b54f1d4"}

The first reconcile sets the hash – a one-time rollout when switching it on. Now change the ConfigMap:

kubectl -n lab patch configmap app-config --type merge \
  -p '{"data":{"greeting":"moin"}}'
kubectl -n lab create deployment plain --image=nginxinc/nginx-unprivileged:1.31-alpine
sleep 2
COLS='NAME:.metadata.name,GEN:.metadata.generation'
COLS+=',HASH:.spec.template.metadata.annotations.restarter\.lab\.example/hash'
kubectl -n lab get deploy -o custom-columns="$COLS"

Expected:

configmap/app-config patched
deployment.apps/plain created
NAME    GEN   HASH
plain   1     <none>
web     4     795887ea11db139d

New hash, new generation: in a real cluster, the deployment controller would now start new pods. plain has no annotation – the predicate never let it into the queue in the first place.

Save your progress? Sign up for free

Drill 6: Read and switch logs

The log lines automatically carry the controller, the object and a reconcileID – the builder put that into the ctx, and log.FromContext(ctx) gets it out:

grep 'Rollout ausgelöst' restarter.log | tail -1 | tr ',' '\n' | head -9

Expected (shortened):

2026-10-10T10:14:33+02:00   INFO    Rollout ausgelöst   {"controller": "restarter"
 "controllerGroup": "apps"
 "controllerKind": "Deployment"
 "Deployment": {"name":"web"
"namespace":"lab"}
 "namespace": "lab"
 "name": "web"
 "reconcileID": "c5516067-d715-4096-8269-f755aae0beb7"
 "configmap": "app-config"

For production you want JSON. opts.BindFlags registered the zap flags:

kill -INT %1; sleep 1
./restarter -zap-devel=false > restarter.json 2>&1 &
sleep 2; kubectl -n lab patch configmap app-config --type merge \
  -p '{"data":{"greeting":"servus"}}'
sleep 2; grep Rollout restarter.json | jq -c '{level, msg, hash}'
kill -INT %1

Expected:

configmap/app-config patched
{"level":"info","msg":"Rollout ausgelöst","hash":"da2ca8f2b03831a9"}

Tip: -zap-log-level=debug shows more, -zap-log-level=2 even more (V(2) logs). Structured fields ("configmap", cmName) instead of fmt.Sprintf – that’s the only way Loki or Elasticsearch can filter on them.

Save your progress? Sign up for free

Drill 7: Tests with the fake client

controller-runtime ships its own fake client. It needs no informers: you call Reconcile directly and check the object afterwards.

cat > reconciler_test.go <<'EOF'
package main

import (
    "context"
    "testing"

    appsv1 "k8s.io/api/apps/v1"
    corev1 "k8s.io/api/core/v1"
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    "k8s.io/apimachinery/pkg/types"
    ctrl "sigs.k8s.io/controller-runtime"
    "sigs.k8s.io/controller-runtime/pkg/client"
    "sigs.k8s.io/controller-runtime/pkg/client/fake"
)

func deploy(name, cm string) *appsv1.Deployment {
    d := &appsv1.Deployment{ObjectMeta: metav1.ObjectMeta{
        Namespace: "lab", Name: name}}
    if cm != "" {
        d.Annotations = map[string]string{annoConfigMap: cm}
    }
    return d
}

func configMap(name, value string) *corev1.ConfigMap {
    return &corev1.ConfigMap{
        ObjectMeta: metav1.ObjectMeta{Namespace: "lab", Name: name},
        Data:       map[string]string{"setting": value},
    }
}

// reconcileHash calls Reconcile and returns the hash on the pod template.
func reconcileHash(t *testing.T, c client.Client, name string) string {
    t.Helper()
    ctx := context.Background()
    key := types.NamespacedName{Namespace: "lab", Name: name}
    r := &Restarter{Client: c}
    if _, err := r.Reconcile(ctx, ctrl.Request{NamespacedName: key}); err != nil {
        t.Fatalf("Reconcile: %v", err)
    }
    var d appsv1.Deployment
    if err := c.Get(ctx, key, &d); err != nil {
        t.Fatal(err)
    }
    return d.Spec.Template.Annotations[annoHash]
}

func TestReconcileSetztUndAendertHash(t *testing.T) {
    cm := configMap("app-config", "a")
    c := fake.NewClientBuilder().WithObjects(deploy("web", "app-config"), cm).Build()

    first := reconcileHash(t, c, "web")
    if first == "" {
        t.Fatal("kein Hash gesetzt")
    }
    if again := reconcileHash(t, c, "web"); again != first {
        t.Errorf("Hash ohne Änderung neu: %s != %s", again, first)
    }

    cm.Data["setting"] = "b"
    if err := c.Update(context.Background(), cm); err != nil {
        t.Fatal(err)
    }
    if changed := reconcileHash(t, c, "web"); changed == first {
        t.Error("Hash nach ConfigMap-Änderung unverändert")
    }
}

func TestReconcileOhneAnnotationOderConfigMap(t *testing.T) {
    c := fake.NewClientBuilder().
        WithObjects(deploy("plain", ""), deploy("waiting", "fehlt")).Build()
    for _, name := range []string{"plain", "waiting"} {
        if h := reconcileHash(t, c, name); h != "" {
            t.Errorf("%s: unerwarteter Hash %q", name, h)
        }
    }
}

func TestDeploymentsFor(t *testing.T) {
    c := fake.NewClientBuilder().WithObjects(deploy("a", "app-config"),
        deploy("b", "andere"), deploy("c", "app-config")).Build()
    r := &Restarter{Client: c}
    reqs := r.deploymentsFor(context.Background(), configMap("app-config", "x"))
    if len(reqs) != 2 {
        t.Fatalf("got %d requests, want 2: %v", len(reqs), reqs)
    }
}

func TestDataHashStabil(t *testing.T) {
    a := dataHash(map[string]string{"x": "1", "y": "2"})
    b := dataHash(map[string]string{"y": "2", "x": "1"})
    if a != b || len(a) != 16 {
        t.Errorf("hash instabil: %s vs %s", a, b)
    }
}
EOF
go mod tidy && go test -race -count=1 -v . 2>&1 | grep -E '^(---|ok)'

Expected:

--- PASS: TestReconcileSetztUndAendertHash (0.44s)
--- PASS: TestReconcileOhneAnnotationOderConfigMap (0.00s)
--- PASS: TestDeploymentsFor (0.01s)
--- PASS: TestDataHashStabil (0.00s)
ok      example.com/restarter   1.810s

The fake client checks no schemas, defaults or admission. It’s ideal for logic – what the API server guarantees, you test with envtest from Day 3 on.

Save your progress? Sign up for free

Checkpoint

envtest is running in terminal 2:

cd ~/op-lab/tag02-restarter
ok() { echo "✔ $1"; }; nok() { echo "✘ $1"; }
go vet ./... && go test -race -count=1 ./... >/dev/null && ok "vet und Tests" \
  || nok "Tests"
go build -o restarter . || nok "Build"
./restarter > kp.log 2>&1 & PID=$!
kubectl create ns kp >/dev/null
kubectl -n kp create configmap cfg --from-literal=a=1 >/dev/null
kubectl -n kp create deployment app --image=nginx >/dev/null
kubectl -n kp annotate deployment app restarter.lab.example/configmap=cfg >/dev/null
sleep 2; H1=$(kubectl -n kp get deploy app \
  -o jsonpath='{.spec.template.metadata.annotations.restarter\.lab\.example/hash}')
kubectl -n kp patch configmap cfg --type merge -p '{"data":{"a":"2"}}' >/dev/null
sleep 2; H2=$(kubectl -n kp get deploy app \
  -o jsonpath='{.spec.template.metadata.annotations.restarter\.lab\.example/hash}')
[ -n "$H1" ] && ok "Hash gesetzt" || nok "kein Hash"
[ -n "$H2" ] && [ "$H1" != "$H2" ] && ok "Rollout nach Änderung" || nok "kein Rollout"
kill -INT $PID; wait $PID
grep -q 'Wait completed' kp.log && ok "sauber beendet" || nok "Shutdown"

Expected:

✔ vet und Tests
✔ Hash gesetzt
✔ Rollout nach Änderung
✔ sauber beendet

Challenge

A List without client.InNamespace would find deployments in all namespaces that happen to reference a ConfigMap with the same name. Prove with a test that deploymentsFor only considers the ConfigMap’s namespace.

Hint: Use deploy() to create two deployments and set Namespace to "anderer" on one of them.

Solution

cat >> reconciler_test.go <<'EOF'

func TestDeploymentsForNurEigenerNamespace(t *testing.T) {
    fremd := deploy("x", "app-config")
    fremd.Namespace = "anderer"
    c := fake.NewClientBuilder().WithObjects(deploy("a", "app-config"), fremd).Build()
    reqs := (&Restarter{Client: c}).deploymentsFor(context.Background(),
        configMap("app-config", "neu"))
    if len(reqs) != 1 || reqs[0].Namespace != "lab" {
        t.Errorf("reqs = %v, want nur lab/a", reqs)
    }
}
EOF
go test -race -count=1 -run NurEigener -v . 2>&1 | grep -E '^(---|ok)'

Expected:

--- PASS: TestDeploymentsForNurEigenerNamespace (0.00s)
ok      example.com/restarter   1.512s

As a check, remove client.InNamespace(...) in deploymentsFor – the test turns red.

Save your progress? Sign up for free

Enterprise Note

  • The manager caches all ConfigMaps and deployments of the cluster. In large clusters: restrict the cache to namespaces or labels (cache.Options, Days 10 and 13).
  • A rollout storm is a real risk: a ConfigMap that 200 deployments depend on starts 200 rollouts. Factor in rate limits and maxUnavailable.
  • Established alternatives like Stakater Reloader solve exactly this problem – check them before building your own.
  • RBAC: get/list/watch on ConfigMaps, get/list/watch/patch on deployments. ConfigMaps can contain sensitive data – never pull Secrets into the cache unless you have to.
  • Multiple instances only with leader election (LeaderElection: true, Day 11).