Skip to content
Hands-on IT IT Drills

Day 3: Kubebuilder Project and First CRD

Goal: The skeleton of the site-operator is in place: Kubebuilder project, API type StaticSite, generated CRD in envtest, kubectl explain shows your fields · Duration: approx. 120 min · Prerequisite: Day 2

Briefing

  • Kubebuilder generates the project (manager, Makefile, Kustomize) and, per API, a Go type plus a controller skeleton.
  • The truth lives in the Go type: marker comments // +kubebuilder:... control schema, defaults and subresources.
  • controller-gen turns that into CRD YAML, RBAC and DeepCopy code: make manifests generate.
  • The API server validates and applies defaults – without any running operator.
  • From today on, site-operator is the project that grows until Day 13.
 api/v1alpha1/staticsite_types.go      (Go type + markers)
            |
            |  make manifests            make generate
            v                                 v
 config/crd/bases/*.yaml (CRD)      zz_generated.deepcopy.go
 config/rbac/role.yaml   (RBAC)               |
            |                                 v
            v                         go build ./... (manager)
 kube-apiserver: schema, defaults, validation, kubectl explain

Drill 1: Create the project

source ~/op-lab/env.sh
mkdir -p ~/op-lab/tag03-site-operator && cd ~/op-lab/tag03-site-operator
kubebuilder init --domain lab.example --repo example.com/site-operator \
  --project-name site-operator --license none

Expected (shortened, without go: downloading lines):

INFO Writing kustomize manifests for you to edit...
INFO Writing scaffold for you to edit...
INFO Get controller runtime
INFO Update dependencies
Next: define a resource with:
$ kubebuilder create api

--project-name keeps the name stable, even if you later copy the folder for Day 4; --license none saves you the Apache header in every file. Which versions did Kubebuilder choose?

grep -E 'controller-runtime|k8s.io/(api|client-go) |^go ' go.mod
grep -E '^(CONTROLLER_TOOLS|KUSTOMIZE|GOLANGCI_LINT)_VERSION' Makefile

Expected:

go 1.26.0
    k8s.io/client-go v0.37.0
    sigs.k8s.io/controller-runtime v0.25.0
    k8s.io/api v0.37.0 // indirect
KUSTOMIZE_VERSION ?= v5.8.1
CONTROLLER_TOOLS_VERSION ?= v0.22.0
GOLANGCI_LINT_VERSION ?= v2.13.1

Save your progress? Sign up for free

Drill 2: A tour of the skeleton

ls; ls config

Expected:

AGENTS.md   config      go.mod      Makefile    README.md
cmd         Dockerfile  go.sum      PROJECT     test
default     manager     network-policy  prometheus  rbac
Path Contents Who writes it? From day
cmd/main.go Manager: flags, scheme, register controllers you (rarely) 3
api/<version>/ Go types of your CRDs you 3
internal/controller/ Reconciler and envtest suite you 4
config/ Kustomize: CRD, RBAC, manager Deployment make manifests 12
PROJECT Kubebuilder’s memory Kubebuilder 3

The manager in cmd/main.go is the same as on Day 2 – just with more flags (metrics, probes, leader election, webhooks):

grep -nE 'NewManager|LeaderElectionID|HealthProbeBindAddress' cmd/main.go

Expected (your ID is a different random number):

141:    mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
145:        HealthProbeBindAddress: probeAddr,
147:        LeaderElectionID:       "b986d597.lab.example",

Save your progress? Sign up for free

Drill 3: Create the API

kubebuilder create api --group web --version v1alpha1 --kind StaticSite \
  --resource --controller

Expected (shortened):

WARN unable to find boilerplate file. ...
INFO api/v1alpha1/staticsite_types.go
INFO api/v1alpha1/groupversion_info.go
INFO internal/controller/suite_test.go
INFO internal/controller/staticsite_controller.go
INFO internal/controller/staticsite_controller_test.go
INFO Update dependencies
INFO Running make
Downloading sigs.k8s.io/controller-tools/cmd/[email protected]
Next: implement your new API and generate the manifests (e.g. CRDs,CRs) with:
$ make manifests

The WARN line is a consequence of --license none and harmless. --resource --controller answers the two prompts up front. Group and domain make up the API group:

grep -oE '\+groupName=.*|Group: "[^"]*"' api/v1alpha1/groupversion_info.go
tail -6 PROJECT

Expected:

+groupName=web.lab.example
Group: "web.lab.example"
  domain: lab.example
  group: web
  kind: StaticSite
  path: example.com/site-operator/api/v1alpha1
  version: v1alpha1
version: "3"

Save your progress? Sign up for free

Drill 4: The StaticSite type

Replace the sample field foo with the contract of the site-operator. Every line with + is a marker for controller-gen; normal comments become the field description in kubectl explain.

cat > api/v1alpha1/staticsite_types.go <<'EOF'
package v1alpha1

import (
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    "k8s.io/apimachinery/pkg/runtime"
)

// ConfigMapReference zeigt auf eine ConfigMap im selben Namespace.
type ConfigMapReference struct {
    // name der ConfigMap.
    // +kubebuilder:validation:MinLength=1
    // +required
    Name string `json:"name"`
}

// ContentSource beschreibt, woher der HTML-Inhalt kommt.
type ContentSource struct {
    // configMapRef liest den Inhalt aus dem Key index.html einer ConfigMap.
    // +optional
    ConfigMapRef *ConfigMapReference `json:"configMapRef,omitempty"`
}

// StaticSiteSpec ist der Wunschzustand einer StaticSite.
type StaticSiteSpec struct {
    // content ist der HTML-Inhalt der Seite (alternativ zu contentFrom).
    // +kubebuilder:validation:MaxLength=65536
    // +optional
    Content string `json:"content,omitempty"`

    // contentFrom verweist auf eine vorhandene ConfigMap (alternativ zu content).
    // +optional
    ContentFrom *ContentSource `json:"contentFrom,omitempty"`

    // replicas ist die Anzahl der nginx-Pods.
    // +kubebuilder:default=1
    // +kubebuilder:validation:Minimum=1
    // +kubebuilder:validation:Maximum=10
    // +optional
    Replicas int32 `json:"replicas,omitempty"`

    // image ist das Webserver-Image.
    // +kubebuilder:default="nginxinc/nginx-unprivileged:1.31-alpine"
    // +optional
    Image string `json:"image,omitempty"`

    // path ist der URL-Pfad, unter dem die Seite ausgeliefert wird.
    // +kubebuilder:default="/"
    // +optional
    Path string `json:"path,omitempty"`
}

// StaticSiteStatus ist der beobachtete Zustand einer StaticSite.
type StaticSiteStatus struct {
    // readyReplicas ist die Zahl der bereiten Pods des Deployments.
    // +optional
    ReadyReplicas int32 `json:"readyReplicas,omitempty"`

    // url ist die clusterinterne Adresse, z. B. http://demo.default.svc.
    // +optional
    URL string `json:"url,omitempty"`

    // observedGeneration ist die zuletzt verarbeitete metadata.generation.
    // +optional
    ObservedGeneration int64 `json:"observedGeneration,omitempty"`

    // conditions: Available, Progressing, Degraded.
    // +listType=map
    // +listMapKey=type
    // +optional
    Conditions []metav1.Condition `json:"conditions,omitempty"`
}

// +kubebuilder:object:root=true
// +kubebuilder:subresource:status
// +kubebuilder:resource:shortName=site

// StaticSite liefert eine statische Webseite mit nginx aus.
type StaticSite struct {
    metav1.TypeMeta `json:",inline"`

    // +optional
    metav1.ObjectMeta `json:"metadata,omitzero"`

    // +required
    Spec StaticSiteSpec `json:"spec"`

    // +optional
    Status StaticSiteStatus `json:"status,omitzero"`
}

// +kubebuilder:object:root=true

// StaticSiteList enthält eine Liste von StaticSites.
type StaticSiteList struct {
    metav1.TypeMeta `json:",inline"`
    metav1.ListMeta `json:"metadata,omitzero"`
    Items           []StaticSite `json:"items"`
}

func init() {
    SchemeBuilder.Register(func(s *runtime.Scheme) error {
        s.AddKnownTypes(SchemeGroupVersion, &StaticSite{}, &StaticSiteList{})
        return nil
    })
}
EOF
go build ./...

Expected:

# example.com/site-operator/api/v1alpha1
api/v1alpha1/zz_generated.deepcopy.go:74:8: in.Foo undefined (type *StaticSiteSpec ...

That’s right: the generated DeepCopy code still knows the old field foo. Generated code never follows the type on its own – Drill 5 takes care of that.

Marker Effect in the CRD schema Who checks?
+kubebuilder:default=1 default: 1, field is set on save API server
+kubebuilder:validation:Maximum=10 maximum: 10 API server
+kubebuilder:validation:MaxLength=65536 max. 64 KiB of HTML API server
+kubebuilder:subresource:status /status separate from spec API server
+kubebuilder:resource:shortName=site kubectl get site kubectl
+listType=map + +listMapKey=type Conditions mergeable by type (SSA) API server

Warning: Nobody checks yet whether content or contentFrom is set. That rule comes as a CEL expression on Day 8.

Save your progress? Sign up for free

Drill 5: Generate

make manifests generate
ls config/crd/bases
grep -nE 'default:|maximum|minimum: 1|maxLength: 65536|- site' \
  config/crd/bases/web.lab.example_staticsites.yaml
go build ./... && echo "build ok"

Expected (shortened):

".../bin/controller-gen" rbac:roleName=manager-role crd webhook paths="./..." ...
".../bin/controller-gen" object paths="./..."
web.lab.example_staticsites.yaml
15:    - site
47:                maxLength: 65536
66:                default: nginxinc/nginx-unprivileged:1.31-alpine
70:                default: /
75:                default: 1
78:                maximum: 10
79:                minimum: 1
build ok

make generate writes zz_generated.deepcopy.go. Without DeepCopyObject() your type is not a runtime.Object and the client won’t accept it:

grep -n '^func' api/v1alpha1/zz_generated.deepcopy.go | head -4

Expected:

13:func (in *ConfigMapReference) DeepCopyInto(out *ConfigMapReference) {
18:func (in *ConfigMapReference) DeepCopy() *ConfigMapReference {
28:func (in *ContentSource) DeepCopyInto(out *ContentSource) {
38:func (in *ContentSource) DeepCopy() *ContentSource {

RBAC comes from markers too – from the +kubebuilder:rbac lines above Reconcile:

grep -n 'kubebuilder:rbac' internal/controller/staticsite_controller.go | cut -c1-80
grep -A2 'apiGroups' config/rbac/role.yaml | head -3

Expected:

20:// +kubebuilder:rbac:groups=web.lab.example,resources=staticsites,verbs=get;l
21:// +kubebuilder:rbac:groups=web.lab.example,resources=staticsites/status,verb
22:// +kubebuilder:rbac:groups=web.lab.example,resources=staticsites/finalizers,
- apiGroups:
  - web.lab.example
  resources:

Tip: Never edit generated files by hand. Adjust the type or marker, make manifests generate, done. A git diff afterwards shows exactly what the marker changed.

Save your progress? Sign up for free

Drill 6: Install the CRD in envtest

Open a second terminal and start the lab control plane from Day 0 – this time without CRDs, because the Makefile installs them in a moment:

source ~/op-lab/env.sh
envtest-up

Expected (port varies):

envtest läuft: https://127.0.0.1:59710/
export KUBECONFIG=/Users/<du>/op-lab/envtest.kubeconfig
Beenden mit Ctrl+C

Back in the first terminal. make install builds the CRD with Kustomize and sends it via kubectl apply to the cluster from KUBECONFIG – thanks to env.sh, that’s envtest:

kubectl config current-context
make install
kubectl wait --for condition=established crd/staticsites.web.lab.example
kubectl api-resources --api-group=web.lab.example

Expected (shortened):

envtest
Downloading sigs.k8s.io/kustomize/kustomize/[email protected]
customresourcedefinition.apiextensions.k8s.io/staticsites.web.lab.example created
customresourcedefinition.apiextensions.k8s.io/staticsites.web.lab.example condition met
NAME          SHORTNAMES   APIVERSION                 NAMESPACED   KIND
staticsites   site         web.lab.example/v1alpha1   true         StaticSite

Warning: make install, make deploy and go run ./cmd/main.go use the cluster from KUBECONFIG or ~/.kube/config. Check beforehand with kubectl config current-context that it says envtest.

Now the API server knows your schema – including your comments:

kubectl explain staticsite.spec
kubectl explain site.status --recursive | sed -n '/FIELDS/,$p'

Expected (shortened):

FIELDS:
  content   <string>
    content ist der HTML-Inhalt der Seite (alternativ zu contentFrom).

  contentFrom   <Object>
    contentFrom verweist auf eine vorhandene ConfigMap (alternativ zu content).
  ...
FIELDS:
  conditions    <[]Object>
    lastTransitionTime  <string> -required-
    ...
  observedGeneration    <integer>
  readyReplicas <integer>
  url   <string>

Tip: For later days there is a shorter way: envtest-up -crds config/crd/bases, started in the project folder, installs the CRDs right at startup.

Save your progress? Sign up for free

Drill 7: Defaults and validation live

Kubebuilder’s sample manifest gets real content:

cat > config/samples/web_v1alpha1_staticsite.yaml <<'EOF'
apiVersion: web.lab.example/v1alpha1
kind: StaticSite
metadata:
  labels:
    app.kubernetes.io/name: site-operator
    app.kubernetes.io/managed-by: kustomize
  name: staticsite-sample
spec:
  content: |
    <h1>Hallo vom site-operator</h1>
  replicas: 2
EOF
kubectl apply -f config/samples/web_v1alpha1_staticsite.yaml
kubectl get site staticsite-sample -o yaml | sed -n '/^spec:/,$p'

Expected:

staticsite.web.lab.example/staticsite-sample created
spec:
  content: |
    <h1>Hallo vom site-operator</h1>
  image: nginxinc/nginx-unprivileged:1.31-alpine
  path: /
  replicas: 2

The API server added image and path – no operator is running. Now a broken object:

cat > /tmp/kaputt.yaml <<'EOF'
apiVersion: web.lab.example/v1alpha1
kind: StaticSite
metadata:
  name: kaputt
spec:
  replicas: 0
  contentFrom:
    configMapRef: {}
EOF
kubectl apply -f /tmp/kaputt.yaml

Expected (shortened):

The StaticSite "kaputt" is invalid:
* spec.replicas: Invalid value: 0: ... should be greater than or equal to 1
* spec.contentFrom.configMapRef.name: Required value

Why? A default only applies when the field is missing. An explicit 0 is validated and rejected.

The status subresource separates spec (owned by the user) from status (owned by the operator):

kubectl patch site staticsite-sample --type merge \
  -p '{"status":{"url":"http://falsch"}}'
kubectl patch site staticsite-sample --subresource=status --type merge \
  -p '{"status":{"url":"http://staticsite-sample.default.svc"}}'
kubectl patch site staticsite-sample --type merge -p '{"spec":{"replicas":3}}'
kubectl get site staticsite-sample \
  -o jsonpath='{.metadata.generation} {.status.url}{"\n"}'

Expected:

staticsite.web.lab.example/staticsite-sample patched (no change)
staticsite.web.lab.example/staticsite-sample patched
staticsite.web.lab.example/staticsite-sample patched
2 http://staticsite-sample.default.svc

A status field in a normal patch is silently dropped. metadata.generation only counts spec changes – you use that on Day 5 for observedGeneration.

Save your progress? Sign up for free

Drill 8: Skeleton and tests working together

The empty reconciler already runs – as a process on your Mac against envtest. -health-probe-bind-address=0 turns off the probe port; metrics are off by default:

go run ./cmd/main.go -health-probe-bind-address=0

Expected (shortened; press Ctrl+C after a few seconds):

INFO    setup   Starting manager
INFO    Starting EventSource    {"controller": "staticsite", ...
INFO    Starting Controller {"controller": "staticsite", ...
INFO    Starting workers    {"controller": "staticsite", ... "worker count": 1}
^C
INFO    Shutdown signal received, waiting for all workers to finish
INFO    Wait completed, proceeding to shutdown the manager

You pin down what the API server guarantees in a test. The Kubebuilder suite (suite_test.go) starts its own envtest instance and installs the CRDs from config/crd/bases:

cat > internal/controller/staticsite_api_test.go <<'EOF'
package controller

import (
    . "github.com/onsi/ginkgo/v2"
    . "github.com/onsi/gomega"
    apierrors "k8s.io/apimachinery/pkg/api/errors"
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

    webv1alpha1 "example.com/site-operator/api/v1alpha1"
)

var _ = Describe("StaticSite API", func() {
    newSite := func(name string) *webv1alpha1.StaticSite {
        return &webv1alpha1.StaticSite{
            ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: "default"},
            Spec:       webv1alpha1.StaticSiteSpec{Content: "<h1>Test</h1>"},
        }
    }

    It("setzt Defaults im API-Server", func(ctx SpecContext) {
        site := newSite("defaults")
        Expect(k8sClient.Create(ctx, site)).To(Succeed())
        DeferCleanup(func(ctx SpecContext) {
            Expect(k8sClient.Delete(ctx, site)).To(Succeed())
        })
        // Create schreibt die Antwort des Servers zurück in site.
        Expect(site.Spec.Replicas).To(Equal(int32(1)))
        Expect(site.Spec.Image).To(Equal("nginxinc/nginx-unprivileged:1.31-alpine"))
        Expect(site.Spec.Path).To(Equal("/"))
    })

    It("lehnt replicas > 10 ab", func(ctx SpecContext) {
        site := newSite("zu-viele")
        site.Spec.Replicas = 11
        err := k8sClient.Create(ctx, site)
        Expect(apierrors.IsInvalid(err)).To(BeTrue(), "err = %v", err)
    })
})
EOF
go test ./internal/controller/ -v -ginkgo.v -ginkgo.no-color 2>&1 \
  | grep -E '^(StaticSite|Ran [0-9]|SUCCESS)'

Expected (shortened, order may vary):

StaticSite Controller When reconciling a resource should successfully reconcile ...
StaticSite API setzt Defaults im API-Server
StaticSite API lehnt replicas > 10 ab
Ran 3 of 3 Specs in 5.008 seconds
SUCCESS! -- 3 Passed | 0 Failed | 0 Pending | 0 Skipped

Why? Create writes the server’s response back into site – that’s why you see the defaults without another Get. The ERROR Failed to read directory ../../bin/k8s line in the log is harmless: the suite looks for binaries there, but you supply them via KUBEBUILDER_ASSETS.

Save your progress? Sign up for free

Checkpoint

envtest-up is running in the second terminal, the CRD and the sample are installed:

cd ~/op-lab/tag03-site-operator
ok() { echo "✔ $1"; }; nok() { echo "✘ $1"; }
go build ./... && go vet ./... && ok "Build und vet" || nok "Build/vet"
grep -q 'default: nginxinc/nginx-unprivileged:1.31-alpine' \
  config/crd/bases/web.lab.example_staticsites.yaml && ok "CRD mit Defaults" \
  || nok "CRD fehlt – make manifests"
go test ./internal/... >/dev/null 2>&1 && ok "envtest-Suite grün" || nok "Tests"
[ "$(kubectl config current-context)" = envtest ] && ok "Kontext envtest" \
  || nok "Kontext – env.sh gesourct? envtest-up läuft?"
kubectl explain staticsite.spec.contentFrom >/dev/null 2>&1 \
  && ok "CRD installiert (kubectl explain)" || nok "make install"
[ "$(kubectl get site staticsite-sample -o jsonpath='{.spec.image}')" = \
  nginxinc/nginx-unprivileged:1.31-alpine ] && ok "Default von image gesetzt" \
  || nok "Sample fehlt"

Expected:

✔ Build und vet
✔ CRD mit Defaults
✔ envtest-Suite grün
✔ Kontext envtest
✔ CRD installiert (kubectl explain)
✔ Default von image gesetzt

Then stop envtest-up in the second terminal with Ctrl+C.

Challenge

Prove with a test that a StaticSite with contentFrom.configMapRef: {} (without name) is rejected – and that the error message names the field.

Hint: ContentFrom and ConfigMapRef are pointers; +required on the name field makes it mandatory in the schema, and the error is IsInvalid again.

Solution

cat >> internal/controller/staticsite_api_test.go <<'EOF'

var _ = Describe("StaticSite API – contentFrom", func() {
    It("verlangt configMapRef.name", func(ctx SpecContext) {
        site := &webv1alpha1.StaticSite{
            ObjectMeta: metav1.ObjectMeta{Name: "ohne-name", Namespace: "default"},
            Spec: webv1alpha1.StaticSiteSpec{
                ContentFrom: &webv1alpha1.ContentSource{
                    ConfigMapRef: &webv1alpha1.ConfigMapReference{},
                },
            },
        }
        err := k8sClient.Create(ctx, site)
        Expect(apierrors.IsInvalid(err)).To(BeTrue(), "err = %v", err)
        Expect(err.Error()).To(ContainSubstring("spec.contentFrom.configMapRef.name"))
    })
})
EOF
go test ./internal/controller/ -count=1 -v -ginkgo.no-color 2>&1 \
  | grep -E '^(Ran [0-9]|ok)'

Expected:

Ran 4 of 4 Specs in 5.110 seconds
ok      example.com/site-operator/internal/controller   5.451s

Save your progress? Sign up for free

Enterprise Note

  • The API group is a contract: a real domain of your own (web.firma.de), never example or someone else’s domain. Renaming means a new CRD.
  • v1alpha1 signals “may break”. From the first production use on, compatibility rules apply (Day 9).
  • CRDs are cluster-wide: whoever installs them needs cluster-admin rights. Often the platform team ships the CRD separately from the operator.
  • Defaults in the schema instead of in code: they are visible to kubectl, GitOps diffs and other clients.
  • Generated files (config/crd, zz_generated.*) belong in Git; CI checks with make manifests generate && git diff --exit-code.