Skip to content
Hands-on IT IT Drills

Day 3: Sources

Goal: Fetch manifests from someone else’s Git repo, via SemVer tags and as an OCI artifact, run your own registry and push your own artifacts into it. · Duration: approx. 75 min · Prerequisite: Day 2

Briefing

  • A source tells Flux where manifests come from. A Kustomization says what of it is applied and how. They are separate objects. That is why you can swap the source without changing more than sourceRef on the Kustomization.
  • The source-controller knows these kinds:
Source Fetches from Typical use
GitRepository Git (branch, tag, SemVer, commit) the fleet repo, repos of other teams
OCIRepository OCI registry (ghcr, Harbor, ECR …) artifacts from CI, Helm charts
HelmRepository classic Helm repo (index.yaml) third-party charts (Day 6)
Bucket S3-compatible storage environments without Git access
  • Today a third-party app joins: podinfo, a small web app that the Flux project itself maintains for examples. It has Git tags with versions and also publishes its manifests as an OCI artifact.
 github.com/stefanprodan/podinfo ── GitRepository (Drill 1–3)
 ghcr.io/stefanprodan/manifests/podinfo ── OCIRepository (Drill 4)
 registry in the cluster :30500 ◄── flux push artifact ── apps/notes (Drill 5)

Drill 1: Someone else’s repo

The namespace, source and Kustomization for podinfo go into a single file in clusters/lab. You write the namespace yourself; flux create generates the rest:

cd ~/flux-lab/fleet
F=clusters/lab/podinfo.yaml
printf -- '---\napiVersion: v1\nkind: Namespace\nmetadata:\n  name: podinfo\n' > $F
flux create source git podinfo --url=https://github.com/stefanprodan/podinfo \
  --tag-semver="6.16.x" --interval=10m --export >> $F
flux create kustomization podinfo --source=GitRepository/podinfo \
  --path=./kustomize --prune=true --interval=10m --target-namespace=podinfo \
  --export >> $F
cat $F

Expected:

---
apiVersion: v1
kind: Namespace
metadata:
  name: podinfo
---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: podinfo
  namespace: flux-system
spec:
  interval: 10m0s
  ref:
    semver: 6.16.x
  url: https://github.com/stefanprodan/podinfo
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: podinfo
  namespace: flux-system
spec:
  interval: 10m0s
  path: ./kustomize
  prune: true
  sourceRef:
    kind: GitRepository
    name: podinfo
  targetNamespace: podinfo
  • ref.semver: 6.16.x: among all tags of the repo, Flux looks for the highest one that matches 6.16.x. Other options are branch, tag (exactly one tag) and commit.
  • targetNamespace: the podinfo manifests have no namespace. Flux sets it when applying. Flux does not create the namespace in the process, which is why it is in the file itself.
  • The repo is public, so the source needs no secretRef.
git add -A
git commit -m "podinfo aus fremdem Repo"
git push
flux reconcile kustomization flux-system --with-source
flux get sources git

Expected (shortened):

NAME        REVISION                SUSPENDED   READY   MESSAGE
flux-system main@sha1:2be5fe37      False       True    stored artifact for revision 'main@sha1:2be5fe37'
podinfo     6.16.0@sha1:256d0346    False       True    stored artifact for revision '6.16.0@sha1:256d0346'

The revision now names the tag (6.16.0) and the commit behind it.

podinfo only has a Service of type ClusterIP, so without a NodePort. You can still ask the API server to forward the request:

kubectl -n podinfo rollout status deploy/podinfo
kubectl get --raw /api/v1/namespaces/podinfo/services/podinfo:9898/proxy/version

Expected:

deployment "podinfo" successfully rolled out
{
  "commit": "256d03466ae4277ca1746c212f92c9a1d384d307",
  "version": "6.16.0"
}

Save your progress? Sign up for free

Drill 2: A new version

A minor update is one line:

sed -i 's/semver: 6.16.x/semver: 6.17.x/' clusters/lab/podinfo.yaml
git commit -am "podinfo 6.17"
git push
flux reconcile kustomization flux-system --with-source
flux reconcile kustomization podinfo --with-source

Expected (shortened):

✔ applied revision main@sha1:9ab57e39eba62e6f94a7c14dcf97a37907659d5c
...
✔ fetched revision 6.17.0@sha1:fd3a834ed8d14b80ea334cf8a2fddd4597924da6
...
✔ applied revision 6.17.0@sha1:fd3a834ed8d14b80ea334cf8a2fddd4597924da6
kubectl -n podinfo rollout status deploy/podinfo
kubectl get --raw /api/v1/namespaces/podinfo/services/podinfo:9898/proxy/version

Expected (shortened):

  "version": "6.17.0"

A patch release such as 6.17.1 would arrive without a commit: every ten minutes the source checks for new tags and takes the highest one that matches 6.17.x.

Save your progress? Sign up for free

Drill 3: Fetch only what you need

The source downloads the entire podinfo repo, with source code, charts and docs. Only kustomize/ is applied. How large is the artifact?

kubectl -n flux-system get gitrepository podinfo \
  -o jsonpath='{.status.artifact.size}{"\n"}'

Expected (approximately):

429596

About 430 KB, for four YAML files. spec.ignore works like a .gitignore: first exclude everything, then include the folder again. Open clusters/lab/podinfo.yaml in the editor and add below url in the GitRepository:

  url: https://github.com/stefanprodan/podinfo
  ignore: |
    /*
    !/kustomize/
git commit -am "podinfo: nur kustomize/ holen"
git push
flux reconcile kustomization flux-system --with-source
flux reconcile source git podinfo
kubectl -n flux-system get gitrepository podinfo \
  -o jsonpath='{.status.artifact.size}{"\n"}'

Expected (shortened):

✔ fetched revision 6.17.0@sha1:fd3a834ed8d14b80ea334cf8a2fddd4597924da6
1150

A little over 1 KB of the repo remains. With large monorepos this saves storage in the source-controller and time at every reconciliation.

Save your progress? Sign up for free

Drill 4: OCI instead of Git

podinfo publishes the same manifests with every release as an OCI artifact on ghcr. An OCI artifact is a package in a container registry, but not an image; it is an archive with files. For Flux this is the better source: it fetches only the manifests, addressed immutably by a digest, without Git history.

In clusters/lab/podinfo.yaml, replace the entire GitRepository with:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
  name: podinfo
  namespace: flux-system
spec:
  interval: 10m0s
  ref:
    semver: 6.17.x
  url: oci://ghcr.io/stefanprodan/manifests/podinfo

In the Kustomization, two lines change. The artifact contains only the manifests, so there is no longer a subfolder:

  path: ./
  ...
  sourceRef:
    kind: OCIRepository
    name: podinfo
git commit -am "podinfo als OCI-Artefakt"
git push
flux reconcile kustomization flux-system --with-source
flux get sources all

Expected:

NAME                    REVISION                SUSPENDED   READY   MESSAGE
ocirepository/podinfo   6.17.0@sha256:3a5e82b1  False       True    stored artifact for digest '6.17.0@sha256:3a5e82b1'

NAME                        REVISION            SUSPENDED   READY   MESSAGE
gitrepository/flux-system   main@sha1:b76b5b00  False       True    stored artifact for revision 'main@sha1:b76b5b00'

The GitRepository podinfo is gone. It was in Git, you deleted it, and prune in flux-system removed it. Nothing changes for the running podinfo:

flux get kustomization podinfo

Expected:

NAME    REVISION                SUSPENDED   READY   MESSAGE
podinfo 6.17.0@sha256:3a5e82b1  False       True    Applied revision: 6.17.0@sha256:3a5e82b1

Save your progress? Sign up for free

Drill 5: A registry of your own

Anyone who wants to build their own artifacts needs a registry. In the lab it runs in the cluster, from Git, of course. It gets its own area infrastructure/:

mkdir -p infrastructure/registry
cd infrastructure/registry
printf 'apiVersion: v1\nkind: Namespace\nmetadata:\n  name: registry\n' > namespace.yaml
cat > deployment.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
  name: registry
  namespace: registry
spec:
  replicas: 1
  selector:
    matchLabels:
      app: registry
  template:
    metadata:
      labels:
        app: registry
    spec:
      containers:
        - name: registry
          image: registry:3.0.0
          ports:
            - containerPort: 5000
          readinessProbe:
            httpGet:
              path: /v2/
              port: 5000
EOF
cat > service.yaml <<'EOF'
apiVersion: v1
kind: Service
metadata:
  name: registry
  namespace: registry
spec:
  type: NodePort
  selector:
    app: registry
  ports:
    - port: 5000
      targetPort: 5000
      nodePort: 30500
EOF
cat > kustomization.yaml <<'EOF'
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - namespace.yaml
  - deployment.yaml
  - service.yaml
EOF
cd ~/flux-lab/fleet
flux create kustomization registry --source=GitRepository/flux-system \
  --path=./infrastructure/registry --prune=true --interval=10m --wait=true \
  --export > clusters/lab/registry.yaml
git add -A
git commit -m "Registry im Cluster"
git push
flux reconcile kustomization flux-system --with-source
flux reconcile kustomization registry

Expected (shortened):

✔ applied revision main@sha1:610b75c8e163da9208cac09f64960c6496946179
...
✔ applied revision main@sha1:610b75c8e163da9208cac09f64960c6496946179

--wait=true makes the Kustomization report Ready only when the registry is really running. More on that tomorrow.

curl -s localhost:30500/v2/_catalog

Expected:

{"repositories":[]}

The registry is empty, without login and without TLS. That is fine in the lab; something like this never belongs outside.

Save your progress? Sign up for free

Drill 6: An artifact of your own

Now you package the notes-api manifests as an OCI artifact. As the tag you use the short commit hash, and you write the source and revision into it as proof of provenance:

flux push artifact oci://localhost:30500/notes:$(git rev-parse --short HEAD) \
  --path=./apps/notes \
  --source="$(git config --get remote.origin.url)" \
  --revision="main@sha1:$(git rev-parse HEAD)" \
  --insecure-registry

Expected (the digest differs):

► pushing artifact to localhost:30500/notes:610b75c
✔ artifact successfully pushed to localhost:30500/notes@sha256:cd594d31346d754f9011c56a305ebf3420d1ccbd704e8c67421314bef9cfc4a2

--insecure-registry allows HTTP instead of HTTPS. You attach a tag like stable afterwards. That is the moment an artifact is “released”:

flux tag artifact oci://localhost:30500/notes:$(git rev-parse --short HEAD) --tag=stable
flux list artifacts oci://localhost:30500/notes

Expected (shortened):

► tagging artifact
✔ artifact tagged as localhost:30500/notes:stable
ARTIFACT                        DIGEST          SOURCE                                          REVISION
localhost:30500/notes:stable    sha256:cd594d31…    https://github.com/<du>/flux-fleet.git  main@sha1:610b75c8…
localhost:30500/notes:610b75c   sha256:cd594d31…    https://github.com/<du>/flux-fleet.git  main@sha1:610b75c8…

Both tags point to the same digest, that is, to the same package.

Save your progress? Sign up for free

Checkpoint

flux get sources all

Expected: ocirepository/podinfo with 6.17.0@sha256:… and gitrepository/flux-system, both READY True.

flux get kustomizations

Expected: flux-system, notes, podinfo and registry, all READY True.

curl -s localhost:30500/v2/notes/tags/list

Expected (your hash differs):

{"name":"notes","tags":["610b75c","stable"]}

Challenge

Switch the Kustomization notes to your OCI artifact with the tag stable. Check that flux get kustomization notes shows the OCI revision, and then switch back to Git.

Hint: Inside the cluster, the registry is not called localhost, but like every Service: <service>.<namespace>.svc.cluster.local. And the source-controller, too, has to know that there is no HTTPS there.

Solution

flux create source oci notes \
  --url=oci://registry.registry.svc.cluster.local:5000/notes \
  --tag=stable --insecure --interval=1m --export > clusters/lab/notes-oci.yaml
sed -i -e 's/kind: GitRepository/kind: OCIRepository/' \
  -e 's/name: flux-system/name: notes/' \
  -e 's#path: ./apps/notes#path: ./#' clusters/lab/notes.yaml
git add -A
git commit -m "notes aus OCI"
git push
flux reconcile kustomization flux-system --with-source
flux get kustomization notes

Expected (shortened):

NAME    REVISION                SUSPENDED   READY   MESSAGE
notes   stable@sha256:cd594d31  False       True    Applied revision: stable@sha256:cd594d31

notes-api keeps running without interruption, because the artifact contains the same manifests. Back to Git:

git revert --no-edit HEAD
git push
flux reconcile kustomization flux-system --with-source
flux get kustomization notes

Expected (shortened):

NAME    REVISION            SUSPENDED   READY   MESSAGE
notes   main@sha1:546006db  False       True    Applied revision: main@sha1:546006db

Save your progress? Sign up for free

Enterprise Note

  • Git → CI → OCI → Flux: many teams no longer let Flux read directly from Git. The CI checks the manifests, pushes them as a signed OCI artifact to the registry, and Flux fetches only released tags. The cluster then no longer needs access to Git, and verify in the OCIRepository checks the signature with cosign (see “Secure supply chains hands-on”).
  • Credentials: for private registries, the OCIRepository gets a secretRef or, in the cloud, provider: aws|azure|gcp with workload identity, without any stored password.
  • Pinning third-party repos: semver: 6.17.x fetches patches automatically. Anyone who wants to see every change pins an exact tag or even the digest and lets updates arrive as pull requests, for example with Renovate.