Day 0: The Training Ground
Goal: A lab VM with k3s, the Flux CLI and Helm, a GitHub token and notes-api deployed once by hand in the cluster, so you see which problem Flux solves. · Duration: approx. 45 min · Prerequisite: an account on github.com
Briefing
- GitOps means: the desired state of the cluster lives in a Git repo. A program in the cluster reads the repo and continuously reconciles the cluster with it. Whoever wants to change something changes Git, not the cluster.
- Flux is such a program. It consists of several
small controllers that run in the
flux-systemnamespace. It is controlled through its own Kubernetes objects such asGitRepositoryorKustomization. - The Flux CLI
fluxinstalls Flux, creates these objects and shows their state. It is not required; everything also works withkubectl. But it saves a lot of typing. - In this book the repo is called the fleet repo
(
flux-fleet). It is private in your GitHub account.
Laptop ──ssh──► VM flux
└─ k3s
├─ flux-system: Flux controllers (from Day 1)
└─ notes: notes-api (by hand today, from Git from Day 2)
▲
github.com/<du>/flux-fleet ── Flux pulls ──┘
curl ─► http://<VM-IP>:30080 notes-api
Drill 1: The lab VM
On the laptop (Multipass from
multipass.run):
multipass launch 24.04 --name flux --cpus 2 --memory 4G --disk 30G
multipass exec flux -- bash -c "echo '$(cat ~/.ssh/id_ed25519.pub)' \
>> ~/.ssh/authorized_keys"
multipass info flux | grep IPv4
ssh ubuntu@<IP aus der Ausgabe>
- 4 GB are enough for Flux, notes-api, a small registry and later kro. The Flux controllers together need about 300 MB.
- Any other Linux VM with
sudoand internet access will do as well. The VM must reach github.com; GitHub does not need to reach the VM.
From here on, everything runs in the VM.
Save your progress? Sign up for free
Drill 2: k3s
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.37.2+k3s1 \
sh -s - --disable traefik --disable servicelbExpected (shortened):
[INFO] Using v1.37.2+k3s1 as release
...
[INFO] systemd: Enabling k3s unit
[INFO] systemd: Starting k3s
get.k3s.iois the official installation script. It downloads k3s, creates the service and linkskubectl.--disable traefik --disable servicelb: you do not need an ingress controller or a load balancer; everything runs through NodePorts.
The credentials belong to root. You make yourself a
copy:
mkdir -p ~/.kube
sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
sudo chown "$(id -u):$(id -g)" ~/.kube/config
chmod 600 ~/.kube/config
echo 'export KUBECONFIG=~/.kube/config' >> ~/.bashrc
export KUBECONFIG=~/.kube/config
kubectl get nodesExpected:
NAME STATUS ROLES AGE VERSION
flux Ready control-plane 20s v1.37.2+k3s1
KUBECONFIG is mandatory with k3s. Without the variable,
its kubectl always reads
/etc/rancher/k3s/k3s.yaml and fails with
permission denied. The Flux CLI reads the same
variable.
Save your progress? Sign up for free
Drill 3: Flux CLI and Helm
Both programs come as ready-made binaries from GitHub:
mkdir -p ~/.local/bin ~/flux-lab
cd ~/flux-lab
curl -fsSLO https://github.com/fluxcd/flux2/releases/download/v2.10.0/flux_2.10.0_linux_amd64.tar.gz
tar xzf flux_2.10.0_linux_amd64.tar.gz -C ~/.local/bin flux
rm flux_2.10.0_linux_amd64.tar.gzYou need Helm from Day 6 on to package a chart:
curl -fsSLO https://get.helm.sh/helm-v4.3.1-linux-amd64.tar.gz
tar xzf helm-v4.3.1-linux-amd64.tar.gz
mv linux-amd64/helm ~/.local/bin/
rm -r linux-amd64 helm-v4.3.1-linux-amd64.tar.gzUbuntu adds ~/.local/bin to the PATH at
login as soon as the folder exists. If you log out once and back in, the
shell finds both programs:
exitssh ubuntu@<VM-IP>
flux version --clientExpected:
flux: v2.10.0
helm version --shortExpected:
v4.3.1+gc0d08d0
The Flux CLI completes commands with the Tab key once you enable it:
echo 'source <(flux completion bash)' >> ~/.bashrc
source ~/.bashrcNow flux checks whether your cluster fits:
flux check --preExpected:
► checking prerequisites
✔ Kubernetes 1.37.2+k3s1 >=1.35.0-0
✔ prerequisites checks passed
Flux 2.10 supports the three newest Kubernetes versions, that is 1.35 to 1.37.
Save your progress? Sign up for free
Drill 4: The GitHub token
Tomorrow Flux creates a repo in your account and writes to it. For that it needs a token. You create it in the browser:
- github.com → profile picture → Settings → Developer settings → Personal access tokens → Fine-grained tokens → Generate new token.
- Name
flux-lab, expiration 30 days. - Repository access: All repositories. The repo does not exist yet, after all.
- Permissions → Repository permissions:
- Administration: Read and write (create the repo, add the deploy key)
- Contents: Read and write (commit files)
- Commit statuses: Read and write (from Day 11)
- Workflows: Read and write (for the CI on Day 13)
- Generate token and copy the token. GitHub shows it only once.
In the VM you put it into a file that only you may read.
read -s shows nothing while you paste, and the token does
not end up in the shell history:
read -s T && echo "$T" > ~/.github-token && chmod 600 ~/.github-tokenPaste the token, press Enter. Then your username and the token belong
in environment variables, because that is exactly where the Flux CLI
looks for them. Replace <du> with your GitHub
username:
echo 'export GITHUB_USER=<du>' >> ~/.bashrc
echo 'export GITHUB_TOKEN=$(cat ~/.github-token)' >> ~/.bashrc
source ~/.bashrc
curl -s -H "Authorization: Bearer $GITHUB_TOKEN" https://api.github.com/user \
| grep '"login"'Expected (with your name):
"login": "<du>",
If you see "message": "Bad credentials" instead, the
token was copied incorrectly. If the token lacks a permission, you only
notice it during the bootstrap. Day 1 shows the messages for that.
Save your progress? Sign up for free
Drill 5: The companion code
git clone -q https://github.com/thomaszachmann/flux-zum-anfassen.git ~/flux-lab/code
ls ~/flux-lab/code ~/flux-lab/code/loesungExpected:
/home/ubuntu/flux-lab/code:
LICENSE loesung README.md wettkampf
/home/ubuntu/flux-lab/code/loesung:
pruefung tag03 tag05 tag07 tag09 tag11 tag13
tag02 tag04 tag06 tag08 tag10 tag12 tag14
loesung/tagNNcontains the state of your fleet repo at the end of each day. You write everything yourself and compare withdiffat the end.- You only need
wettkampf/on Day 15.
You do not have to build an image. notes-api is ready in the GitHub
Container Registry: ghcr.io/thomaszachmann/notes-api, in
versions 0.5 to 0.7.
Save your progress? Sign up for free
Drill 6: notes-api by hand
Before Flux comes into play, you bring notes-api into the cluster the
way it is usual without GitOps: write manifests,
kubectl apply. The four files will later live in the fleet
repo, today in ~/flux-lab/notes.
mkdir -p ~/flux-lab/notes
cd ~/flux-lab/notes
cat > namespace.yaml <<'EOF'
apiVersion: v1
kind: Namespace
metadata:
name: notes
EOF
cat > secret.yaml <<'EOF'
apiVersion: v1
kind: Secret
metadata:
name: notes-api
namespace: notes
stringData:
apiKey: lab-api-key-123
EOFThe API key is still in plain text here. With a Secret in a Git repo it cannot stay that way; Day 8 cleans it up.
cat > deployment.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
name: notes
namespace: notes
spec:
replicas: 1
selector:
matchLabels:
app: notes
template:
metadata:
labels:
app: notes
spec:
containers:
- name: api
image: ghcr.io/thomaszachmann/notes-api:0.5.1
ports:
- containerPort: 8080
env:
- name: NOTES_API_KEY
valueFrom:
secretKeyRef:
name: notes-api
key: apiKey
readinessProbe:
httpGet:
path: /healthz
port: 8080
EOF
cat > service.yaml <<'EOF'
apiVersion: v1
kind: Service
metadata:
name: notes
namespace: notes
spec:
type: NodePort
selector:
app: notes
ports:
- port: 80
targetPort: 8080
nodePort: 30080
EOFA kustomization.yaml lists the files.
kubectl understands it with -k, and Flux works
the same way from tomorrow on:
cat > kustomization.yaml <<'EOF'
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- namespace.yaml
- secret.yaml
- deployment.yaml
- service.yaml
EOF
cd ~/flux-lab
kubectl apply -k notes
kubectl -n notes rollout status deploy/notesExpected:
namespace/notes created
secret/notes-api created
service/notes created
deployment.apps/notes created
Waiting for deployment "notes" rollout to finish: 0 of 1 updated replicas are available...
deployment "notes" successfully rolled out
curl -s localhost:30080/versionExpected:
0.5.1
If curl reports nothing (exit code 7), k3s is still
setting up the NodePort. Wait a few seconds and try again.
Save your progress? Sign up for free
Drill 7: The drift
A colleague is in a hurry and moves notes-api to a new version directly in the cluster:
kubectl -n notes set image deploy/notes api=ghcr.io/thomaszachmann/notes-api:0.7.1
kubectl -n notes rollout status deploy/notes
curl -s localhost:30080/versionExpected (shortened):
deployment.apps/notes image updated
...
deployment "notes" successfully rolled out
0.7.1
Your files still say 0.5.1. The cluster and the files
have diverged, and nobody notices. It only becomes visible when someone
looks:
kubectl diff -k notesExpected (shortened):
- image: ghcr.io/thomaszachmann/notes-api:0.7.1
+ image: ghcr.io/thomaszachmann/notes-api:0.5.1
This divergence is called drift. Without GitOps, it depends on discipline whether someone notices it and rolls it back. From Day 2 on, Flux does that on its own every few minutes.
Save your progress? Sign up for free
Checkpoint
kubectl get nodesExpected: one node flux with STATUS
Ready.
flux check --preExpected: ✔ prerequisites checks passed.
echo "$GITHUB_USER ${#GITHUB_TOKEN}"Expected: your GitHub name and a number greater than zero, the length of the token. That way the token itself stays invisible.
Challenge
Roll back the drift by hand and check that kubectl diff
is silent afterwards.
Hint: kubectl diff exits with exit code
0 when there is nothing to show, and with 1 when it finds
differences.
Solution
kubectl apply -k notes
kubectl -n notes rollout status deploy/notes
kubectl diff -k notes; echo "Exit-Code $?"Expected (shortened):
deployment.apps/notes configured
...
Exit-Code 0
You no longer need the namespace. Tomorrow Flux creates it again:
kubectl delete -k notesSave your progress? Sign up for free
Enterprise Note
- Push or pull? A CI pipeline that runs
kubectl applyneeds write access to the cluster and only runs when someone pushes. Flux, on the other hand, pulls from the inside: the cluster only needs read access to the repo, and no machine outside holds cluster credentials. - Token or GitHub App? A personal token is tied to a
person. If they leave the company, Flux stops. In operations you use a
token of a technical account or a GitHub App, which
flux bootstrap githubalso supports. - Flux or Argo CD? Both implement GitOps. Flux gets by without its own UI and is controlled entirely through Kubernetes objects. Argo CD brings a web UI and its own permission model.