Day 0: The Training Ground
Goal: Two lab VMs, k3s, the
argocdCLI and Helm, a fleet repo with a deploy key, and notes-api deployed once by hand in the cluster, so you see which problem Argo CD solves. · Duration: approx. 45 min · Prerequisite: an account on github.com
Briefing
- GitOps means: the desired state of the cluster lives in a Git repo. A program in the cluster continuously compares the repo with the cluster and reconciles. If you want to change something, you change Git, not the cluster.
- Argo CD is such a program. It runs in the
argocdnamespace, has a web UI, a CLI, and its own Kubernetes objects such asApplication. - In this book the repo is called the fleet repo
(
argocd-fleet). It is private in your GitHub account. Argo CD reads it through a deploy key, an SSH key that is valid for this one repo only.
Laptop ──ssh──► VM argo VM prod (from Day 11)
└─ k3s └─ k3s
├─ argocd: Argo CD (from Day 1) ───► manages prod
└─ notes: notes-api (by hand today, from Git from Day 2)
▲
github.com/<du>/argocd-fleet ── Argo CD fetches via SSH ──┘
curl ─► http://<argo-IP>:30080 notes-api
Drill 1: The Lab VMs
You need two Linux VMs with Ubuntu 24.04 on the same network:
| VM | vCPU | RAM | Disk | from |
|---|---|---|---|---|
argo |
4 | 8 GB | 40 GB | Day 0 |
prod |
2 | 4 GB | 30 GB | Day 11 |
On your laptop you can do this, for example, with
Multipass from multipass.run:
multipass launch 24.04 --name argo --cpus 4 --memory 8G --disk 40G
multipass launch 24.04 --name prod --cpus 2 --memory 4G --disk 30G
multipass info argo | grep IPv4
- Any other Linux VM with
sudoand internet access works too. The VMs must reach github.com; GitHub does not need to reach them. - You only need
prodon Day 11. Until then it can stay off.
From here on, everything runs in the VM
argo. In the book it has the address
172.16.0.182, and prod has
172.16.0.123. Use your own addresses.
Save your progress? Sign up for free
Drill 2: k3s
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.37.2+k3s1 \
sh -s - --disable traefik --disable servicelbExpected (abridged):
[INFO] Using v1.37.2+k3s1 as release
…
[INFO] systemd: Enabling k3s unit
[INFO] systemd: Starting k3s
get.k3s.iois the official installation script. It downloads k3s, creates the service, and linkskubectl.--disable traefik --disable servicelb: you do not need an ingress controller or a load balancer; everything runs through NodePorts.
The credentials belong to root. You make yourself a
copy:
mkdir -p ~/.kube
sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
sudo chown "$(id -u):$(id -g)" ~/.kube/config
chmod 600 ~/.kube/config
echo 'export KUBECONFIG=~/.kube/config' >> ~/.bashrc
export KUBECONFIG=~/.kube/config
kubectl get nodesExpected:
NAME STATUS ROLES AGE VERSION
argo Ready control-plane 21s v1.37.2+k3s1
KUBECONFIG is mandatory with k3s. Without the variable,
its kubectl always reads
/etc/rancher/k3s/k3s.yaml and fails with
permission denied. The argocd CLI reads the
same variable.
Save your progress? Sign up for free
Drill 3: argocd CLI and Helm
Both programs come as ready-made binaries. The argocd
CLI is a single file on GitHub:
mkdir -p ~/.local/bin ~/argo-lab
cd ~/argo-lab
curl -fsSL -o argocd \
https://github.com/argoproj/argo-cd/releases/download/v3.5.7/argocd-linux-amd64
install -m 755 argocd ~/.local/bin/argocd
rm argocdYou need Helm on Day 1 for the installation and on Day 5 for your own charts:
curl -fsSLO https://get.helm.sh/helm-v4.3.2-linux-amd64.tar.gz
tar xzf helm-v4.3.2-linux-amd64.tar.gz
mv linux-amd64/helm ~/.local/bin/
rm -r linux-amd64 helm-v4.3.2-linux-amd64.tar.gzUbuntu adds ~/.local/bin to the PATH at
login as soon as the folder exists. Log out once and log back in:
exitssh [email protected]
argocd version --client --shortExpected:
argocd: v3.5.7+6d73296
helm version --shortExpected:
v4.3.2+g8346aaa
Save your progress? Sign up for free
Drill 4: Fleet Repo and Deploy Key
Create a private repo in the browser: github.com →
New repository → name argocd-fleet,
Private, without a README. With the GitHub CLI on your
laptop you can also use
gh repo create argocd-fleet --private.
In the VM, you generate a separate key just for this repo:
ssh-keygen -q -t ed25519 -N "" -C argocd-fleet -f ~/.ssh/argocd-fleet
cat ~/.ssh/argocd-fleet.pubExpected (your key will differ):
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFaWVjfNDS1nnR5RjlbUFtrY8/U1V30V864SgtlgfuyJ argocd-fleet
You add the public part to the repo: Settings →
Deploy keys → Add deploy key, title
argo-lab, paste the key, check Allow write
access. The key may write because you push from the VM and Argo
CD makes commits itself on Day 12.
git should use exactly this key for github.com:
cat > ~/.ssh/config <<'EOF'
Host github.com
IdentityFile ~/.ssh/argocd-fleet
EOF
chmod 600 ~/.ssh/config
ssh-keyscan -t ed25519 github.com >> ~/.ssh/known_hosts
ssh -T [email protected]Expected (abridged):
# github.com:22 SSH-2.0-11529790
Hi <du>/argocd-fleet! You've successfully authenticated, but GitHub does not provide shell access.
GitHub shows the repo name instead of your account. This tells you
that a deploy key is in effect and not a personal key.
ssh-keyscan fetches GitHub’s host key. If needed, compare
it with the fingerprints at docs.github.com → “GitHub’s SSH
key fingerprints”.
Now you clone the empty repo and configure git:
cd ~/argo-lab
git clone [email protected]:<du>/argocd-fleet.git
git config --global user.name "Dein Name"
git config --global user.email "[email protected]"
git config --global init.defaultBranch mainExpected:
Cloning into 'argocd-fleet'...
warning: You appear to have cloned an empty repository.
Save your progress? Sign up for free
Drill 5: The Companion Code
git clone -q https://github.com/thomaszachmann/argocd-zum-anfassen.git ~/argo-lab/code
ls ~/argo-lab/codeExpected:
LICENSE README.md loesung wettkampf
loesung/tagNNcontains the state of your fleet repo at the end of each day. You write everything yourself and compare withdiffat the end.- You only need
wettkampf/on Day 15.
You do not need to build an image. notes-api is ready in the GitHub
Container Registry: ghcr.io/thomaszachmann/notes-api, among
others in versions 0.5.1 to 0.7.1.
Save your progress? Sign up for free
Drill 6: notes-api by Hand
Before Argo CD comes into play, you bring notes-api into the cluster
the way it is usually done without GitOps: write a manifest,
kubectl apply. Without a database, notes-api keeps the
notes in memory, which is enough for today. Postgres comes on Day 4.
mkdir -p ~/argo-lab/hand
cd ~/argo-lab/hand
cat > notes.yaml <<'EOF'
apiVersion: v1
kind: Namespace
metadata:
name: notes
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: notes
namespace: notes
spec:
replicas: 1
selector:
matchLabels:
app: notes
template:
metadata:
labels:
app: notes
spec:
containers:
- name: api
image: ghcr.io/thomaszachmann/notes-api:0.5.1
ports:
- containerPort: 8080
env:
- name: NOTES_API_KEY
value: lab-api-key-123
readinessProbe:
httpGet:
path: /healthz
port: 8080
---
apiVersion: v1
kind: Service
metadata:
name: notes
namespace: notes
spec:
type: NodePort
selector:
app: notes
ports:
- port: 80
targetPort: 8080
nodePort: 30080
EOF
kubectl apply -f notes.yaml
kubectl -n notes rollout status deploy/notesExpected:
namespace/notes created
deployment.apps/notes created
service/notes created
Waiting for deployment "notes" rollout to finish: 0 of 1 updated replicas are available...
deployment "notes" successfully rolled out
curl -s localhost:30080/versionExpected:
0.5.1
If curl does not answer, k3s is still setting up the
NodePort. Wait a few seconds and try again.
Save your progress? Sign up for free
Drill 7: The Drift
A colleague is in a hurry and bumps notes-api to a new version directly in the cluster:
kubectl -n notes set image deploy/notes api=ghcr.io/thomaszachmann/notes-api:0.7.1
kubectl -n notes rollout status deploy/notesExpected (abridged):
deployment.apps/notes image updated
…
deployment "notes" successfully rolled out
curl -s localhost:30080/versionExpected:
0.7.1
Your file still says 0.5.1. The cluster and the file
have diverged, and nobody notices. It only becomes visible when someone
checks:
kubectl diff -f notes.yamlExpected (abridged):
- image: ghcr.io/thomaszachmann/notes-api:0.7.1
+ image: ghcr.io/thomaszachmann/notes-api:0.5.1
This deviation is called drift. Without GitOps, it depends on discipline whether anyone notices it and reverts it. From Day 3 on, Argo CD does that on its own.
Save your progress? Sign up for free
Checkpoint
kubectl get nodesExpected: one node argo with STATUS
Ready.
argocd version --client --shortExpected: argocd: v3.5.7+6d73296.
git -C ~/argo-lab/argocd-fleet remote -vExpected: [email protected]:<du>/argocd-fleet.git
twice, once with (fetch), once with
(push).
Challenge
Revert the drift by hand and check that kubectl diff is
silent afterwards. Then clean up the notes namespace,
because from Day 2 on Argo CD creates it.
Hint: kubectl apply enforces the state
of the file. kubectl diff exits without output when nothing
differs.
Solution
kubectl apply -f notes.yaml
kubectl -n notes rollout status deploy/notes
kubectl diff -f notes.yamlExpected (abridged):
namespace/notes unchanged
deployment.apps/notes configured
service/notes unchanged
…
deployment "notes" successfully rolled out
After that, kubectl diff no longer prints anything.
Clean up:
kubectl delete -f notes.yamlExpected:
namespace "notes" deleted
deployment.apps "notes" deleted from notes namespace
service "notes" deleted from notes namespace
Save your progress? Sign up for free
Enterprise Note
- In company clusters, a platform team installs Argo CD. Teams get
projects and permissions, not
cluster-admin(Day 9). - Deploy keys are valid for exactly one repo. For many repos, you use a GitHub App or a machine account with a read token, never a person’s key.
- With GitOps, running
kubectldirectly against the production cluster is the exception, for example in an emergency. Many teams give humans only read access there.