Day 11: Deployment to Kubernetes
Goal: k3s on the lab VM, notes-api as a deployment, and a workflow that rolls out a chosen version after approval, waits for the rollout and rolls back in case of failure. · Duration: approx. 90 min · Prerequisite: Day 10
Briefing
- The cluster runs on your VM and is not reachable from outside. GitHub’s runners can’t get to it – your own runner from Day 5 can.
- The workflow takes the version as an input, runs in the
produktionenvironment and callskubectl. GitHub records every deployment together with its result. kubectl rollout statuswaits until the new pods are ready and fails after a deadline – then the next step rolls back.
gh workflow run deploy.yml -f version=0.5.1
─► approval ─► runner gha-vm ─► kubectl set image ─► rollout status
└── failure ─► rollout undo
k3s on the VM: namespace notes ─ deployment notes-api ─ service :30081
Drill 1: k3s
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.37.1+k3s1 \
sh -s - --disable traefik --disable servicelbExpected (shortened):
[INFO] Using v1.37.1+k3s1 as release
...
[INFO] systemd: Starting k3s
get.k3s.iois the official installation script. It downloads k3s, creates the service and linkskubectl.--disable traefik --disable servicelb: no ingress and no load balancer that grab ports 80 and 443 of the VM. A NodePort is enough for the lab.
The credentials are in /etc/rancher/k3s/k3s.yaml and
belong to root. A copy for you:
mkdir -p ~/.kube
sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
sudo chown "$(id -u):$(id -g)" ~/.kube/config
chmod 600 ~/.kube/config
echo 'export KUBECONFIG=~/.kube/config' >> ~/.bashrc
export KUBECONFIG=~/.kube/config
kubectl get nodesExpected:
NAME STATUS ROLES AGE VERSION
gha Ready control-plane 6m7s v1.37.1+k3s1
KUBECONFIG is mandatory here: otherwise the k3s
kubectl always reads /etc/rancher/k3s/k3s.yaml
and fails with permission denied. The line in
~/.bashrc sets the variable on every new login.
Save your progress? Sign up for free
Drill 2: notes-api in the cluster
The manifests belong in the repository, next to the code:
cd ~/gha-lab/notes-actions
mkdir -p k8s
cat > k8s/notes-api.yaml <<'EOF'
apiVersion: v1
kind: Namespace
metadata:
name: notes
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: notes-api
namespace: notes
spec:
replicas: 2
selector:
matchLabels: {app: notes-api}
template:
metadata:
labels: {app: notes-api}
spec:
containers:
- name: notes-api
image: ghcr.io/OWNER/notes-api:0.5.0
ports: [{containerPort: 8080}]
env:
- name: NOTES_API_KEY
valueFrom:
secretKeyRef: {name: notes-api, key: api-key}
readinessProbe:
httpGet: {path: /healthz, port: 8080}
periodSeconds: 2
---
apiVersion: v1
kind: Service
metadata:
name: notes-api
namespace: notes
spec:
type: NodePort
selector: {app: notes-api}
ports: [{port: 8080, nodePort: 30081}]
EOF
sed -i "s/OWNER/$(gh api user --jq .login)/" k8s/notes-api.yaml
grep image: k8s/notes-api.yamlExpected:
image: ghcr.io/<du>/notes-api:0.5.0
readinessProbe: a pod only gets traffic once/healthzresponds. That is whatrollout statuswaits for later.{app: notes-api}is YAML in short form – the same as an indented lineapp: notes-api.
The secret is created by hand, once – it does not belong in the repository. First the namespace and the secret, then the rest:
kubectl create namespace notes
kubectl -n notes create secret generic notes-api --from-literal=api-key=lab-api-key-123
kubectl apply -f k8s/notes-api.yaml
kubectl -n notes rollout status deployment/notes-api --timeout=180s
curl -s localhost:30081/versionExpected:
namespace/notes created
secret/notes-api created
namespace/notes unchanged
deployment.apps/notes-api created
service/notes-api created
deployment "notes-api" successfully rolled out
0.5.0
- The other way round, the pods would first come up without the secret
(
CreateContainerConfigError) and only recover after a back-off pause –rollout statuscould then easily run into the deadline. - The image is public (Day 6); k3s needs no pull secret.
git add k8s
git commit -qm "k8s: Manifeste für notes-api"
git push -qSave your progress? Sign up for free
Drill 3: The deploy workflow
The runner runs as a service and does not read
~/.bashrc. Environment variables for all of its jobs go
into the file .env in the runner folder:
echo "KUBECONFIG=$HOME/.kube/config" >> ~/gha-lab/runner/.env
cd ~/gha-lab/runner
sudo ./svc.sh stop
sudo ./svc.sh start
cd ~/gha-lab/notes-actionsExpected (shortened, at the end):
Active: active (running) since ...
The runner reads .env only at startup – hence the
restart.
deploy.yml gets an input and real work:
cat > .github/workflows/deploy.yml <<'EOF'
name: deploy
on:
workflow_dispatch:
inputs:
version:
description: Image-Tag, z. B. 0.5.1
required: true
permissions:
contents: read
concurrency:
group: deploy-produktion
cancel-in-progress: false
jobs:
deploy:
runs-on: [self-hosted, lab]
environment:
name: produktion
url: http://localhost:30081/version
env:
IMAGE: ghcr.io/${{ github.repository_owner }}/notes-api:${{ inputs.version }}
steps:
- name: Version setzen
run: kubectl -n notes set image deployment/notes-api notes-api="$IMAGE"
- name: Rollout abwarten
run: kubectl -n notes rollout status deployment/notes-api --timeout=60s
- name: Zurückrollen
if: failure()
run: kubectl -n notes rollout undo deployment/notes-api
EOF
git commit -qam "deploy: Version auf k3s ausrollen"
git push -qruns-on: [self-hosted, lab]: only your runner can reach the cluster. ViaKUBECONFIGfrom.env, it uses the file~/.kube/configof the userubuntu.concurrencywithcancel-in-progress: false: two deployments run one after the other, neither cancels the other – nobody wants half-finished rollouts.environment.urllater appears next to the deployment in GitHub.- No checkout:
set imageonly changes the image of the existing deployment. You apply the manifests themselves by hand when they change. In production you would rather write the version into the file and let the cluster fetch it (GitOps) – the sister book on Argo CD shows that.
gh workflow run deploy.yml -f version=0.5.1Approve with the ID from
gh run list --workflow deploy.yml --limit 1 – as on Day 10,
with the ID of your environment:
gh api -X POST repos/{owner}/{repo}/actions/runs/<ID>/pending_deployments \
-F 'environment_ids[]=26123220715' -f state=approved -f comment='0.5.1' \
--jq '.[].environment'As soon as the run is finished:
curl -s localhost:30081/version
kubectl -n notes get deployment notes-api -o wideExpected:
0.5.1
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES ...
notes-api 2/2 2 2 2m59s notes-api ghcr.io/<du>/...
-o wide additionally shows containers and images – on
the far right is ghcr.io/<du>/notes-api:0.5.1.
Save your progress? Sign up for free
Drill 4: A rollout that fails
A version that doesn’t exist:
gh workflow run deploy.yml -f version=9.9.9Approve as before. After a little over a minute:
gh run view <ID> -v | grep -E 'Rollout|Zurück'Expected:
X Rollout abwarten
✓ Zurückrollen
gh run view <ID> --log | grep -E 'Z (error|deployment.apps)'
curl -s localhost:30081/versionExpected (shortened):
deploy Version setzen ... deployment.apps/notes-api image updated
deploy Rollout abwarten ... error: timed out waiting for the condition
deploy Zurückrollen ... deployment.apps/notes-api rolled back
0.5.1
- The image
9.9.9does not exist; the new pod stays inImagePullBackOff, androllout statusgives up after 60 seconds. - The old pods kept running the whole time – a rolling update only
replaces them when new ones are ready.
rollout undoclears away the broken attempt.
GitHub has recorded both deployments:
First the deployments:
gh api repos/{owner}/{repo}/deployments --jq '.[0:2][] | "\(.id) \(.environment)"'Expected (IDs differ):
7517792135 produktion
7517780729 produktion
Then the status of one of them:
gh api repos/{owner}/{repo}/deployments/7517792135/statuses --jq '.[0].state'Expected:
failure
In the browser, Deployments shows the same list with the
url from the workflow.
Save your progress? Sign up for free
Checkpoint
kubectl -n notes get podsExpected:
NAME READY STATUS RESTARTS AGE
notes-api-55dd946cfc-h67wt 1/1 Running 0 105s
notes-api-55dd946cfc-xm2s9 1/1 Running 0 109s
curl -s localhost:30081/versionExpected:
0.5.1
gh run list --workflow deploy.yml --limit 2 --json conclusion --jq '.[].conclusion'Expected:
failure
success
Challenge
After every release, the deployment should start on its own (the
approval stays). Add a job to release.yml that triggers
deploy.yml with the new version, and test it with
v0.5.2.
Hint: A release that a workflow creates with the
GITHUB_TOKEN triggers no other workflows –
protection against infinite loops. The exception:
workflow_dispatch. gh workflow run needs
actions: write.
Solution
cat >> .github/workflows/release.yml <<'EOF'
deploy:
needs: release
if: ${{ !contains(github.ref_name, '-') }}
runs-on: ubuntu-24.04
permissions:
actions: write
steps:
- name: Deployment anstoßen
env:
GH_TOKEN: ${{ github.token }}
run: >-
gh workflow run deploy.yml --repo "$GITHUB_REPOSITORY"
-f version="${GITHUB_REF_NAME#v}"
EOF
git commit -qam "release: Deployment anstoßen"
git push -q
git tag v0.5.2
git push -q origin v0.5.2!contains(…, '-'): pre-releases such asv0.6.0-rc.2don’t go to production.- This job needs no checkout – only
ghand the token.
After about two minutes (release and image must be finished):
gh run list --workflow deploy.yml --limit 1 --json event,status --jq '.[0]'Expected:
{"event":"workflow_dispatch","status":"waiting"}
Approve as in Drill 3. Then:
curl -s localhost:30081/versionExpected:
0.5.2
Watch out, race condition: deploy only waits for
release, not for image.yml – both run in
parallel. Here the approval saves you, since you only give it once the
image is there. Without approval, waiting for the image belongs in the
deploy job.
Save your progress? Sign up for free
Enterprise Note
- Push or pull: Here CI pushes into the cluster (push). With GitOps (Argo CD, Flux), CI only writes the new version into Git, and the cluster fetches it – no runner needs access to the cluster.
- Access to the cluster: Instead of a kubeconfig with
admin rights, the runner gets a ServiceAccount that may only change
deployments in the
notesnamespace. With cloud clusters, it obtains access via OIDC (Day 10). - Approval before production: Many teams deploy to
stagingwithout approval and toproduktionwith it – two environments, the same workflow. - Verify signatures: An admission controller in the cluster can require that only images with a signature and provenance from your repository start (Day 8).