Skip to content
Hands-on IT IT Drills

Day 14: Kong Ingress Controller

Goal: Run Kong as an ingress controller in k3s, publish notes through the Gateway API, secure it with Kong resources, upgrade without downtime and take a look at Enterprise. · Duration: approx. 90 min · Prerequisite: Day 13

Briefing

  • The Kong Ingress Controller (KIC) reads Kubernetes resources and writes from them the configuration of a Kong without a database: a kong.yaml that emerges from the cluster.
  • You describe routes with the Gateway API (GatewayClass, Gateway, HTTPRoute), Kong-specific things with KongPlugin, KongConsumer and Secrets.
  • The Helm chart kong/ingress installs two Deployments: kong-controller (KIC) and kong-gateway (the proxy).
  • An upgrade is a helm upgrade with a new image tag, pod by pod.
 HTTPRoute, KongPlugin, KongConsumer ──► kong-controller ──Admin API──► kong-gateway
                                                                         :80 (ServiceLB)
 curl http://<IP of k3s>/api/v1/notes ──────────────────────────────────►┘ ─► notes-api

Drill 1: k3s on a VM

On the laptop:

multipass launch 24.04 --name k3s --cpus 2 --memory 3G --disk 20G
multipass shell k3s

In the VM k3s. Traefik stays off; Kong takes over its role:

curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.36.5+k3s1 sh -s - \
  --disable traefik --write-kubeconfig-mode 644
mkdir -p ~/.kube && cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
kubectl wait --for=condition=Ready node --all --timeout=120s

Expected (shortened):

[INFO]  systemd: Starting k3s
error: no matching resources found

The error is harmless: right after startup the node has not registered yet, so there is nothing for kubectl wait to wait for. A few seconds later:

sleep 20
kubectl get nodes

Expected:

NAME   STATUS   ROLES           AGE   VERSION
k3s    Ready    control-plane   33s   v1.36.5+k3s1

Save your progress? Sign up for free

Drill 2: Helm, Gateway API and Companion Code

cd /tmp && V=v4.3.1
curl -fsSLO https://get.helm.sh/helm-$V-linux-amd64.tar.gz
curl -fsSLO https://get.helm.sh/helm-$V-linux-amd64.tar.gz.sha256sum
sha256sum -c helm-$V-linux-amd64.tar.gz.sha256sum
tar xzf helm-$V-linux-amd64.tar.gz linux-amd64/helm
sudo install linux-amd64/helm /usr/local/bin/helm && rm -rf linux-amd64 helm-$V*
helm version --short
GW=https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.3.0
kubectl apply -f $GW/standard-install.yaml | tail -1
git clone https://github.com/thomaszachmann/kong-zum-anfassen.git ~/kong-lab/code

Expected (shortened; on Apple Silicon arm64 instead of amd64):

helm-v4.3.1-linux-amd64.tar.gz: OK
v4.3.1+gc0d08d0
customresourcedefinition.apiextensions.k8s.io/referencegrants.gateway.networking.k8s.io
created
Cloning into '/home/ubuntu/kong-lab/code'...

KIC 3.5 builds on Gateway API v1.3.0. Kubernetes does not ship the Gateway API resources; you install them as CRDs.

Save your progress? Sign up for free

Drill 3: The Image Without a Registry

k3s has no Docker. You do not rebuild the notes-api image, you bring it over from the VM kong. There:

docker save -o ~/notes-api.tar notes-api:0.2.0

On the laptop:

multipass transfer kong:/home/ubuntu/notes-api.tar k3s:/home/ubuntu/

In the VM k3s:

sudo k3s ctr images import ~/notes-api.tar
sudo k3s ctr images ls | grep notes-api | cut -c1-40

Expected (shortened):

docker.io/library/notes api:0.2.0           saved
application/vnd.oci.image.index.v1+json sha256:7b0d2646bc6e…
Importing   elapsed: 1.0 s  total:   0.0 B  (0.0 B/s)
docker.io/library/notes-api:0.2.0

containerd expands notes-api:0.2.0 to docker.io/library/notes-api:0.2.0. The Deployment takes the image from there with imagePullPolicy: Never.

Save your progress? Sign up for free

Drill 4: Install the Kong Ingress Controller

cd ~/kong-lab/code/tag14
grep -v '^#' kic-values.yaml
helm repo add kong https://charts.konghq.com
helm install kong kong/ingress --version 0.24.0 -n kong --create-namespace \
  -f kic-values.yaml --wait --timeout 5m | grep -E 'STATUS|REVISION'
kubectl -n kong get pods
kubectl -n kong get svc kong-gateway-proxy

Expected (shortened, your IP and names will differ):

controller:
  ingressController:
    image:
      tag: "3.5.14"
gateway:
  image:
    repository: kong
    tag: "3.9.3"        # bewusst eine Version zurück: Upgrade in Drill 7
  env:
    plugins: bundled
"kong" has been added to your repositories
STATUS: deployed
REVISION: 1
NAME                               READY   STATUS    RESTARTS   AGE
kong-controller-6985bb6dc4-bs5m8   1/1     Running   0          54s
kong-gateway-854656f547-mwlvt      1/1     Running   0          54s
NAME                 TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)
kong-gateway-proxy   LoadBalancer   10.43.101.109   192.168.64.9   80:30980/TCP,443:…

Otherwise the chart sets floating tags (3.9, 3.5); the values file pins them. k3s’s ServiceLB gives the proxy the VM’s IP on port 80.

Save your progress? Sign up for free

Drill 5: Gateway and Routes

cd ~/kong-lab/code/tag14/k8s
kubectl apply -f gateway.yaml
kubectl apply -f notes.yaml | tail -1
kubectl -n notes rollout status deploy/notes-api --timeout=120s
kubectl apply -f routes.yaml
sleep 8
kubectl get gateway -n kong
P=http://localhost
curl -s $P/api/v1/notes; echo
curl -s $P/echo/get | jq -r .url

Expected (shortened):

gatewayclass.gateway.networking.k8s.io/kong created
gateway.gateway.networking.k8s.io/kong created
service/echo created
deployment "notes-api" successfully rolled out
httproute.gateway.networking.k8s.io/notes-lesen created
httproute.gateway.networking.k8s.io/echo created
NAME   CLASS   ADDRESS        PROGRAMMED   AGE
kong   kong    192.168.64.9   True         16s
[]

http://localhost/get

gatewayclass-unmanaged in gateway.yaml tells the KIC: Kong is already running, use the proxy from the Helm chart. The HTTPRoute notes-lesen corresponds to the route from Day 2: GET /api/v1/notes, rewritten to /notes with the URLRewrite filter.

Save your progress? Sign up for free

Drill 6: Writing with a Key

grep -E '^kind:|^plugin:|konghq.com/|name: X-API-Key' schreiben.yaml
kubectl apply -f schreiben.yaml
sleep 10
H='Content-Type: application/json'
curl -s -o /dev/null -w '%{http_code}\n' -X POST -H "$H" -d '{"text":"x"}' \
  $P/api/v1/notes
curl -s -w ' %{http_code}\n' -X POST -H 'apikey: lab-key-anna-456' -H "$H" \
  -d '{"text":"aus Kubernetes"}' $P/api/v1/notes
kubectl -n notes get kongconsumer

Expected (shortened):

kind: KongPlugin
plugin: key-auth
kind: Secret
    konghq.com/credential: key-auth
kind: KongConsumer
kind: HTTPRoute
    konghq.com/plugins: key-auth
              - {name: X-API-Key, value: lab-api-key-123}
kongplugin.configuration.konghq.com/key-auth created
secret/anna-key created
kongconsumer.configuration.konghq.com/anna created
httproute.gateway.networking.k8s.io/notes-schreiben created
401
{"id":"b4b67efb-…","text":"aus Kubernetes"}
 201
NAME   USERNAME   AGE   PROGRAMMED
anna   anna       8s    True

Anna’s key is a Secret with the label konghq.com/credential: key-auth; the consumer refers to it. The annotation konghq.com/plugins attaches key-auth to the route. The backend key is set by the standard filter RequestHeaderModifier.

Warning: An additional KongPlugin of type request-transformer has no effect here. The KIC implements URLRewrite itself as a request-transformer, and only one of these plugins runs per route. Gateway API filters take precedence over handcrafted ones.

Query the list a few times:

for i in 1 2 3 4; do curl -s $P/api/v1/notes; echo; done

Expected (the order varies):

[]

[{"id":"b4b67efb-…","text":"aus Kubernetes"}]

[]

[{"id":"b4b67efb-…","text":"aus Kubernetes"}]

Two notes-api pods, each with its own storage. Without a shared database (“Kubernetes hands-on”, Day 7), each client sees different data depending on the pod.

Save your progress? Sign up for free

Drill 7: Upgrade Without Downtime

Kong 3.9.3 is running; 3.9.4 is the current version of the line. During the upgrade, a loop in the background counts every response:

curl -si $P/echo/get | grep -i '^via'
( ok=0; f=0; ende=$((SECONDS+90))
  while [ $SECONDS -lt $ende ]; do
    c=$(curl -s -o /dev/null -w '%{http_code}' --max-time 2 $P/echo/get)
    [ "$c" = 200 ] && ok=$((ok+1)) || f=$((f+1)); sleep 0.2
  done; echo "ok=$ok fehler=$f" ) &
helm upgrade kong kong/ingress --version 0.24.0 -n kong --reuse-values \
  --set gateway.image.tag=3.9.4 --wait --timeout 5m | grep -E 'STATUS|REVISION'
wait
curl -si $P/echo/get | grep -i '^via'

Expected (the number varies):

Via: 1.1 kong/3.9.3
STATUS: deployed
REVISION: 2
ok=372 fehler=0
Via: 1.1 kong/3.9.4

Not a single error: Kubernetes starts the new pod, waits until it is ready and only then terminates the old one. Without a database there are no migrations; with a database, kong migrations up would come before the switch and kong migrations finish after it.

Save your progress? Sign up for free

Drill 8: From kong.yaml to Kubernetes, and Enterprise

Back on the VM kong. decK translates your configuration into KIC resources:

cd ~/kong-lab/kong
deck file kong2kic -s kong.yaml --class-name kong -o /tmp/kic.yaml
grep '^kind:' /tmp/kic.yaml | sort | uniq -c

Expected:

      8 kind: HTTPRoute
      4 kind: KongClusterPlugin
      4 kind: KongConsumer
     16 kind: KongPlugin
      2 kind: KongUpstreamPolicy
     10 kind: Secret
      6 kind: Service

A good start for a migration, but not a finished result: you still have to check and adjust the namespaces, the backend services and the Secrets.

And the Enterprise edition? It runs without a license:

docker run -d --name ee -p 127.0.0.1:18001:8001 -e KONG_DATABASE=off \
  -e KONG_ADMIN_LISTEN=0.0.0.0:8001 kong/kong-gateway:3.16.0.1 > /dev/null
sleep 20
curl -s localhost:18001 | jq -c '{version, edition,
  plugins: (.plugins.available_on_server | length)}'
curl -s localhost:18001/license; echo
docker rm -f ee

Expected:

{"version":"3.16.0.1","edition":"enterprise","plugins":114}
{"message":"Not found"}
ee

Version 3.16 instead of 3.9, 114 plugins instead of 45, but no license. What of it is usable without a license is governed by Kong’s licensing model; for a lab, OSS 3.9 remains the free choice.

Save your progress? Sign up for free

Checkpoint

In the VM k3s:

kubectl get gateway -n kong kong \
  -o jsonpath='{.status.conditions[?(@.type=="Programmed")].status}'; echo

Expected: the KIC has taken over the gateway.

True
kubectl -n kong get deploy kong-gateway \
  -o jsonpath='{.spec.template.spec.containers[0].image}'; echo

Expected: the version after the upgrade.

kong:3.9.4
curl -s -o /dev/null -w '%{http_code}\n' -X POST -H 'apikey: lab-key-anna-456' \
  -H 'Content-Type: application/json' -d '{"text":"Kontrolle"}' \
  http://localhost/api/v1/notes

Expected: writing with a key works.

201

Challenge

Limit reading to five requests per minute, as on Day 8, but this time as a Kubernetes resource.

Note: A KongPlugin of type rate-limiting and the annotation konghq.com/plugins on the HTTPRoute notes-lesen.

Solution

cat <<'EOF' | kubectl apply -f -
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: limit-lesen
  namespace: notes
plugin: rate-limiting
config:
  minute: 5
  policy: local
EOF
kubectl -n notes annotate httproute notes-lesen konghq.com/plugins=limit-lesen
sleep 10
for i in $(seq 1 7); do
  curl -s -o /dev/null -w '%{http_code} ' http://localhost/api/v1/notes
done; echo

Expected:

kongplugin.configuration.konghq.com/limit-lesen created
httproute.gateway.networking.k8s.io/notes-lesen annotated
200 200 200 200 200 429 429

With kubectl apply and an annotation, without the Admin API and without decK. The KIC translates, Kong implements.

Save your progress? Sign up for free

Enterprise Note

  • Kong Operator (formerly Gateway Operator) manages control planes, data planes and their upgrades declaratively in Kubernetes, with Konnect too.
  • DB-less in the cluster means: the truth lives in the Kubernetes resources, that is, in Git and Argo CD or Flux; Kong Manager only displays.
  • Upgrades across minor versions (3.9 to 3.10 and beyond) follow Kong’s upgrade paths; with Postgres via migrations up, blue-green or new nodes.
  • Gateway API first: Standard filters for routing and headers, Kong resources only for what the Gateway API cannot do (auth, rate limits, custom plugins).