Day 14: Kong Ingress Controller
Goal: Run Kong as an ingress controller in k3s, publish notes through the Gateway API, secure it with Kong resources, upgrade without downtime and take a look at Enterprise. · Duration: approx. 90 min · Prerequisite: Day 13
Briefing
- The Kong Ingress Controller (KIC) reads Kubernetes
resources and writes from them the configuration of a Kong without a
database: a
kong.yamlthat emerges from the cluster. - You describe routes with the Gateway API
(
GatewayClass,Gateway,HTTPRoute), Kong-specific things withKongPlugin,KongConsumerand Secrets. - The Helm chart
kong/ingressinstalls two Deployments:kong-controller(KIC) andkong-gateway(the proxy). - An upgrade is a
helm upgradewith a new image tag, pod by pod.
HTTPRoute, KongPlugin, KongConsumer ──► kong-controller ──Admin API──► kong-gateway
:80 (ServiceLB)
curl http://<IP of k3s>/api/v1/notes ──────────────────────────────────►┘ ─► notes-api
Drill 1: k3s on a VM
On the laptop:
multipass launch 24.04 --name k3s --cpus 2 --memory 3G --disk 20G
multipass shell k3sIn the VM k3s. Traefik stays off; Kong takes over its
role:
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.36.5+k3s1 sh -s - \
--disable traefik --write-kubeconfig-mode 644
mkdir -p ~/.kube && cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
kubectl wait --for=condition=Ready node --all --timeout=120sExpected (shortened):
[INFO] systemd: Starting k3s
error: no matching resources found
The error is harmless: right after startup the node has not
registered yet, so there is nothing for kubectl wait to
wait for. A few seconds later:
sleep 20
kubectl get nodesExpected:
NAME STATUS ROLES AGE VERSION
k3s Ready control-plane 33s v1.36.5+k3s1
Save your progress? Sign up for free
Drill 2: Helm, Gateway API and Companion Code
cd /tmp && V=v4.3.1
curl -fsSLO https://get.helm.sh/helm-$V-linux-amd64.tar.gz
curl -fsSLO https://get.helm.sh/helm-$V-linux-amd64.tar.gz.sha256sum
sha256sum -c helm-$V-linux-amd64.tar.gz.sha256sum
tar xzf helm-$V-linux-amd64.tar.gz linux-amd64/helm
sudo install linux-amd64/helm /usr/local/bin/helm && rm -rf linux-amd64 helm-$V*
helm version --short
GW=https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.3.0
kubectl apply -f $GW/standard-install.yaml | tail -1
git clone https://github.com/thomaszachmann/kong-zum-anfassen.git ~/kong-lab/codeExpected (shortened; on Apple Silicon arm64 instead of
amd64):
helm-v4.3.1-linux-amd64.tar.gz: OK
v4.3.1+gc0d08d0
customresourcedefinition.apiextensions.k8s.io/referencegrants.gateway.networking.k8s.io
created
Cloning into '/home/ubuntu/kong-lab/code'...
KIC 3.5 builds on Gateway API v1.3.0. Kubernetes does not ship the Gateway API resources; you install them as CRDs.
Save your progress? Sign up for free
Drill 3: The Image Without a Registry
k3s has no Docker. You do not rebuild the notes-api image, you bring
it over from the VM kong. There:
docker save -o ~/notes-api.tar notes-api:0.2.0On the laptop:
multipass transfer kong:/home/ubuntu/notes-api.tar k3s:/home/ubuntu/In the VM k3s:
sudo k3s ctr images import ~/notes-api.tar
sudo k3s ctr images ls | grep notes-api | cut -c1-40Expected (shortened):
docker.io/library/notes api:0.2.0 saved
application/vnd.oci.image.index.v1+json sha256:7b0d2646bc6e…
Importing elapsed: 1.0 s total: 0.0 B (0.0 B/s)
docker.io/library/notes-api:0.2.0
containerd expands notes-api:0.2.0 to
docker.io/library/notes-api:0.2.0. The Deployment takes the
image from there with imagePullPolicy: Never.
Save your progress? Sign up for free
Drill 4: Install the Kong Ingress Controller
cd ~/kong-lab/code/tag14
grep -v '^#' kic-values.yaml
helm repo add kong https://charts.konghq.com
helm install kong kong/ingress --version 0.24.0 -n kong --create-namespace \
-f kic-values.yaml --wait --timeout 5m | grep -E 'STATUS|REVISION'
kubectl -n kong get pods
kubectl -n kong get svc kong-gateway-proxyExpected (shortened, your IP and names will differ):
controller:
ingressController:
image:
tag: "3.5.14"
gateway:
image:
repository: kong
tag: "3.9.3" # bewusst eine Version zurück: Upgrade in Drill 7
env:
plugins: bundled
"kong" has been added to your repositories
STATUS: deployed
REVISION: 1
NAME READY STATUS RESTARTS AGE
kong-controller-6985bb6dc4-bs5m8 1/1 Running 0 54s
kong-gateway-854656f547-mwlvt 1/1 Running 0 54s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
kong-gateway-proxy LoadBalancer 10.43.101.109 192.168.64.9 80:30980/TCP,443:…
Otherwise the chart sets floating tags (3.9,
3.5); the values file pins them. k3s’s
ServiceLB gives the proxy the VM’s IP on port 80.
Save your progress? Sign up for free
Drill 5: Gateway and Routes
cd ~/kong-lab/code/tag14/k8s
kubectl apply -f gateway.yaml
kubectl apply -f notes.yaml | tail -1
kubectl -n notes rollout status deploy/notes-api --timeout=120s
kubectl apply -f routes.yaml
sleep 8
kubectl get gateway -n kong
P=http://localhost
curl -s $P/api/v1/notes; echo
curl -s $P/echo/get | jq -r .urlExpected (shortened):
gatewayclass.gateway.networking.k8s.io/kong created
gateway.gateway.networking.k8s.io/kong created
service/echo created
deployment "notes-api" successfully rolled out
httproute.gateway.networking.k8s.io/notes-lesen created
httproute.gateway.networking.k8s.io/echo created
NAME CLASS ADDRESS PROGRAMMED AGE
kong kong 192.168.64.9 True 16s
[]
http://localhost/get
gatewayclass-unmanaged in gateway.yaml
tells the KIC: Kong is already running, use the proxy from the Helm
chart. The HTTPRoute notes-lesen corresponds to the route
from Day 2: GET /api/v1/notes, rewritten to
/notes with the URLRewrite filter.
Save your progress? Sign up for free
Drill 6: Writing with a Key
grep -E '^kind:|^plugin:|konghq.com/|name: X-API-Key' schreiben.yaml
kubectl apply -f schreiben.yaml
sleep 10
H='Content-Type: application/json'
curl -s -o /dev/null -w '%{http_code}\n' -X POST -H "$H" -d '{"text":"x"}' \
$P/api/v1/notes
curl -s -w ' %{http_code}\n' -X POST -H 'apikey: lab-key-anna-456' -H "$H" \
-d '{"text":"aus Kubernetes"}' $P/api/v1/notes
kubectl -n notes get kongconsumerExpected (shortened):
kind: KongPlugin
plugin: key-auth
kind: Secret
konghq.com/credential: key-auth
kind: KongConsumer
kind: HTTPRoute
konghq.com/plugins: key-auth
- {name: X-API-Key, value: lab-api-key-123}
kongplugin.configuration.konghq.com/key-auth created
secret/anna-key created
kongconsumer.configuration.konghq.com/anna created
httproute.gateway.networking.k8s.io/notes-schreiben created
401
{"id":"b4b67efb-…","text":"aus Kubernetes"}
201
NAME USERNAME AGE PROGRAMMED
anna anna 8s True
Anna’s key is a Secret with the label
konghq.com/credential: key-auth; the consumer refers to it.
The annotation konghq.com/plugins attaches
key-auth to the route. The backend key is set by the
standard filter RequestHeaderModifier.
Warning: An additional
KongPluginof typerequest-transformerhas no effect here. The KIC implementsURLRewriteitself as arequest-transformer, and only one of these plugins runs per route. Gateway API filters take precedence over handcrafted ones.
Query the list a few times:
for i in 1 2 3 4; do curl -s $P/api/v1/notes; echo; doneExpected (the order varies):
[]
[{"id":"b4b67efb-…","text":"aus Kubernetes"}]
[]
[{"id":"b4b67efb-…","text":"aus Kubernetes"}]
Two notes-api pods, each with its own storage. Without a shared database (“Kubernetes hands-on”, Day 7), each client sees different data depending on the pod.
Save your progress? Sign up for free
Drill 7: Upgrade Without Downtime
Kong 3.9.3 is running; 3.9.4 is the current version of the line. During the upgrade, a loop in the background counts every response:
curl -si $P/echo/get | grep -i '^via'
( ok=0; f=0; ende=$((SECONDS+90))
while [ $SECONDS -lt $ende ]; do
c=$(curl -s -o /dev/null -w '%{http_code}' --max-time 2 $P/echo/get)
[ "$c" = 200 ] && ok=$((ok+1)) || f=$((f+1)); sleep 0.2
done; echo "ok=$ok fehler=$f" ) &
helm upgrade kong kong/ingress --version 0.24.0 -n kong --reuse-values \
--set gateway.image.tag=3.9.4 --wait --timeout 5m | grep -E 'STATUS|REVISION'
wait
curl -si $P/echo/get | grep -i '^via'Expected (the number varies):
Via: 1.1 kong/3.9.3
STATUS: deployed
REVISION: 2
ok=372 fehler=0
Via: 1.1 kong/3.9.4
Not a single error: Kubernetes starts the new pod, waits until it is
ready and only then terminates the old one. Without a database there are
no migrations; with a database, kong migrations up would
come before the switch and kong migrations finish after
it.
Save your progress? Sign up for free
Drill 8: From kong.yaml to Kubernetes, and Enterprise
Back on the VM kong. decK translates your configuration
into KIC resources:
cd ~/kong-lab/kong
deck file kong2kic -s kong.yaml --class-name kong -o /tmp/kic.yaml
grep '^kind:' /tmp/kic.yaml | sort | uniq -cExpected:
8 kind: HTTPRoute
4 kind: KongClusterPlugin
4 kind: KongConsumer
16 kind: KongPlugin
2 kind: KongUpstreamPolicy
10 kind: Secret
6 kind: Service
A good start for a migration, but not a finished result: you still have to check and adjust the namespaces, the backend services and the Secrets.
And the Enterprise edition? It runs without a license:
docker run -d --name ee -p 127.0.0.1:18001:8001 -e KONG_DATABASE=off \
-e KONG_ADMIN_LISTEN=0.0.0.0:8001 kong/kong-gateway:3.16.0.1 > /dev/null
sleep 20
curl -s localhost:18001 | jq -c '{version, edition,
plugins: (.plugins.available_on_server | length)}'
curl -s localhost:18001/license; echo
docker rm -f eeExpected:
{"version":"3.16.0.1","edition":"enterprise","plugins":114}
{"message":"Not found"}
ee
Version 3.16 instead of 3.9, 114 plugins instead of 45, but no license. What of it is usable without a license is governed by Kong’s licensing model; for a lab, OSS 3.9 remains the free choice.
Save your progress? Sign up for free
Checkpoint
In the VM k3s:
kubectl get gateway -n kong kong \
-o jsonpath='{.status.conditions[?(@.type=="Programmed")].status}'; echoExpected: the KIC has taken over the gateway.
True
kubectl -n kong get deploy kong-gateway \
-o jsonpath='{.spec.template.spec.containers[0].image}'; echoExpected: the version after the upgrade.
kong:3.9.4
curl -s -o /dev/null -w '%{http_code}\n' -X POST -H 'apikey: lab-key-anna-456' \
-H 'Content-Type: application/json' -d '{"text":"Kontrolle"}' \
http://localhost/api/v1/notesExpected: writing with a key works.
201
Challenge
Limit reading to five requests per minute, as on Day 8, but this time as a Kubernetes resource.
Note: A KongPlugin of type
rate-limiting and the annotation
konghq.com/plugins on the HTTPRoute
notes-lesen.
Solution
cat <<'EOF' | kubectl apply -f -
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
name: limit-lesen
namespace: notes
plugin: rate-limiting
config:
minute: 5
policy: local
EOF
kubectl -n notes annotate httproute notes-lesen konghq.com/plugins=limit-lesen
sleep 10
for i in $(seq 1 7); do
curl -s -o /dev/null -w '%{http_code} ' http://localhost/api/v1/notes
done; echoExpected:
kongplugin.configuration.konghq.com/limit-lesen created
httproute.gateway.networking.k8s.io/notes-lesen annotated
200 200 200 200 200 429 429
With kubectl apply and an annotation, without the Admin
API and without decK. The KIC translates, Kong implements.
Save your progress? Sign up for free
Enterprise Note
- Kong Operator (formerly Gateway Operator) manages control planes, data planes and their upgrades declaratively in Kubernetes, with Konnect too.
- DB-less in the cluster means: the truth lives in the Kubernetes resources, that is, in Git and Argo CD or Flux; Kong Manager only displays.
- Upgrades across minor versions (3.9 to 3.10 and
beyond) follow Kong’s upgrade paths; with Postgres via
migrations up, blue-green or new nodes. - Gateway API first: Standard filters for routing and headers, Kong resources only for what the Gateway API cannot do (auth, rate limits, custom plugins).