Day 10: Ingress with Gateway API
Goal: The app is reachable from outside via HTTPS – once through a Gateway outside the mesh, once through a proxy in the mesh – and you know what that means for mTLS and policies. · Duration: approx. 75 min · Prerequisite: Day 9
Briefing
- Linkerd ships no ingress of its own. It works with the Gateway or ingress controller you already have.
- Variant A: Gateway outside the mesh – here Cilium (Envoy on every node). The first hop into the mesh is not mTLS, and the target only sees an IP. Policies need a NetworkAuthentication for that.
- Variant B: Ingress proxy in the mesh – from the entry Pod onward, everything runs with mTLS and identity.
- The lab has no LoadBalancer IPs; you reach both via NodePorts.
- Today in
tag10with your own copy ofwebandnotes-api.
Laptop ─HTTPS─> cp1:NodePort ─> Cilium-Envoy ──plaintext─> web ─mTLS─> notes-api
Laptop ─HTTPS─> cp1:30443 ───> edge (in mesh) ──mTLS─────> web ─mTLS─> notes-api
Drill 1: Today’s App
mkdir -p ~/linkerd/tag10 && cd ~/linkerd/tag10
kubectl create namespace tag10
kubectl annotate namespace tag10 linkerd.io/inject=enabled
cat > app.yaml <<'EOF'
apiVersion: v1
kind: ServiceAccount
metadata: {name: web, namespace: tag10}
---
apiVersion: v1
kind: ServiceAccount
metadata: {name: notes-api, namespace: tag10}
---
apiVersion: apps/v1
kind: Deployment
metadata: {name: notes-api, namespace: tag10}
spec:
replicas: 1
selector: {matchLabels: {app: notes-api}}
template:
metadata: {labels: {app: notes-api}}
spec:
serviceAccountName: notes-api
containers:
- name: notes-api
image: ghcr.io/thomaszachmann/notes-api:0.7.1
env: [{name: NOTES_API_KEY, value: lab-key-123}]
ports: [{containerPort: 8080, name: http}]
readinessProbe:
httpGet: {path: /healthz, port: http}
---
apiVersion: v1
kind: Service
metadata: {name: notes-api, namespace: tag10}
spec:
selector: {app: notes-api}
ports: [{port: 80, targetPort: http, name: http}]
---
apiVersion: v1
kind: ConfigMap
metadata: {name: web-conf, namespace: tag10}
data:
default.conf: |
server {
listen 8080;
location = / { default_type text/plain; return 200 "notes web\n"; }
location /api/ {
proxy_set_header X-API-Key lab-key-123;
proxy_pass http://notes-api.tag10.svc.cluster.local/;
}
}
---
apiVersion: apps/v1
kind: Deployment
metadata: {name: web, namespace: tag10}
spec:
replicas: 1
selector: {matchLabels: {app: web}}
template:
metadata: {labels: {app: web}}
spec:
serviceAccountName: web
containers:
- name: web
image: docker.io/nginxinc/nginx-unprivileged:1.31.7-alpine
ports: [{containerPort: 8080, name: http}]
volumeMounts: [{name: conf, mountPath: /etc/nginx/conf.d}]
volumes: [{name: conf, configMap: {name: web-conf}}]
---
apiVersion: v1
kind: Service
metadata: {name: web, namespace: tag10}
spec:
selector: {app: web}
ports: [{port: 80, targetPort: http, name: http}]
EOF
kubectl apply -f app.yaml
kubectl -n tag10 rollout status deploy --timeout=180s
kubectl -n tag10 get podsExpected (abridged):
NAME READY STATUS RESTARTS AGE
notes-api-7d55845b4f-cmmlx 2/2 Running 0 10s
web-7d94ccc84d-ngbvt 2/2 Running 0 10s
notes-api runs without a database, using in-memory
storage – that is enough for today.
Save your progress? Sign up for free
Drill 2: A Gateway with Cilium
kubectl get gatewayclass
cat > gateway.yaml <<'EOF'
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata: {name: notes, namespace: tag10}
spec:
gatewayClassName: cilium
listeners:
- {name: http, protocol: HTTP, port: 80}
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata: {name: web, namespace: tag10}
spec:
parentRefs:
- {name: notes}
hostnames: [notes.lab.internal]
rules:
- backendRefs:
- {name: web, port: 80}
EOF
kubectl apply -f gateway.yaml
sleep 15
kubectl -n tag10 get gateway notes
kubectl -n tag10 get svc cilium-gateway-notesExpected (abridged):
NAME CONTROLLER ACCEPTED AGE
cilium io.cilium/gateway-controller True 12h
NAME CLASS ADDRESS PROGRAMMED AGE
notes cilium False 16s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
cilium-gateway-notes LoadBalancer 10.99.157.141 <pending> 80:31972/TCP 16s
The parent is now a Gateway, not a Service as on Day
9. Cilium creates a Service of type LoadBalancer for it.
The IP stays <pending>: Cilium could assign one
(LB-IPAM is enabled), but the lab has no IP pool and no L2 announcement.
PROGRAMMED False means exactly that
(AddressNotAssigned). Every LoadBalancer Service also has a
NodePort, though – and that is enough for the lab:
NP=$(kubectl -n tag10 get svc cilium-gateway-notes \
-o jsonpath='{.spec.ports[?(@.port==80)].nodePort}')
curl -s -i -H 'Host: notes.lab.internal' http://172.16.0.126:$NP/ | head -2
curl -s -H 'Host: notes.lab.internal' http://172.16.0.228:$NP/api/notes; echo
curl -s -o /dev/null -w '%{http_code}\n' http://172.16.0.126:$NP/Expected:
HTTP/1.1 200 OK
server: envoy
[]
404
Every node accepts the NodePort, including worker1.
Without a matching Host header, Envoy answers with 404 –
the route only applies to notes.lab.internal.
Who serves which route? The parentRef decides:
kubectl -n tag10 get httproute web \
-o jsonpath='{.status.parents[*].controllerName}{"\n"}'
kubectl -n tag10 get endpointslices -l io.cilium.gateway/owning-gateway=notesExpected:
io.cilium/gateway-controller
NAME ADDRESSTYPE PORTS ENDPOINTS AGE
cilium-gateway-notes IPv4 9999 192.192.192.192 16s
| parentRef | implemented by | where |
|---|---|---|
Gateway notes |
Cilium | Envoy on the node, north-south |
| Service (Day 9) | Linkerd and Cilium | client proxy or Envoy, east-west |
Only Cilium knows this route; Linkerd ignores routes with a Gateway
parent. The EndpointSlice with 192.192.192.192:9999 is
Cilium’s placeholder for the Gateway Service and belongs to the Gateway
– it disappears along with it. Routes with a Service parent behave
differently: there it remains after the route is deleted (Day 9, Drill
7).
Save your progress? Sign up for free
Drill 3: HTTPS with cert-manager
A small lab CA in tag10, and from it a certificate for
notes.lab.internal:
cat > tls.yaml <<'EOF'
apiVersion: cert-manager.io/v1
kind: Issuer
metadata: {name: selfsigned, namespace: tag10}
spec:
selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata: {name: lab-ca, namespace: tag10}
spec:
isCA: true
commonName: lab-ca.tag10
secretName: lab-ca
privateKey: {algorithm: ECDSA, size: 256}
issuerRef: {name: selfsigned, kind: Issuer}
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata: {name: lab-ca, namespace: tag10}
spec:
ca: {secretName: lab-ca}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata: {name: notes-tls, namespace: tag10}
spec:
secretName: notes-tls
dnsNames: [notes.lab.internal]
duration: 720h
privateKey: {algorithm: ECDSA, size: 256}
issuerRef: {name: lab-ca, kind: Issuer}
EOF
kubectl apply -f tls.yaml
kubectl -n tag10 wait --for=condition=Ready certificate --all --timeout=60s
cat > gateway-tls.yaml <<'EOF'
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata: {name: notes, namespace: tag10}
spec:
gatewayClassName: cilium
listeners:
- {name: http, protocol: HTTP, port: 80}
- name: https
protocol: HTTPS
port: 443
hostname: notes.lab.internal
tls:
mode: Terminate
certificateRefs: [{name: notes-tls}]
EOF
kubectl apply -f gateway-tls.yaml
sleep 15
kubectl -n tag10 get svc cilium-gateway-notesExpected (abridged):
certificate.cert-manager.io/lab-ca condition met
certificate.cert-manager.io/notes-tls condition met
gateway.gateway.networking.k8s.io/notes configured
NAME TYPE ... EXTERNAL-IP PORT(S)
cilium-gateway-notes LoadBalancer ... <pending> 80:31972/TCP,443:32290/TCP
Without a sectionName, the route from Drill 2 attaches
to both listeners. Now verify with the CA instead of
-k:
kubectl -n tag10 get secret lab-ca -o jsonpath='{.data.ca\.crt}' | base64 -d \
> lab-ca.crt
NP=$(kubectl -n tag10 get svc cilium-gateway-notes \
-o jsonpath='{.spec.ports[?(@.port==443)].nodePort}')
G="--cacert lab-ca.crt --resolve notes.lab.internal:$NP:172.16.0.126 \
https://notes.lab.internal:$NP"
curl -s -w ' %{http_code}\n' $G/api/notes
curl -sv $G/ 2>&1 | grep -E 'SSL connection|issuer:|^< server'Expected:
[]
200
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / X25519 / id-ecPublicKey
* issuer: CN=lab-ca.tag10
< server: envoy
From your laptop it works the same way – fetch the CA, then run
curl with the same address:
scp [email protected]:linkerd/tag10/lab-ca.crt .
curl -s -w ' %{http_code}\n' --cacert lab-ca.crt \
--resolve notes.lab.internal:32290:172.16.0.126 \
https://notes.lab.internal:32290/api/notesExpected:
[]
200
Replace 32290 with your NodePort. Cilium reads the
Secret notes-tls itself and mirrors it to
cilium-secrets – no restart needed.
Save your progress? Sign up for free
Drill 4: What the Mesh Sees
timeout 20 linkerd viz tap -n tag10 deploy/web --path /api/notes > tap.txt &
sleep 3
for i in 1 2 3; do curl -s -o /dev/null $G/api/notes; done
wait
head -2 tap.txt
kubectl get ciliumnodes \
-o jsonpath='{range .items[*]}{.metadata.name} {.spec.ingress.ipv4}{"\n"}{end}'Expected (abridged):
req id=0:0 proxy=in src=10.0.0.235:41865 dst=10.0.1.28:8080 tls=no_tls_from_remote ...
rsp id=0:0 proxy=in src=10.0.0.235:41865 dst=10.0.1.28:8080 tls=no_tls_from_remote ...
cp1 10.0.0.235
worker1 10.0.1.169
worker2 10.0.2.57
tls=no_tls_from_remote: the request arrived in
plaintext, without an identity. The sender is not a Pod IP but Cilium’s
ingress IP on cp1 – the node whose
NodePort was hit. Every node has its own. You cannot bring the Gateway
itself into the mesh: Cilium’s Envoy runs as the DaemonSet
cilium-envoy in the host network, not as a Pod with a
sidecar.
Warning: The path laptop → Envoy is TLS, but Envoy →
webis plaintext over the Pod network. Fromwebonward, Linkerd takes over. If you need encryption all the way to the first Pod, that is an argument for Variant B.
Save your progress? Sign up for free
Drill 5: Adding the Gateway to the Policy
You know Server and AuthorizationPolicy
from Day 6. A Server without a policy blocks everything:
cat > server.yaml <<'EOF'
apiVersion: policy.linkerd.io/v1beta3
kind: Server
metadata: {name: web-http, namespace: tag10}
spec:
podSelector: {matchLabels: {app: web}}
port: http
EOF
kubectl apply -f server.yaml
sleep 5
curl -s -w ' %{http_code}\n' $G/api/notesExpected:
server.policy.linkerd.io/web-http created
403
A MeshTLSAuthentication does not help here – Envoy has
no identity. That leaves the sender IP. Enter the ingress IPs from Drill
4:
cat > authz-gateway.yaml <<'EOF'
apiVersion: policy.linkerd.io/v1alpha1
kind: NetworkAuthentication
metadata: {name: cilium-ingress, namespace: tag10}
spec:
networks:
- cidr: 10.0.0.235/32
- cidr: 10.0.1.169/32
- cidr: 10.0.2.57/32
---
apiVersion: policy.linkerd.io/v1alpha1
kind: AuthorizationPolicy
metadata: {name: web-from-gateway, namespace: tag10}
spec:
targetRef: {group: policy.linkerd.io, kind: Server, name: web-http}
requiredAuthenticationRefs:
- {group: policy.linkerd.io, kind: NetworkAuthentication, name: cilium-ingress}
EOF
kubectl apply -f authz-gateway.yaml
sleep 5
curl -s -w ' %{http_code}\n' $G/api/notes
kubectl -n tag10 run ohne --image=curlimages/curl:8.23.0 --restart=Never \
--annotations=linkerd.io/inject=disabled --command -- sleep 3600
kubectl -n tag10 wait --for=condition=Ready pod/ohne --timeout=60s
kubectl -n tag10 exec ohne -- curl -s -w ' %{http_code}\n' http://web/api/notesExpected:
networkauthentication.policy.linkerd.io/cilium-ingress created
authorizationpolicy.policy.linkerd.io/web-from-gateway created
[]
200
pod/ohne created
pod/ohne condition met
403
The Gateway gets through, but a Pod without a proxy going directly to
web does not. The price: the rule depends on IP addresses.
When a node is added, the list needs a new entry – entering an entire
Pod CIDR would let in every Pod without a proxy.
Save your progress? Sign up for free
Drill 6: An Ingress Proxy in the Mesh
Variant B: an nginx as the entry point, with a proxy, TLS with the same Secret, reachable via NodePort 30443:
cat > edge.yaml <<'EOF'
apiVersion: v1
kind: ServiceAccount
metadata: {name: edge, namespace: tag10}
---
apiVersion: v1
kind: ConfigMap
metadata: {name: edge-conf, namespace: tag10}
data:
default.conf: |
server {
listen 8443 ssl;
server_name notes.lab.internal;
ssl_certificate /tls/tls.crt;
ssl_certificate_key /tls/tls.key;
location / {
proxy_pass http://web.tag10.svc.cluster.local;
}
}
---
apiVersion: apps/v1
kind: Deployment
metadata: {name: edge, namespace: tag10}
spec:
replicas: 1
selector: {matchLabels: {app: edge}}
template:
metadata: {labels: {app: edge}}
spec:
serviceAccountName: edge
containers:
- name: nginx
image: docker.io/nginxinc/nginx-unprivileged:1.31.7-alpine
ports: [{containerPort: 8443, name: https}]
volumeMounts:
- {name: conf, mountPath: /etc/nginx/conf.d}
- {name: tls, mountPath: /tls, readOnly: true}
volumes:
- {name: conf, configMap: {name: edge-conf}}
- {name: tls, secret: {secretName: notes-tls}}
---
apiVersion: v1
kind: Service
metadata: {name: edge, namespace: tag10}
spec:
type: NodePort
selector: {app: edge}
ports: [{port: 443, targetPort: https, nodePort: 30443, name: https}]
EOF
kubectl apply -f edge.yaml
kubectl -n tag10 rollout status deploy/edge --timeout=120s
sleep 10
E="--cacert lab-ca.crt --resolve notes.lab.internal:30443:172.16.0.126 \
https://notes.lab.internal:30443"
curl -s -w ' %{http_code}\n' $E/api/notesExpected:
deployment "edge" successfully rolled out
403
That is correct: web only lets the Gateway in. But
edge has an identity – so the authorization uses mTLS
instead of IPs:
cat > authz-edge.yaml <<'EOF'
apiVersion: policy.linkerd.io/v1alpha1
kind: MeshTLSAuthentication
metadata: {name: edge, namespace: tag10}
spec:
identityRefs:
- {kind: ServiceAccount, name: edge}
---
apiVersion: policy.linkerd.io/v1alpha1
kind: AuthorizationPolicy
metadata: {name: web-from-edge, namespace: tag10}
spec:
targetRef: {group: policy.linkerd.io, kind: Server, name: web-http}
requiredAuthenticationRefs:
- {group: policy.linkerd.io, kind: MeshTLSAuthentication, name: edge}
EOF
kubectl apply -f authz-edge.yaml
sleep 5
for i in $(seq 20); do curl -s -o /dev/null -w '%{http_code}\n' $E/api/notes; done \
| sort | uniq -cExpected:
20 200
nginx addresses the Service web. The
proxy in the edge Pod recognizes the target and handles load balancing,
mTLS and – with HTTPRoutes – retries and canaries, just as for any other
client. Controllers that pick endpoints themselves need
linkerd.io/inject: ingress for this (see the Enterprise
Note).
Save your progress? Sign up for free
Drill 7: Comparing Both Entry Points
for i in $(seq 90); do
curl -s -o /dev/null $E/api/notes; curl -s -o /dev/null $G/api/notes; sleep 0.5
done
linkerd viz edges -n tag10 deploy
linkerd viz authz -n tag10 deploy/web | grep -E 'ROUTE|web-http'Expected (abridged, numbers vary):
SRC DST SRC_NS DST_NS SECURED
edge web tag10 tag10 √
web notes-api tag10 tag10 √
ROUTE SERVER AUTHORIZATION UNAUTHORIZED SUCCESS RPS
default web-http authorizationpolicy/web-from-edge 0.0rps 100.00% 1.5rps
default web-http authorizationpolicy/web-from-gateway 0.0rps 100.00% 1.5rps
Both paths carry the same amount of traffic, but only one is an edge
in the mesh: for edges, the Gateway does not exist.
authz shows which policy lets which traffic through – that
is your evidence for Variant A.
Save your progress? Sign up for free
Exercise: Speed Drills
- Show the Gateway’s two NodePorts. Target time: 20 s
- Show when
notes-tlsexpires and when cert-manager renews it. Target time: 30 s - Show the issuer of the certificate that
edgeserves. Target time: 45 s - Show which policies allow traffic to
web. Target time: 15 s
Solution
kubectl -n tag10 get svc cilium-gateway-notes \
-o jsonpath='{.spec.ports[*].nodePort}{"\n"}'
kubectl -n tag10 get certificate notes-tls \
-o jsonpath='{.status.notAfter} {.status.renewalTime}{"\n"}'
echo | openssl s_client -connect 172.16.0.126:30443 -servername notes.lab.internal \
2>/dev/null | openssl x509 -noout -issuer
linkerd viz authz -n tag10 deploy/webExpected (excerpt):
31972 32290
2026-12-19T16:29:52Z 2026-12-09T16:29:52Z
issuer=CN = lab-ca.tag10
Save your progress? Sign up for free
Checkpoint
curl -s -o /dev/null -w '%{http_code}\n' $G/api/notes
curl -s -o /dev/null -w '%{http_code}\n' $E/api/notesExpected: 200 twice.
kubectl -n tag10 exec ohne -- curl -s -o /dev/null -w '%{http_code}\n' http://web/Expected: 403
linkerd viz edges -n tag10 deploy | grep edgeExpected:
edge web tag10 tag10 √
Challenge
Renew notes-tls with cmctl renew and use
openssl to check which of the two entry points
($NP from Drill 3 and 30443) serves the new certificate
immediately. Make the other one do the same.
Hint: Compare the serial numbers; nginx only reads certificates at startup.
Solution
cmctl renew -n tag10 notes-tls
kubectl -n tag10 wait --for=condition=Ready certificate/notes-tls
sleep 10
for p in $NP 30443; do
echo | openssl s_client -connect 172.16.0.126:$p -servername notes.lab.internal \
2>/dev/null | openssl x509 -noout -serial
done
kubectl -n tag10 rollout restart deploy/edge
kubectl -n tag10 rollout status deploy/edge
echo | openssl s_client -connect 172.16.0.126:30443 -servername notes.lab.internal \
2>/dev/null | openssl x509 -noout -serialExpected (serial numbers will differ):
Manually triggered issuance of Certificate tag10/notes-tls
serial=2A1C7A9FB13DCC28AFD893690D4229F7F6E2CB1D
serial=716A09D2B37D8E82BA2C8C87F6C32E746E9AEE05
deployment "edge" successfully rolled out
serial=2A1C7A9FB13DCC28AFD893690D4229F7F6E2CB1D
Cilium watches the Secret and serves the new certificate immediately.
The kubelet does update the mounted Secret in edge after
one to two minutes, but nginx does not reread it. A real ingress
controller watches Secrets itself; a home-built entry point needs a
reload – or it runs into a wall at the next expiry.
Save your progress? Sign up for free
Cleanup
kubectl delete namespace tag10Interview Note
- Does the ingress have to be in the mesh? No, but only then do you get mTLS from the first Pod onward, identity for policies, and Linkerd metrics per edge.
- How do you let a Gateway outside the mesh through a default-deny
policy? With a
NetworkAuthenticationfor the Gateway’s sender IPs – and only those. - What is ingress mode?
linkerd.io/inject: ingressmakes the proxy read the target from thel5d-dst-overrideheader or the Host – for controllers that pick endpoints themselves.
Enterprise Note
- In production, the Gateway gets a real LoadBalancer IP (Cilium LB-IPAM with L2/BGP, MetalLB or a cloud LB) and a certificate from ACME or the corporate CA.
- Common controllers in the mesh: Envoy Gateway, Traefik, Contour, Emissary. Each controller has its own notes in the Linkerd docs (“Handling ingress traffic”).
- In ingress mode, the controller must strip
l5d-dst-overridefrom client requests – otherwise it becomes an open relay into the cluster. - The
ingressannotation belongs on the controller’s Deployment, not on the namespace. - For many auditors, the path Gateway → Pod without mTLS is a finding.
A Gateway outside the mesh then needs encryption to the backend
(
BackendTLSPolicy) or Variant B.