Day 6: gRPC, TCP and UDP
Goal: You route gRPC by service and method, test with grpcurl and attach pure TCP and UDP services to the same Gateway. · Duration: approx. 60 min · Prerequisite: Day 4
Briefing
- GRPCRoute (
v1, Standard since Gateway API v1.1) matches onservice/methodand headers instead of paths. Ingress could only do this viabackend-protocol: GRPCplus path tricks. - gRPC needs HTTP/2. Towards the backend,
appProtocol: kubernetes.io/h2con the Service says: plaintext HTTP/2. - TCPRoute and UDPRoute have been
v1in the Standard channel since Gateway API v1.6. They have no matches – a listener port belongs to exactly one service. - Envoy Gateway creates one port on the LoadBalancer Service per
listener port. cloud-provider-kind publishes it on
$GW_IP.
grpcurl ─TLS:443 SNI grpc─> Listener https-grpc ─h2c─> payments (Echo)
└─h2c─> payments-v2 (EchoTwo, x-version)
nc ─TCP:9100────────> Listener tcp ─> tcp-echo:9100
nc -u ─UDP:9200────────> Listener udp ─> udp-echo:9200
Drill 1: gRPC Listener With a Certificate
grpcurl comes from Day 0. The platform team adds an
HTTPS listener just for GRPCRoutes; the gateway-shim from Day 4 issues
the certificate.
mkdir -p ~/gw-lab/tag06 && cd ~/gw-lab/tag06
export GW_IP=$(gw-ip.sh) CA=~/gw-lab/tag04/lab-ca.crt
kubectl -n infra patch gateway web --type=json -p '[{"op":"add",
"path":"/spec/listeners/-","value":{"name":"https-grpc","protocol":"HTTPS",
"port":443,"hostname":"grpc.gw.localtest.me",
"tls":{"mode":"Terminate","certificateRefs":[{"name":"grpc-gw-tls"}]},
"allowedRoutes":{"namespaces":{"from":"All"},"kinds":[{"kind":"GRPCRoute"}]}}}]'
sleep 15
kubectl -n infra get certificate grpc-gw-tls
conds.sh gateway web infra | grep -A3 https-grpcExpected:
gateway.gateway.networking.k8s.io/web patched
NAME READY SECRET AGE
grpc-gw-tls True grpc-gw-tls 15s
listener https-grpc (attachedRoutes=0):
Programmed=True (Programmed): Sending translated listener configuration to the ...
Accepted=True (Accepted): Listener has been successfully translated
ResolvedRefs=True (ResolvedRefs): Listener references have been resolved
kinds: [GRPCRoute] makes sure nobody accidentally
attaches an HTTPRoute to the gRPC hostname.
Save your progress? Sign up for free
Drill 2: Second Version and GRPCRoute
To make method routing visible, you need a second backend:
cat > payments-v2.yaml <<'EOF'
# Second version of the gRPC service, to make method routing visible
apiVersion: apps/v1
kind: Deployment
metadata:
name: payments-v2
namespace: payments
labels: {app: payments-v2}
spec:
replicas: 1
selector:
matchLabels: {app: payments-v2}
template:
metadata:
labels: {app: payments-v2}
spec:
containers:
- name: grpc-echo
image: registry.k8s.io/gateway-api/echo-basic:v1.6.0
env:
- {name: GRPC_ECHO_SERVER, value: "1"}
- {name: SERVICE_NAME, value: payments-v2}
- name: POD_NAME
valueFrom: {fieldRef: {fieldPath: metadata.name}}
- name: NAMESPACE
valueFrom: {fieldRef: {fieldPath: metadata.namespace}}
ports:
- {name: grpc, containerPort: 3000}
readinessProbe:
tcpSocket: {port: grpc}
resources:
requests: {cpu: 10m, memory: 16Mi}
limits: {memory: 64Mi}
---
apiVersion: v1
kind: Service
metadata:
name: payments-v2
namespace: payments
spec:
selector: {app: payments-v2}
ports:
- name: grpc
port: 3000
targetPort: grpc
appProtocol: kubernetes.io/h2c
EOF
kubectl apply -f payments-v2.yaml
kubectl -n payments rollout status deploy/payments-v2 --timeout=90sExpected (shortened):
deployment.apps/payments-v2 created
service/payments-v2 created
deployment "payments-v2" successfully rolled out
Now the route. Rule 1 lets server reflection through, so that
grpcurl can work without a .proto file:
cat > grpcroute.yaml <<'EOF'
# gRPC routing by service/method and header
apiVersion: gateway.networking.k8s.io/v1
kind: GRPCRoute
metadata:
name: payments
namespace: payments
spec:
parentRefs:
- name: web
namespace: infra
sectionName: https-grpc
hostnames:
- grpc.gw.localtest.me
rules:
# Reflection, so that grpcurl list/describe work
- matches:
- method:
service: grpc.reflection.v1.ServerReflection
- method:
service: grpc.reflection.v1alpha.ServerReflection
backendRefs:
- {name: payments, port: 3000}
# Echo -> v1; EchoTwo -> v2; testers with x-version: v2 always -> v2
- matches:
- method:
service: gateway_api_conformance.echo_basic.grpcecho.GrpcEcho
method: Echo
backendRefs:
- {name: payments, port: 3000}
- matches:
- method:
service: gateway_api_conformance.echo_basic.grpcecho.GrpcEcho
method: EchoTwo
- headers:
- {name: x-version, value: v2}
backendRefs:
- {name: payments-v2, port: 3000}
EOF
kubectl apply -f grpcroute.yaml
conds.sh grpcroute payments paymentsExpected:
grpcroute.gateway.networking.k8s.io/payments created
parent infra/web/https-grpc:
Accepted=True (Accepted): Route is accepted
ResolvedRefs=True (ResolvedRefs): Resolved all the Object references for the Route
Multiple entries under matches are OR, fields within one
entry are AND – just like with the HTTPRoute (Day 2).
Save your progress? Sign up for free
Drill 3: grpcurl Through the Gateway
g() { grpcurl -cacert "$CA" -authority grpc.gw.localtest.me "$@"; }
g "$GW_IP:443" list
g "$GW_IP:443" describe gateway_api_conformance.echo_basic.grpcecho.GrpcEchoExpected:
gateway_api_conformance.echo_basic.grpcecho.GrpcEcho
grpc.reflection.v1.ServerReflection
grpc.reflection.v1alpha.ServerReflection
gateway_api_conformance.echo_basic.grpcecho.GrpcEcho is a service:
service GrpcEcho {
rpc Echo ( .gateway_api_conformance.echo_basic.grpcecho.EchoRequest ) returns ...
rpc EchoThree ( .gateway_api_conformance.echo_basic.grpcecho.EchoRequest ) ...
rpc EchoTwo ( .gateway_api_conformance.echo_basic.grpcecho.EchoRequest ) ...
}
-authority sets :authority and the TLS
server name (SNI) – the counterpart to --resolve with curl.
A call shows what arrives at the backend:
S=gateway_api_conformance.echo_basic.grpcecho.GrpcEcho
g "$GW_IP:443" $S/Echo | jq -c '.assertions | {authority, context}'Expected:
{"authority":"grpc.gw.localtest.me","context":{"namespace":"payments",
"serviceName":"payments","pod":"payments-6cfbdd9c64-hrk6z"}}
(Output wrapped for readability.)
Save your progress? Sign up for free
Drill 4: Method Routing and Precedence
for m in Echo EchoTwo EchoThree; do
printf '%-10s ' $m
g "$GW_IP:443" $S/$m 2>&1 | grep -oE '"serviceName": "[^"]+"|Code: [A-Za-z]+'
doneExpected:
Echo "serviceName": "payments"
EchoTwo "serviceName": "payments-v2"
EchoThree Code: Unimplemented
EchoThree matches no rule. Envoy itself responds with
gRPC status 12 Unimplemented – the gRPC counterpart of HTTP
404. Now with the tester header:
g -H 'x-version: v2' "$GW_IP:443" $S/Echo | jq -r .assertions.context.serviceName
g -H 'x-version: v2' "$GW_IP:443" $S/EchoThreeExpected:
payments
ERROR:
Code: Unimplemented
Message: method EchoThree not implemented
Two lessons: Echo with the header stays with
payments, because a method match takes precedence over a
pure header match. EchoThree with the header now reaches
payments-v2 – this time the error message comes from the
backend, not from Envoy.
Tip: Precedence in GRPCRoute: exact hostname before wildcard, then length of
service, thenmethod, then the number of header matches.
Save your progress? Sign up for free
Drill 5: TCPRoute
A TCP echo (same image, different mode) and a UDP echo for Drill 6:
cat > l4-echo.yaml <<'EOF'
# TCP and UDP echo: same image echo-basic, different mode
apiVersion: apps/v1
kind: Deployment
metadata:
name: tcp-echo
namespace: payments
spec:
replicas: 1
selector:
matchLabels: {app: tcp-echo}
template:
metadata:
labels: {app: tcp-echo}
spec:
containers:
- name: echo
image: registry.k8s.io/gateway-api/echo-basic:v1.6.0
env:
- {name: TCP_ECHO_SERVER, value: "1"}
- {name: TCP_PORT, value: "9100"}
- {name: SERVICE_NAME, value: tcp-echo}
- name: POD_NAME
valueFrom: {fieldRef: {fieldPath: metadata.name}}
resources:
requests: {cpu: 10m, memory: 16Mi}
---
apiVersion: v1
kind: Service
metadata:
name: tcp-echo
namespace: payments
spec:
selector: {app: tcp-echo}
ports:
- {name: tcp, port: 9100, targetPort: 9100, protocol: TCP}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: udp-echo
namespace: payments
spec:
replicas: 1
selector:
matchLabels: {app: udp-echo}
template:
metadata:
labels: {app: udp-echo}
spec:
containers:
- name: echo
image: registry.k8s.io/gateway-api/echo-basic:v1.6.0
env:
- {name: UDP_ECHO_SERVER, value: "1"}
- {name: UDP_PORT, value: "9200"}
- {name: SERVICE_NAME, value: udp-echo}
- name: POD_NAME
valueFrom: {fieldRef: {fieldPath: metadata.name}}
resources:
requests: {cpu: 10m, memory: 16Mi}
---
apiVersion: v1
kind: Service
metadata:
name: udp-echo
namespace: payments
spec:
selector: {app: udp-echo}
ports:
- {name: udp, port: 9200, targetPort: 9200, protocol: UDP}
EOF
kubectl apply -f l4-echo.yaml
kubectl -n payments rollout status deploy/tcp-echo --timeout=60s
kubectl -n payments rollout status deploy/udp-echo --timeout=60sExpected (shortened):
deployment "tcp-echo" successfully rolled out
deployment "udp-echo" successfully rolled out
Listener for TCP port 9100, plus its route:
kubectl -n infra patch gateway web --type=json -p '[{"op":"add",
"path":"/spec/listeners/-","value":{"name":"tcp","protocol":"TCP","port":9100,
"allowedRoutes":{"namespaces":{"from":"All"},"kinds":[{"kind":"TCPRoute"}]}}}]'
cat > tcproute.yaml <<'EOF'
apiVersion: gateway.networking.k8s.io/v1
kind: TCPRoute
metadata:
name: tcp-echo
namespace: payments
spec:
parentRefs:
- name: web
namespace: infra
sectionName: tcp
rules:
- backendRefs:
- {name: tcp-echo, port: 9100}
EOF
kubectl apply -f tcproute.yaml
sleep 5
conds.sh tcproute tcp-echo payments
printf 'TEST\nIS_TLS\n' | nc -w1 "$GW_IP" 9100Expected:
gateway.gateway.networking.k8s.io/web patched
tcproute.gateway.networking.k8s.io/tcp-echo created
parent infra/web/tcp:
Accepted=True (Accepted): Route is accepted
ResolvedRefs=True (ResolvedRefs): Resolved all the Object references for the Route
Gateway API Test TCP Server
{"namespace":"","ingress":"","service":"tcp-echo","pod":"tcp-echo-6dd48d5d-b4ldh",
"tcpport":"9100","tlsport":"8443","isTLS":false}
false
(JSON line wrapped.) No hostname, no path, no header: with TCP only the port decides. Two TCPRoutes on the same listener would be a conflict.
Save your progress? Sign up for free
Drill 6: UDPRoute
kubectl -n infra patch gateway web --type=json -p '[{"op":"add",
"path":"/spec/listeners/-","value":{"name":"udp","protocol":"UDP","port":9200,
"allowedRoutes":{"namespaces":{"from":"All"},"kinds":[{"kind":"UDPRoute"}]}}}]'
cat > udproute.yaml <<'EOF'
apiVersion: gateway.networking.k8s.io/v1
kind: UDPRoute
metadata:
name: udp-echo
namespace: payments
spec:
parentRefs:
- name: web
namespace: infra
sectionName: udp
rules:
- backendRefs:
- {name: udp-echo, port: 9200}
EOF
kubectl apply -f udproute.yaml
sleep 5
echo -n hallo-udp | nc -u -w2 "$GW_IP" 9200; echo
kubectl -n payments logs deploy/udp-echo | tail -1Expected:
gateway.gateway.networking.k8s.io/web patched
udproute.gateway.networking.k8s.io/udp-echo created
{"request":"hallo-udp","namespace":"","ingress":"","service":"udp-echo",
"pod":"udp-echo-6cdb5cc897-6tv8q"}
Received UDP hallo-udp from 10.244.0.10:47085
The source address in the log is the Envoy pod, not your laptop: for UDP, Envoy is a non-transparent proxy.
Tip: Nothing comes back via UDP? Test from inside the cluster to rule out cloud-provider-kind:
kubectl run nc --rm -i --restart=Never --image=busybox:1.37 -- sh -c "echo -n x | nc -u -w2 $GW_IP 9200".
Save your progress? Sign up for free
Drill 7: What Envoy Gateway Builds From It
kubectl -n envoy-gateway-system get svc \
-l gateway.envoyproxy.io/owning-gateway-name=web \
-o jsonpath='{range .items[0].spec.ports[*]}{.port}/{.protocol} {end}{"\n"}'
kubectl -n infra get gateway web -o json \
| jq -r '.status.listeners[] | "\(.name) \(.attachedRoutes)"'Expected:
80/TCP 443/TCP 8443/TCP 9100/TCP 9200/UDP
http 1
https-shop 1
https-api 2
tls-secure 1
https-grpc 1
tcp 1
udp 1
Three HTTPS listeners share port 443 (SNI), each L4 listener gets its own port. The same applies on RKE2 with Cilium LB IPAM (Day 9) – check there whether your LB implementation supports mixed TCP/UDP Services.
| Resource | API (v1.6) | Channel | Matching |
|---|---|---|---|
| GRPCRoute | v1 |
Standard | Hostname, service/method, headers |
| TLSRoute | v1 |
Standard | SNI |
| TCPRoute | v1 |
Standard | listener port only |
| UDPRoute | v1 |
Standard | listener port only |
Save your progress? Sign up for free
Checkpoint
cd ~/gw-lab/tag06
check() { if eval "$2" >/dev/null 2>&1; then echo "✔ $1"; else echo "✘ $1"; fi; }
export GW_IP=$(gw-ip.sh) CA=~/gw-lab/tag04/lab-ca.crt
g() { grpcurl -cacert "$CA" -authority grpc.gw.localtest.me "$@"; }
S=gateway_api_conformance.echo_basic.grpcecho.GrpcEcho
check "GRPCRoute Accepted" \
'conds.sh grpcroute payments payments | grep -q "Accepted=True"'
check "Echo -> payments" \
'[ "$(g $GW_IP:443 $S/Echo | jq -r .assertions.context.serviceName)" = payments ]'
check "EchoTwo -> payments-v2" \
'g $GW_IP:443 $S/EchoTwo | grep -q "\"payments-v2\""'
check "EchoThree ohne Route" '! g $GW_IP:443 $S/EchoThree'
check "TCP-Echo" 'printf "TEST\n" | nc -w1 $GW_IP 9100 | grep -q tcp-echo'
check "UDP-Echo" 'echo -n x | nc -u -w2 $GW_IP 9200 | grep -q udp-echo'Expected:
✔ GRPCRoute Accepted
✔ Echo -> payments
✔ EchoTwo -> payments-v2
✔ EchoThree ohne Route
✔ TCP-Echo
✔ UDP-Echo
Challenge
Roll out Echo as a canary: 80% to payments,
20% to payments-v2. Measure the distribution over 50
calls.
Hint: backendRefs[].weight works in
GRPCRoute exactly as in HTTPRoute.
Solution
kubectl -n payments patch grpcroute payments --type=json -p '[
{"op":"replace","path":"/spec/rules/1/backendRefs","value":[
{"name":"payments","port":3000,"weight":80},
{"name":"payments-v2","port":3000,"weight":20}]}]'
sleep 3
for _ in $(seq 50); do
g "$GW_IP:443" $S/Echo | jq -r .assertions.context.serviceName
done | sort | uniq -cExpected (shortened, may differ depending on the version; numbers fluctuate):
grpcroute.gateway.networking.k8s.io/payments patched
41 payments
9 payments-v2
Envoy distributes per request, not per connection – even though gRPC reuses one HTTP/2 connection for many calls.
Save your progress? Sign up for free
Enterprise Note
- Long gRPC streams run into Envoy timeouts (default 15 s route
timeout). Adjust them for streams via
BackendTrafficPolicy(Day 10). - gRPC health checks and per-request instead of per-connection load balancing are the reason to run gRPC through an L7 gateway instead of an L4 load balancer.
- Expose reflection externally only in dev/test; in production, leave out the reflection rule or protect it with a SecurityPolicy.
- TCP/UDP listeners each use one port on the external LB: manage firewall rules, port catalogs and conflicts between teams centrally (Day 7).
- UDP (DNS, syslog, non-QUIC protocols) only sees the Envoy IP in the backend. If the service needs the client IP, a direct LoadBalancer Service is often the better choice.