Skip to content
Hands-on IT IT Drills

Appendix A: Cheat Sheet

As of: Gateway API v1.6.1, Envoy Gateway v1.9.2, cert-manager v1.21.2, Cilium 1.20. Names and hosts match the lab: Gateway infra/web, GatewayClasses eg and cilium, *.gw.localtest.me in kind, *.apps.lab.internal on RKE2.

Resource Model

Resource API Owner Purpose
GatewayClass gateway.networking.k8s.io/v1 Cluster operator Choose the implementation (controllerName)
Gateway .../v1 Platform team Listeners: port, protocol, hostname, TLS
ListenerSet .../v1 (since v1.5) App team own listeners on someone else’s Gateway
HTTPRoute .../v1 App team HTTP by host, path, header, query, method
GRPCRoute .../v1 App team gRPC by service, method, header
TLSRoute .../v1 (since v1.5) App team TLS passthrough by SNI
TCPRoute, UDPRoute .../v1 (since v1.6) App team L4, listener port only
ReferenceGrant .../v1 (storage v1beta1) Owner of the target Allow cross-namespace references
BackendTLSPolicy .../v1 (since v1.4) App team TLS from the Gateway to the backend
Ingress Gateway API
IngressClass GatewayClass
(hidden in the controller) Gateway with listeners
Ingress rules HTTPRoute, GRPCRoute, TLSRoute …
Annotations Filters, rule fields, policies
Channel Content Use
Standard GA resources and fields Production
Experimental Standard plus alpha fields (retry, sessionPersistence) and x-k8s.io types Lab, targeted

Listeners and allowedRoutes

Field Values Remember
protocol HTTP, HTTPS, TLS, TCP, UDP HTTPS/TLS need tls
tls.mode Terminate, Passthrough Passthrough only with TLSRoute
allowedRoutes.namespaces.from Same, All, Selector Default Same
allowedRoutes.kinds e.g. [GRPCRoute] restricts route kinds
allowedListeners (Gateway) like namespaces allows ListenerSets
infrastructure.parametersRef EnvoyProxy Envoy Deployment and Service

Route Matching and Precedence

Rule Meaning
Entries in one matches[] element AND
several matches[] elements OR
Rule without matches implicitly PathPrefix /
PathPrefix /v2 matches /v2, /v2/x, not /v2x (segments)
Header names / values case-insensitive / case-sensitive
Route without hostnames inherits the listener’s hostname
Route and listener hostname must intersect, otherwise NoMatchingListenerHostname

Precedence for HTTPRoute, from top to bottom:

Level Criterion
1 exact hostname before wildcard
2 Exact before longest PathPrefix
3 method, then more header matches, then more query matches
4 oldest route (creationTimestamp)
5 alphabetical namespace/name

GRPCRoute: exact hostname before wildcard, then longest service, then method, then the number of header matches. No matching method: gRPC status 12 Unimplemented.

Filters and Rule Fields

Filter / field Replaces (ingress-nginx) Level
RequestRedirect (scheme, hostname, path, port, statusCode) ssl-redirect, permanent-redirect Core
RequestHeaderModifier (set/add/remove) configuration-snippet Core
ResponseHeaderModifier more_set_headers Extended
URLRewrite (hostname, ReplacePrefixMatch/ReplaceFullPath) rewrite-target, upstream-vhost Extended
RequestMirror (backendRef, percent/fraction) mirror-target Extended
CORS enable-cors, cors-* Extended
ExtensionRef → EG HTTPRouteFilter Regex rewrite with capture group Implementation
rules[].timeouts.request / backendRequest proxy-read-timeout Extended
rules[].retry (codes, attempts, backoff) – Experimental
backendRefs[].weight canary-weight Core
  • Redirect and rewrite are mutually exclusive in one rule.
  • ReplacePrefixMatch only together with a PathPrefix match.
  • retry.attempts: 3 = three retries, up to four attempts.
  • weight: 0 = no traffic, but the backend stays in the route for the rollback.

Status Conditions

Object Condition (reason) Meaning
GatewayClass Accepted=Unknown (Pending) no controller with this controllerName
Gateway Accepted=True, Programmed=True spec valid, data plane ready with an address
Gateway Accepted=True (ListenersNotValid) at least one listener broken, the rest is running
Listener ResolvedRefs=False (InvalidCertificateRef) Secret missing or invalid
Listener ResolvedRefs=False (RefNotPermitted) Secret in another namespace without a ReferenceGrant
Route Accepted=False (NotAllowedByListeners) namespace/kind not in allowedRoutes
Route Accepted=False (NoMatchingListenerHostname) hostnames do not intersect
Route Accepted=False (NoMatchingParent) sectionName/port matches no listener
Route ResolvedRefs=False (BackendNotFound) Service missing; Envoy responds with 500
Route ResolvedRefs=False (RefNotPermitted) backend in another namespace without a ReferenceGrant
Route RouteRulesOverlap=True equally specific rules, the oldest wins
Policy Accepted=False in status.ancestors wrong target or conflict with another policy
ListenerSet Accepted=False (NotAllowed) allowedListeners on the Gateway forbids it
(no status at all) – Gateway/parent does not exist or foreign controller

Always compare observedGeneration with metadata.generation: if it is lower, the controller has not seen the current spec yet.

Envoy Response Flags (Access Log flags)

Flag Meaning Typical cause
UH no healthy upstream Service without endpoints (canary on a dead v2)
UF upstream connection failed wrong port, pod starting
UT upstream timeout timeouts.request exceeded
URX retry limit reached retry/numRetries used up
RL locally or globally rate-limited (429) BackendTrafficPolicy rateLimit
RLSE rate limit service unreachable Redis/envoy-ratelimit gone
NR no route hostname/path without a rule (404)
- no flag normal request or direct response

Envoy Gateway Policies and Targets

All gateway.envoyproxy.io/v1alpha1. One policy applies per target and policy type; the more specific one (route before Gateway) wins.

Resource Target (targetRefs) Used for Day
EnvoyProxy via parametersRef on the Gateway replicas, HPA, PDB, Service, telemetry, shutdown 9–12
ClientTrafficPolicy Gateway, listener (sectionName) client timeouts, HTTP/2, client IP, Proxy Protocol 10
BackendTrafficPolicy Gateway, route, rule rate limit, retries, circuit breaker, health checks 10
SecurityPolicy Gateway, route, rule basic auth, JWT, OIDC, CORS, IP rules, extAuth 10
HTTPRouteFilter via ExtensionRef from the route regex rewrite, direct response 13
Backend as a backendRef FQDN/IP backends (needs enableBackend) 10
EnvoyPatchPolicy Gateway raw xDS patches, last resort –
BackendTLSPolicy (Gateway API) Service, Backend CA and SNI towards the backend 4, 10
Helm value (config.envoyGateway) Effect
rateLimit.backend.redis.url global rate limit, starts envoy-ratelimit
rateLimit.failClosed default false: Redis gone = no limit; true = 500
rateLimit.timeout default 20 ms to the rate limit service
extensionApis.enableBackend enable the Backend CRD
extensionManager.backendResources foreign backend kinds, e.g. InferencePool

Lab Scripts and kubectl

# Address of a Gateway (default: web in infra), compact conditions
gw-ip.sh; gw-ip.sh web-cilium infra
conds.sh gatewayclass eg
conds.sh gateway web infra
conds.sh httproute shop shop
conds.sh securitypolicy api-jwt shop        # policies: status.ancestors
# curl pattern in kind: bend the name, Host header and SNI stay
GW_IP=$(gw-ip.sh)
curl -s --cacert ~/gw-lab/tag04/lab-ca.crt \
  --resolve shop.gw.localtest.me:443:$GW_IP https://shop.gw.localtest.me/
# RKE2: real DNS, separate lab CA
export KUBECONFIG=~/.kube/rke2-lab.yaml
curl -s --cacert ~/gw-lab/tag09/rke2-lab-ca.crt https://shop.apps.lab.internal/
# API state and fields
kubectl api-resources --api-group=gateway.networking.k8s.io
kubectl explain httproute.spec.rules.filters.type
~/gw-lab/code/tag12/crd-check.sh            # bundle, channel, served versions
# All routes claiming a hostname
kubectl get httproute -A -o json | jq -r --arg h shop.apps.lab.internal \
  '.items[] | select(.spec.hostnames // [] | index($h))
   | "\(.metadata.namespace)/\(.metadata.name)"'
# Envoy pods and Service of a Gateway
kubectl -n envoy-gateway-system get deploy,svc,hpa,pdb \
  -l gateway.envoyproxy.io/owning-gateway-name=web
# Access log (JSON from Day 11): status, flags, route
kubectl -n envoy-gateway-system logs -l gateway.envoyproxy.io/owning-gateway-name=web \
  -c envoy --since=5m | grep '^{' | jq -r '"\(.status) \(.flags) \(.route)"' \
  | sort | uniq -c

egctl

egctl version --remote=false
egctl x status all -A --quiet | grep -v True       # only anomalies
egctl x status httproute -n shop
# What Envoy really has (pod name without "pod/")
POD=$(kubectl -n envoy-gateway-system get pod -o name \
  -l gateway.envoyproxy.io/owning-gateway-name=web | head -1); POD=${POD#pod/}
egctl config envoy-proxy route -n envoy-gateway-system "$POD"
egctl config envoy-proxy cluster -n envoy-gateway-system "$POD"
egctl config envoy-gateway all -n envoy-gateway-system
egctl x dashboard envoy-proxy -n envoy-gateway-system "$POD"   # admin UI

Envoy cluster names: httproute/<ns>/<name>/rule/<n>, grpcroute/<ns>/<name>/rule/<n>. Envoy Deployment: envoy-<namespace>-<gateway>-<hash> in the namespace envoy-gateway-system, so for infra/web it is envoy-infra-web-a97fc3bf.

cert-manager

Annotation on the Gateway Effect
cert-manager.io/cluster-issuer: lab-ca one Certificate per HTTPS listener (gateway-shim)
cert-manager.io/private-key-algorithm, -size e.g. ECDSA, 256
cert-manager.io/duration, renew-before e.g. 720h, 240h

Prerequisites: config.gatewayAPI.enabled=true, Gateway API CRDs before cert-manager starts, listener with hostname and tls.mode: Terminate. Secret name = certificateRefs[0].name, SAN = listener hostname.

kubectl wait clusterissuer/lab-ca --for=condition=Ready --timeout=60s
kubectl -n infra get certificate
cmctl status certificate -n infra apps-wildcard-tls
cmctl renew -n infra apps-wildcard-tls          # force rotation
~/gw-lab/code/tag12/cert-check.sh               # served = Secret?
openssl s_client -connect 10.10.20.100:443 -servername shop.apps.lab.internal \
  </dev/null 2>/dev/null | openssl x509 -noout -subject -enddate -checkend 1209600

Cilium LB IPAM and L2

Object / field Lab value Remember
CiliumLoadBalancerIPPool (cilium.io/v2) lab-pool, .100–.120 kubectl get ippools
CiliumL2AnnouncementPolicy (v2alpha1) lab-l2, workers only, ^eth0$ at least two candidates
Annotation lbipam.cilium.io/ips .100 web, .101 web-cilium, .110 legacy fixed IP per Service
externalTrafficPolicy Cluster mandatory with L2 (Day 9)
Lease cilium-l2announce-<ns>-<svc> in kube-system holder = ARP responder
kubectl get ippools
kubectl -n kube-system get lease | grep l2announce
kubectl get ciliuml2announcementpolicy lab-l2 -o yaml
arping -c 2 10.10.20.100                       # MAC = lease holder
kubectl -n kube-system exec ds/cilium -c cilium-agent -- cilium-dbg status \
  | grep KubeProxyReplacement

PromQL

# RPS per route rule
sum by (envoy_cluster_name) (rate(envoy_cluster_upstream_rq_total{
  envoy_cluster_name=~"httproute/.*"}[5m]))
# 5xx rate
sum by (envoy_cluster_name) (rate(envoy_cluster_upstream_rq_xx{
  envoy_response_code_class="5"}[5m]))
/ sum by (envoy_cluster_name) (rate(envoy_cluster_upstream_rq_xx[5m]))
# p95 in milliseconds
histogram_quantile(0.95, sum by (envoy_cluster_name, le) (
  rate(envoy_cluster_upstream_rq_time_bucket[5m])))
# Certificate from Envoy's point of view
min(envoy_server_days_until_first_cert_expiring) < 14
# Status via kube-state-metrics (Custom Resource State, Day 11)
gatewayapi_gateway_status{type="Programmed"} == 0
gatewayapi_httproute_accepted == 0
# Controller: xDS rejected
increase(xds_nack_total[10m]) > 0

Recording rules from Day 11: route:rps:rate5m, route:error5xx:ratio5m, route:latency:p95_5m. Alerts: GatewayNotProgrammed, HTTPRouteNotAccepted, RouteHigh5xx, EnvoyGatewayXdsNack.

ingress2gateway

ingress2gateway print --providers=ingress-nginx -n shop --no-color > out.yaml
ingress2gateway print --providers=ingress-nginx --input-file legacy-ingress.yaml -A
ingress2gateway print --providers=ingress-nginx -n shop --emitter envoy-gateway
ingress2gateway print --providers=traefik --input-file traefik-ingress.yaml -A
ingress-nginx annotation Gateway API / Envoy Gateway i2gw
canary, canary-weight, canary-by-header backendRefs[].weight, header matches yes
rewrite-target without / with $1 URLRewrite / EG HTTPRouteFilter yes / no
use-regex RegularExpression, better PathPrefix yes
ssl-redirect, permanent-redirect RequestRedirect yes
enable-cors, cors-* Filter CORS yes
proxy-*-timeout timeouts (check the value!) partly
proxy-body-size EG BackendTrafficPolicy.requestBuffer emitter
whitelist-source-range EG SecurityPolicy.authorization emitter
auth-url, auth-type: basic EG SecurityPolicy.extAuth / basicAuth no
limit-rps EG BackendTrafficPolicy.rateLimit no
backend-protocol: GRPC GRPCRoute yes
ssl-passthrough TLSRoute + passthrough listener yes
*-snippet Filter or EnvoyPatchPolicy, case by case no

Process: inventory → convert → rework → run in parallel (--resolve to both IPs) → switch DNS (lower the TTL beforehand) → observe → decommission. Rollback until the very end: DNS record back to 10.10.20.110.