Day 8: Rate Limiting and Protection
Goal: Limit requests per minute, per client IP and per consumer, reject large requests, restrict the Admin API to your own machine and lock out bots. · Duration: approx. 60 min · Prerequisite: Day 7
Briefing
rate-limitingcounts requests per time window and answers with 429 once the limit is reached.limit_bydefines whom Kong counts:ip,consumer,credential,header.policydefines where counting happens:localin a node’s memory,clusterin Postgres,redisin Redis.request-size-limiting,ip-restrictionandbot-detectionreject requests before they cost the backend anything.- From today on: change the file, then
ksync(decK sync, then wait 10 seconds).
Request ─► key-auth/jwt ─► request-size-limiting (951) ─► acl (950)
─► rate-limiting (910) ─► Backend 429 / 413 / 403 earlier
Drill 1: A Limit for Reading
source ~/kong-lab/setup/env.sh
mkdir -p ~/kong-lab/tag08 && cd ~/kong-lab/tag08
cp ~/kong-lab/code/tag08/*.yaml .
grep -A6 'name: rate-limiting' limit-lesen.yaml
deck file add-plugins -s ~/kong-lab/kong/kong.yaml limit-lesen.yaml \
-o ~/kong-lab/kong/kong.yaml
ksync | tail -3
for i in $(seq 1 7); do
curl -s -o /dev/null -w '%{http_code} ' $PROXY/api/v1/notes
done
echoExpected:
- name: rate-limiting
config:
minute: 5
policy: local
limit_by: ip
Created: 1
Updated: 0
Deleted: 0
200 200 200 200 200 429 429
decK may write to the same file it reads. ksync applies
it and waits until the workers have it. decK prints the warning about
passwords (Day 5) on every call; from here on, the book leaves it out.
The sixth request in this minute gets 429.
Save your progress? Sign up for free
Drill 2: What the Client Learns
curl -si $PROXY/api/v1/notes | grep -i -E '^(HTTP|ratelimit|x-ratelimit|retry-after)'
curl -s $PROXY/api/v1/notesExpected (shortened, your seconds will differ):
HTTP/1.1 429 Too Many Requests
Retry-After: 55
X-RateLimit-Remaining-Minute: 0
X-RateLimit-Limit-Minute: 5
RateLimit-Remaining: 0
RateLimit-Limit: 5
RateLimit-Reset: 55
{
"message":"API rate limit exceeded",
"request_id":"69065b54…"
}
Retry-After tells the client when it may try again.
Well-built clients wait that long instead of asking again immediately.
The RateLimit-* headers also come along with successful
responses so that clients can slow down beforehand.
Save your progress? Sign up for free
Drill 3: Per Consumer, Counted in Postgres
For writing, what counts is not the IP but who writes.
limit-schreiben.yaml sets 3 per minute per consumer with
policy: cluster, plus a size limit of 1 KB:
deck file add-plugins -s ~/kong-lab/kong/kong.yaml limit-schreiben.yaml \
-o ~/kong-lab/kong/kong.yaml
ksync | tail -3
w() {
curl -s -o /dev/null -w '%{http_code} ' -X POST -H 'Content-Type: application/json' \
"$@" -d '{"text":"limit"}' $PROXY/api/v1/notes
}
for i in 1 2 3 4; do w -H 'apikey: lab-key-anna-456'; done; echo
T=$(token); for i in 1 2 3 4; do w -H "Authorization: Bearer $T"; done; echo
cd ~/kong-lab/kong
docker compose exec kong-db psql -U kong -d kong \
-c "select identifier, period, value from ratelimiting_metrics"Expected (shortened, your IDs will differ):
Created: 2
Updated: 0
Deleted: 0
201 201 201 429
201 201 201 429
identifier | period | value
--------------------------------------+--------+-------
2e1392da-3e39-4aad-94d6-e63ee3aa6bf0 | minute | 3
2b3ac38f-8971-4689-9a0c-c282dfd89a92 | minute | 3
(2 rows)
anna and the consumer keycloak have separate counters,
recognizable by their IDs. The rejected fourth request does not
count.
policy |
Counter lives | Good for |
|---|---|---|
local |
in each node’s memory | one node, or rough limits |
cluster |
in Postgres | several nodes with a database, low load |
redis |
in Redis | several nodes, high load, DB-less too |
With three nodes and local, a client is effectively
allowed three times as much. cluster puts load on the
database with every request, which is why redis is the
usual choice for heavy traffic.
Save your progress? Sign up for free
Drill 4: Requests That Are Too Large
cd ~/kong-lab/tag08
head -c 2000 /dev/zero | tr '\0' 'a' > gross.txt
curl -s -w ' %{http_code}\n' -X POST -H 'apikey: lab-key-anna-456' \
-H 'Content-Type: application/json' \
--data-binary "{\"text\":\"$(cat gross.txt)\"}" $PROXY/api/v1/notesExpected (shortened):
{
"message":"Request size limit exceeded",
"request_id":"a9913371…"
} 413
2 KB is more than the one allowed. request-size-limiting
has priority 951 and runs before rate-limiting: even
someone who has already used up their limit gets 413 here, not 429. The
backend never sees the request.
Save your progress? Sign up for free
Drill 5: The Admin API Only from the VM
Since Day 5, the Admin API has also been reachable from the network
via /admin-api, with a password. Better: only from the VM
itself. First attempt with 127.0.0.1:
grep -A3 'name: ip-restriction' admin-ip.yaml
deck file add-plugins -s ~/kong-lab/kong/kong.yaml admin-ip.yaml \
-o ~/kong-lab/kong/kong.yaml
ksync | tail -3
curl -s -w ' %{http_code}\n' -u anna:lab-user-pw-123 $PROXY/admin-api/statusExpected:
- name: ip-restriction
config:
allow: ["127.0.0.1"]
Created: 1
Updated: 0
Deleted: 0
{
"message":"IP address not allowed: 172.18.0.1",
"request_id":"4d7e0431…"
} 403
Even you no longer get through. Your requests to
localhost:8000 reach Kong via the Docker network, from the
gateway of the lab network. The message reveals the
address. Enter exactly that one (yours may differ, for example
172.19.0.1):
P='$.services[?(@.name=="admin-api")].plugins[?(@.name=="ip-restriction")]'
deck file patch -s ~/kong-lab/kong/kong.yaml --selector "$P.config" \
--value 'allow:["172.18.0.1"]' -o ~/kong-lab/kong/kong.yaml
ksync | tail -3
curl -s -o /dev/null -w '%{http_code}\n' -u anna:lab-user-pw-123 $PROXY/admin-api/statusExpected:
Created: 0
Updated: 1
Deleted: 0
200
From the laptop, via the VM’s IP, the door stays shut. The regular API keeps working:
curl -s -w ' %{http_code}\n' -u anna:lab-user-pw-123 \
http://$KONG_IP:8000/admin-api/status
curl -s -o /dev/null -w '%{http_code}\n' http://$KONG_IP:8000/api/v1/notesExpected (shortened, your address will differ):
{
"message":"IP address not allowed: 192.168.64.1",
"request_id":"e579d75e…"
} 403
200
Docker passes the client’s real address on to Kong. If there is a
load balancer in front, Kong sees its address; then you need
trusted_ips and real_ip_header.
Save the state:
cd ~/kong-lab/kong
git add kong.yaml && git commit -qm "Tag 8: Rate Limits, Größe, IP-Sperre"Save your progress? Sign up for free
Checkpoint
curl -s $ADMIN/plugins | jq -r '.data[] | select(.name | test("rate|size|ip-"))
| .name' | sortExpected: three new plugin types, rate-limiting
twice.
ip-restriction
rate-limiting
rate-limiting
request-size-limiting
curl -si $PROXY/api/v1/notes | grep -i '^ratelimit-limit'Expected: the limit for reading.
RateLimit-Limit: 5
deck gateway diff ~/kong-lab/kong/kong.yaml 2>/dev/null | tail -3Expected: file and database match.
Created: 0
Updated: 0
Deleted: 0
Challenge
Requests with the user agent sqlmap, a tool for SQL
injection, show up in the logs. Lock it out of the echo
service. First check what the built-in list of
bot-detection already rejects.
Note: bot-detection has
allow and deny with regular expressions;
curl -A sets the user agent.
Solution
cd ~/kong-lab/tag08
deck file add-plugins -s ~/kong-lab/kong/kong.yaml \
--selector '$.services[?(@.name=="echo")]' \
--config '{"name":"bot-detection","config":{}}' -o ~/kong-lab/kong/kong.yaml
ksync | tail -1
for ua in curl/8.5.0 Googlebot/2.1 sqlmap/1.8; do
printf '%-15s' $ua; curl -s -o /dev/null -w '%{http_code}\n' -A $ua $PROXY/echo/get
doneExpected:
Deleted: 0
curl/8.5.0 200
Googlebot/2.1 403
sqlmap/1.8 200
The built-in list knows search engines and crawlers, but not
sqlmap. So add it yourself:
P='$.services[?(@.name=="echo")].plugins[?(@.name=="bot-detection")]'
deck file patch -s ~/kong-lab/kong/kong.yaml --selector "$P.config" \
--value 'deny:["(?i)sqlmap"]' -o ~/kong-lab/kong/kong.yaml
ksync | tail -1
curl -s -o /dev/null -w '%{http_code}\n' -A sqlmap/1.8 $PROXY/echo/get
git -C ~/kong-lab/kong commit -qam "echo: sqlmap gesperrt"Expected:
Deleted: 0
403
A user agent is a claim made by the client.
bot-detection stops casual bots, not an attacker who
lies.
Save your progress? Sign up for free
Enterprise Note
- rate-limiting-advanced (Enterprise) counts with sliding windows, several limits per plugin, and consumer groups with their own limits.
- Redis for counters must be run highly available; if
it fails,
fault_tolerantdecides whether Kong lets requests through or rejects them. - Behind a load balancer, set
trusted_ipsandreal_ip_header, otherwiselimit_by: ipcounts all clients as one. - Protection in layers: Rate limits in the gateway do not replace DDoS protection in front of it or limits in the backend.