Skip to content
Hands-on IT IT Drills

Day 7: Resources and Quotas

Goal: You give containers suitable requests and limits and work in namespaces with LimitRange and ResourceQuota. · Duration: approx. 60 min · Prerequisite: Day 1

Briefing

CKAD: Application Environment, Configuration and Security (25 %) – “Understand requests, limits, quotas”, “Define resource requirements”.

  • Request: a reservation; the scheduler places pods based on it.
  • Limit: an upper bound at runtime. CPU above it is throttled, memory above it kills the container (OOMKilled).
  • LimitRange: defaults and bounds per container in the namespace.
  • ResourceQuota: the total for the whole namespace.
mkdir -p ~/ckad/tag07 && cd ~/ckad/tag07
k create ns tag07
k config set-context --current --namespace=tag07

Drill 1: Requests, limits and QoS classes

k run be --image=nginx:1.29
k create deployment web --image=nginx:1.29 --replicas=2
k set resources deploy/web --requests=cpu=100m,memory=64Mi \
  --limits=cpu=200m,memory=128Mi
cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: gold
spec:
  containers:
  - name: nginx
    image: nginx:1.29
    resources:
      requests:
        cpu: 100m
        memory: 64Mi
      limits:
        cpu: 100m
        memory: 64Mi
EOF
k rollout status deploy/web
k get pods -o custom-columns=NAME:.metadata.name,QOS:.status.qosClass

Expected (shortened):

NAME                   QOS
be                     BestEffort
gold                   Guaranteed
web-6d46f9dccf-9zz75   Burstable
web-6d46f9dccf-vvstx   Burstable
QoS Condition Under memory pressure
Guaranteed requests = limits for CPU and RAM, every container evicted last
Burstable any request or limit set in between
BestEffort nothing set at all evicted first

To query a single pod: k get pod gold -o jsonpath='{.status.qosClass}'.

Save your progress? Sign up for free

Drill 2: OOMKilled

tail /dev/zero never finds the end of a line and keeps more and more in memory.

cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: hog
spec:
  restartPolicy: Never
  containers:
  - name: hog
    image: busybox:1.37
    command: ["tail", "/dev/zero"]
    resources:
      limits:
        memory: 50Mi
EOF
k get pod hog
k describe pod hog | grep -A3 "State:"

Expected (after a few seconds):

NAME   READY   STATUS      RESTARTS   AGE
hog    0/1     OOMKilled   0          11s
    State:          Terminated
      Reason:       OOMKilled
      Exit Code:    137

Exit code 137 = 128 + 9 (SIGKILL). With restartPolicy: Always you would see CrashLoopBackOff; the reason then appears under Last State.

Save your progress? Sign up for free

Drill 3: CPU is throttled, not killed

cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: burn
spec:
  containers:
  - name: burn
    image: busybox:1.37
    command: ["sh", "-c", "while true; do :; done"]
    resources:
      requests:
        cpu: 100m
      limits:
        cpu: 200m
EOF
k wait --for=condition=Ready pod/burn

After about a minute (metrics-server collects at intervals):

k top pod burn
k exec burn -- cat /sys/fs/cgroup/cpu.max
k exec burn -- grep nr_throttled /sys/fs/cgroup/cpu.stat

Expected (shortened):

NAME   CPU(cores)   MEMORY(bytes)
burn   200m         0Mi
20000 100000
nr_throttled 619

The endless loop wants a whole core but only gets 20 ms per 100 ms. The pod keeps running, just slower – for web servers this shows up as high latency.

Save your progress? Sign up for free

Drill 4: Change resources in a running pod

Stable since Kubernetes 1.35: in-place resize via the resize subresource.

k patch pod burn --subresource resize \
  -p '{"spec":{"containers":[{"name":"burn","resources":{"limits":{"cpu":"300m"}}}]}}'
k get pod burn -o jsonpath='{.status.containerStatuses[0].resources}{"\n"}'
k get pod burn -o jsonpath='{.status.containerStatuses[0].restartCount}{"\n"}'
k exec burn -- cat /sys/fs/cgroup/cpu.max

Expected:

pod/burn patched
{"limits":{"cpu":"300m"},"requests":{"cpu":"100m"}}
0
30000 100000

No restart. A normal k patch without --subresource resize fails with pod updates may not change fields other than …. Whether a resize restarts the container is controlled by resizePolicy on the container (NotRequired or RestartContainer). For Deployments you change the template instead – which creates new pods.

k delete pod hog burn be gold --now
k delete deploy web

Save your progress? Sign up for free

Drill 5: ResourceQuota

k create quota team-quota --hard=requests.cpu=500m,requests.memory=512Mi,\
limits.cpu=1,limits.memory=1Gi,pods=4,count/deployments.apps=2
k describe quota team-quota
k run naked --image=nginx:1.29

Expected:

resourcequota/team-quota created
Name:                   team-quota
Namespace:              tag07
Resource                Used  Hard
--------                ----  ----
count/deployments.apps  0     2
limits.cpu              0     1
limits.memory           0     1Gi
pods                    0     4
requests.cpu            0     500m
requests.memory         0     512Mi
Error from server (Forbidden): pods "naked" is forbidden: failed quota: team-quota:
must specify limits.cpu for: naked; limits.memory for: naked; requests.cpu for: naked;
requests.memory for: naked

As soon as a quota counts CPU or memory, every new pod must specify the corresponding values. count/<resource>.<group> counts arbitrary objects.

Save your progress? Sign up for free

Drill 6: LimitRange supplies defaults

cat <<'EOF' | k apply -f -
apiVersion: v1
kind: LimitRange
metadata:
  name: defaults
spec:
  limits:
  - type: Container
    defaultRequest:
      cpu: 50m
      memory: 32Mi
    default:
      cpu: 100m
      memory: 64Mi
    max:
      cpu: 500m
      memory: 256Mi
EOF
k run naked --image=nginx:1.29
k get pod naked -o jsonpath='{.spec.containers[0].resources}{"\n"}'

Expected:

limitrange/defaults created
pod/naked created
{"limits":{"cpu":"100m","memory":"64Mi"},"requests":{"cpu":"50m","memory":"32Mi"}}

default are limits, defaultRequest are requests. They are filled in at creation time; existing pods do not change. max is also enforced at creation time:

k run big --image=nginx:1.29 \
  --overrides='{"spec":{"containers":[{"name":"big","image":"nginx:1.29",
  "resources":{"limits":{"cpu":"1"}}}]}}'

Expected:

Error from server (Forbidden): pods "big" is forbidden: maximum cpu usage per Container
is 500m, but limit is 1

Save your progress? Sign up for free

Drill 7: When the quota is full

k create deployment api --image=nginx:1.29 --replicas=5
k get deploy api
k get events --field-selector reason=FailedCreate | tail -1
k describe quota team-quota | grep pods

Expected (shortened):

NAME   READY   UP-TO-DATE   AVAILABLE   AGE
api    3/5     3            3           6s
... Error creating: pods "api-5f8476ddb6-j5h7g" is forbidden: exceeded quota:
team-quota, requested: pods=1, used: pods=4, limited: pods=4
pods                    4      4

The Deployment is created, but the ReplicaSet does not get all its pods – the error only shows up in the events, not on the Deployment. Now request more CPU:

k scale deploy api --replicas=2
k set resources deploy api --requests=cpu=300m,memory=64Mi \
  --limits=cpu=400m,memory=128Mi
k get rs -l app=api
k get events --field-selector reason=FailedCreate | tail -1

Expected (after a few seconds, shortened):

NAME             DESIRED   CURRENT   READY   AGE
api-5f8476ddb6   1         1         1       22s
api-7c85f58585   2         1         1       8s
... exceeded quota: team-quota, requested: requests.cpu=300m, used: requests.cpu=400m,
limited: requests.cpu=500m

The rollout is stuck: one new pod with 300m still fit (50m naked + 50m old pod + 300m), the second one did not. The fix is smaller requests, fewer replicas or a larger quota.

Save your progress? Sign up for free

Exercise: Speed Drills

  1. In the Deployment api, set requests to cpu=50m,memory=32Mi and limits to cpu=100m,memory=64Mi. Target time: 30 s
  2. Create a quota small that allows at most 3 ConfigMaps and 2 Services. Target time: 45 s
  3. Show the pod with the highest memory usage in the namespace. Target time: 20 s

Solution

k set resources deploy api --requests=cpu=50m,memory=32Mi \
  --limits=cpu=100m,memory=64Mi
k create quota small --hard=configmaps=3,services=2
k top pod --sort-by=memory

Expected (shortened):

deployment.apps/api resource requirements updated
resourcequota/small created
NAME                   CPU(cores)   MEMORY(bytes)
api-66669dd9b9-9cgxr   0m           3Mi
naked                  0m           3Mi

Save your progress? Sign up for free

Checkpoint

k rollout status deploy/api

Expected: deployment "api" successfully rolled out

k get pod naked -o jsonpath='{.status.qosClass}{"\n"}'

Expected: Burstable

k describe quota small

Expected: configmaps with Used 1 (the automatic kube-root-ca.crt) and Hard 3, services with 0 and 2.

Challenge

A pod report should run with Guaranteed QoS but have only limits (cpu: 200m, memory: 64Mi) in its manifest. Does that work despite the LimitRange with defaultRequest? Create it and check.

Hint: Where does a missing request come from when a limit is set?

Solution

cat <<'EOF' | k apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: report
spec:
  containers:
  - name: report
    image: busybox:1.37
    command: ["sleep", "3600"]
    resources:
      limits:
        cpu: 200m
        memory: 64Mi
EOF
k get pod report -o jsonpath='{.spec.containers[0].resources.requests}{"\n"}'
k get pod report -o jsonpath='{.status.qosClass}{"\n"}'

Expected:

{"cpu":"200m","memory":"64Mi"}
Guaranteed

If a request is missing, Kubernetes copies the limit – before the LimitRange’s defaultRequest applies. That makes requests and limits equal: Guaranteed.

Clean up:

k config set-context --current --namespace=default
k delete ns tag07

Save your progress? Sign up for free

Exam Tip

  • k set resources instead of editing YAML; for Deployments, target a container with -c <container>.
  • Pod not showing up? With quotas the reason is in k get events, not in k get pods.
  • Units: 100m = 0.1 core; Mi (1024²) instead of M (1000²). 1m of memory means a milli-byte – a common typo.

Enterprise Note

  • Every team namespace gets a LimitRange and a ResourceQuota, often via a template or a policy engine.
  • CPU limits are controversial: many teams set only requests to avoid throttling; memory limits remain mandatory.
  • Set requests based on real measurements (Prometheus, Vertical Pod Autoscaler in recommendation mode).
  • PriorityClasses determine who gets evicted first when resources are scarce.