Day 8: Validate
Goal: Enforce Pod Security, labels, resources, image tags and registries with
ValidatingPolicy, and write messages that teams understand immediately. · Duration: approx. 90 min · Prerequisite: Day 7
Briefing
- A
ValidatingPolicy= selection (matchConstraints,matchConditions) + helper values (variables) + checks (validations) with CEL. namespaceSelectornarrows by namespace label,matchConditionsvia CEL on the object. Both reduce webhook load and false alarms.messageExpressionbuilds the error message dynamically: container name, image, value.- Kyverno ships its own CEL libraries, e.g.
image()andresource.Get(). - The demo Pods from Day 0 meet “restricted”. Everything that gets rejected today is broken on purpose.
Drill 1: Enforce Pod Security “baseline”
mkdir -p ~/guard-lab/tag08 && cd ~/guard-lab/tag08
kubectl delete vpol require-app-label --ignore-not-found
cp ~/guard-lab/code/kyverno/tag08/*.yaml .
sed -n '/variables:/,$p' pss-baseline.yamlExpected:
variables:
- name: allContainers
expression: >-
object.spec.containers
+ object.spec.?initContainers.orValue([])
+ object.spec.?ephemeralContainers.orValue([])
validations:
- expression: >-
!object.spec.?hostNetwork.orValue(false) &&
!object.spec.?hostPID.orValue(false) &&
!object.spec.?hostIPC.orValue(false)
message: "hostNetwork, hostPID und hostIPC sind verboten."
- expression: >-
variables.allContainers.all(c,
!c.?securityContext.?privileged.orValue(false))
messageExpression: >-
'privileged ist verboten: ' + variables.allContainers
.filter(c, c.?securityContext.?privileged.orValue(false))
.map(c, c.name).join(', ')
- expression: >-
object.spec.?volumes.orValue([]).all(v, !has(v.hostPath))
message: "hostPath-Volumes sind verboten."
- expression: >-
variables.allContainers.all(c, c.?ports.orValue([])
.all(p, p.?hostPort.orValue(0) == 0))
message: "hostPort ist verboten."Why .? and orValue()? Optional fields are
usually missing. object.spec.hostPID would then throw an
error; object.spec.?hostPID.orValue(false) cleanly returns
false. The variable allContainers makes sure
nobody dodges via an init or debug container. The policy applies to all
namespaces except kube-system.
kubectl apply -f pss-baseline.yaml
kubectl apply -f bad-pods.yamlExpected:
validatingpolicy.policies.kyverno.io/pss-baseline created
pod/lazy created
Error from server: error when creating "bad-pods.yaml": admission webhook
"vpol.validate.kyverno.svc-fail-e52f2f0f" denied the request: Policy pss-baseline
failed: hostNetwork, hostPID und hostIPC sind verboten.
root-shell (hostPID + privileged) is out;
lazy is fine for “baseline”.
Save your progress? Sign up for free
Drill 2: Only the first violation counts
A Pod with two problems: Container b is privileged, and
there is a hostPath volume.
cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata: {name: priv, namespace: tenant-a, labels: {app: priv}}
spec:
containers:
- {name: a, image: "nginxinc/nginx-unprivileged:1.31-alpine"}
- name: b
image: nginxinc/nginx-unprivileged:1.31-alpine
securityContext: {privileged: true}
volumes:
- name: host
hostPath: {path: /etc}
EOFExpected:
Error from server: error when creating "STDIN": admission webhook
"vpol.validate.kyverno.svc-fail-e52f2f0f" denied the request: Policy pss-baseline
failed: privileged ist verboten: b
The message names the container, but not the hostPath volume. Kyverno 1.19 stops after the first failed validation of a policy. So the team fixes one error and runs into the next.
Tip: Order
validationsfrom the most common to the rarest error. Or collect all violations in a variable and check them in a single validation (Drill 5).
Save your progress? Sign up for free
Drill 3: “restricted” in audit first
“restricted” requires runAsNonRoot,
allowPrivilegeEscalation: false,
capabilities.drop: [ALL] and a seccomp profile. That breaks
many existing workloads, so measure first.
grep -E 'validationActions|message:' pss-restricted.yaml
kubectl apply -f pss-restricted.yaml
sleep 30
kubectl -n shop get polr --no-headers | awk '{print $2, $3, "pass="$4, "fail="$5}' \
| grep -E 'lazy|frontend'Expected (abridged):
validationActions: [Audit]
message: "runAsNonRoot: true fehlt (Pod oder Container)."
message: "allowPrivilegeEscalation: false fehlt."
message: "capabilities.drop muss ALL enthalten."
message: "seccompProfile RuntimeDefault oder Localhost fehlt."
validatingpolicy.policies.kyverno.io/pss-restricted created
Pod frontend-5f9bb6b549-7g86l pass=2 fail=0
Deployment frontend pass=2 fail=0
ReplicaSet frontend-5f9bb6b549 pass=2 fail=0
Pod lazy pass=0 fail=1
frontend passes both policies (pass=2),
lazy fails “restricted”. The variable
podNonRoot allows both: runAsNonRoot at Pod
level (as in the demo app) or per container.
kubectl -n shop delete pod lazyTip: Kyverno also ships the PSS controls ready-made as a policy library (
helm install kyverno-policies kyverno/kyverno-policies). That is enough to get started; for production you want to control messages and exceptions yourself.
Save your progress? Sign up for free
Drill 4: Labels as a contract
Day 3 made labels a contract between teams: NetworkPolicies select by
team, app and tier. Now you
enforce them. Version 1 narrows via matchConditions:
sed -n '/matchConditions/,$p' require-labels.yamlExpected:
matchConditions:
- name: nur-demo-objekte
expression: >-
object.kind == 'Deployment' ||
object.metadata.?labels[?'guard-lab'].orValue('') == 'demo'
variables:
- name: required
expression: >-
object.kind == 'Namespace' ? ['team'] : ['app', 'tier']
- name: missing
expression: >-
variables.required.filter(l,
!(l in object.metadata.?labels.orValue({})))
validations:
- expression: "size(variables.missing) == 0"
messageExpression: >-
object.kind + ' ' + object.metadata.name + ': Labels fehlen: '
+ variables.missing.join(', ')
reason: InvalidOne policy, two resource types: The variable required
decides depending on kind. Watch the syntax
labels[?'guard-lab']: For keys with a hyphen or slash only
index access works; labels.?guard-lab is a syntax
error.
kubectl apply -f require-labels.yaml
kubectl apply -f label-objs.yamlExpected:
validatingpolicy.policies.kyverno.io/require-labels created
namespace/scratch created
Error from server: error when creating "label-objs.yaml": admission webhook
"vpol.validate.kyverno.svc-fail-finegrained-require-labels" denied the request: Policy
require-labels failed: Namespace tenant-c: Labels fehlen: team
Error from server: error when creating "label-objs.yaml": admission webhook
"vpol.validate.kyverno.svc-fail-finegrained-require-labels" denied the request: Policy
require-labels failed: Deployment worker: Labels fehlen: tier
scratch has no guard-lab=demo and is free.
With matchConditions, Kyverno registers a separate
“finegrained” webhook for the policy. Now the catch. Wait for the
background scan:
sleep 30
kubectl get polr -A -o json | jq -r '.items[].results[]?
| select(.result=="fail" and .policy=="require-labels") | .message'Expected (abridged):
Deployment coredns: Labels fehlen: app, tier
Deployment kyverno-admission-controller: Labels fehlen: app, tier
Deployment kyverno-background-controller: Labels fehlen: app, tier
...
The condition object.kind == 'Deployment' applies in
every namespace. In the report that is just ugly. But the next
helm upgrade kyverno would be rejected by its own policy.
Version 2 narrows with namespaceSelector. For namespace
objects, the selector checks the labels of the namespace itself, so one
rule is enough for both types:
diff require-labels.yaml require-labels-v2.yaml | grep '^[<>]' | head -8
kubectl apply -f require-labels-v2.yaml
kubectl delete ns scratch
kubectl apply -f label-objs.yaml --dry-run=server 2>&1 | cut -c1-60Expected (abridged):
< matchConditions:
< - name: nur-demo-objekte
< expression: >-
< object.kind == 'Deployment' ||
< object.metadata.?labels[?'guard-lab'].orValue('') == 'demo'
> namespaceSelector:
> matchLabels:
> guard-lab: demo
validatingpolicy.policies.kyverno.io/require-labels configured
namespace "scratch" deleted
namespace/scratch created (server dry run)
Error from server: error when creating "label-objs.yaml": adm
Error from server: error when creating "label-objs.yaml": adm
The old failures only disappear from the reports after the next background scan (default: hourly).
Save your progress? Sign up for free
Drill 5: Tags, registries, resources – with good messages
image-hygiene.yaml first collects the violations in
variables and then only checks their size. That way the message names
exactly the guilty images or containers.
sed -n '/variables:/,/validations:/p' image-hygiene.yamlExpected:
variables:
- name: allowedRegistries
expression: "['index.docker.io', 'kind-registry:5000']"
- name: images
expression: >-
(object.spec.containers + object.spec.?initContainers.orValue([]))
.map(c, c.image)
- name: latest
expression: >-
variables.images.filter(i,
!image(i).containsDigest() && image(i).tag() == 'latest')
- name: foreign
expression: >-
variables.images.filter(i,
!(image(i).registry() in variables.allowedRegistries))
- name: noResources
expression: >-
object.spec.containers.filter(c,
!('cpu' in c.?resources.?requests.orValue({})) ||
!('memory' in c.?resources.?requests.orValue({})) ||
!('memory' in c.?resources.?limits.orValue({})))
.map(c, c.name)
validations:image() is Kyverno’s image library. It normalizes
references: nginxinc/nginx-unprivileged becomes registry
index.docker.io with tag latest. A missing tag
therefore automatically falls under “latest”. With string comparisons,
on the other hand, you trip over the colon in the registry port
(kind-registry:5000/shop/web).
kubectl apply -f image-hygiene.yaml
kubectl apply -f hygiene-pods.yaml 2>&1 | sed 's/.*denied the request: //'
kubectl -n shop delete pod digestExpected:
validatingpolicy.policies.kyverno.io/image-hygiene created
pod/digest created
Policy image-hygiene failed: Fester Tag oder Digest nötig, nicht :latest:
nginxinc/nginx-unprivileged:latest
Policy image-hygiene failed: Fester Tag oder Digest nötig, nicht :latest:
kind-registry:5000/shop/web
Policy image-hygiene failed: Registry nicht erlaubt: quay.io/prometheus/busybox:glibc
(erlaubt: index.docker.io, kind-registry:5000)
Policy image-hygiene failed: requests.cpu, requests.memory und limits.memory fehlen
bei: web
pod/digest deleted from shop namespace
The Pod digest gets through: A digest is the strongest
“fixed tag”. Its image does not exist yet (Day 10), so delete it again
right away.
Save your progress? Sign up for free
Drill 6: Autogen – errors on the Deployment, not the Pod
kubectl create deploy web -n tenant-b --image=nginxinc/nginx-unprivileged \
--dry-run=client -o yaml | kubectl label --local -f - tier=web -o yaml \
| kubectl apply --dry-run=server -f - 2>&1 | sed 's/.*denied the request: //'Expected:
Policy image-hygiene failed: Fester Tag oder Digest nötig, nicht :latest:
nginxinc/nginx-unprivileged
Without autogen, the Deployment would be created, and only the
ReplicaSet controller would get the rejection for the Pod.
kubectl get deploy would then show 0/1, and
the team searches through the events. With autogen, the developer sees
the error directly in their apply or in Argo CD. You can
control this via
spec.autogen.podControllers.controllers.
Save your progress? Sign up for free
Drill 7: Parameters from a ConfigMap
The registry list does not belong in every policy, but in one central
place. In 1.19, ValidatingPolicy has no
paramKind like the native
ValidatingAdmissionPolicy
(kubectl explain vpol.spec does not show it). Kyverno
solves this with resource.Get():
grep -A3 'name: allowedRegistries' image-hygiene-cm.yaml
kubectl -n kyverno create configmap allowed-registries \
--from-literal=registries='index.docker.io,kind-registry:5000,quay.io'
kubectl apply -f image-hygiene-cm.yaml
kubectl apply -f hygiene-pods.yaml --dry-run=server 2>&1 | grep -E 'quay|digest'Expected:
- name: allowedRegistries
expression: >-
resource.Get("v1", "configmaps", "kyverno", "allowed-registries")
.data.registries.split(",")
configmap/allowed-registries created
validatingpolicy.policies.kyverno.io/image-hygiene configured
pod/quay created (server dry run)
pod/digest created (server dry run)
The list takes effect immediately, without touching the policy. And what if the ConfigMap is missing?
kubectl -n kyverno delete configmap allowed-registries
kubectl apply -f hygiene-pods.yaml --dry-run=server 2>&1 | grep quay \
| sed 's/.*denied the request: //'
kubectl apply -f image-hygiene.yamlExpected:
configmap "allowed-registries" deleted from kyverno namespace
Policy image-hygiene error: error: no such key: data
validatingpolicy.policies.kyverno.io/image-hygiene configured
Fail-closed: Without the parameters, everything is rejected. That is secure, but an outage risk. That is why the ConfigMap belongs in the same GitOps repo as the policy.
Security: Whoever may change the ConfigMap controls the policy. Grant write access to the
kyvernonamespace only to the platform team.
Save your progress? Sign up for free
Checkpoint
ok() { echo "✔ $1"; }; nok() { echo "✘ $1"; }
for p in pss-baseline pss-restricted require-labels image-hygiene; do
[ "$(kubectl get vpol $p -o jsonpath='{.status.conditionStatus.ready}' 2>/dev/null)" \
= true ] && ok "$p bereit" || nok "$p fehlt"
done
deny() { kubectl apply --dry-run=server -f - >/dev/null 2>&1 && nok "$1" || ok "$1"; }
kubectl run x -n shop -l app=x,tier=web --image=nginxinc/nginx-unprivileged:latest \
--dry-run=client -o yaml | deny ":latest abgelehnt"
kubectl run x -n shop -l app=x --image=quay.io/prometheus/busybox:glibc \
--dry-run=client -o yaml | deny "fremde Registry abgelehnt"
kubectl get deploy -n shop frontend -o yaml | kubectl apply --dry-run=server -f - \
>/dev/null 2>&1 && ok "Demo-frontend weiter gültig" || nok "Demo-frontend abgelehnt"
kyverno apply image-hygiene.yaml -f values.yaml -r ~/guard-lab/code/demo/10-shop.yaml \
>/dev/null 2>&1 && ok "CLI: Demo-App sauber" || nok "CLI: Demo-App verletzt Policy"Challenge
Limit the memory limit of every container in demo namespaces to at
most 512Mi. The message names the container and the value. Test only
with the CLI against big-pod.yaml and the demo app.
Hint: The Kubernetes CEL library knows
quantity('1Gi') with isGreaterThan(); with
that you compare 1Gi and 512Mi correctly.
Solution
sed -n '/variables:/,$p' max-memory.yaml
kyverno apply max-memory.yaml -f values.yaml -r big-pod.yaml \
-r ~/guard-lab/code/demo/10-shop.yamlExpected:
variables:
- name: tooBig
expression: >-
object.spec.containers.filter(c,
quantity(c.?resources.?limits[?'memory'].orValue('0'))
.isGreaterThan(quantity('512Mi')))
.map(c, c.name + '=' + c.resources.limits['memory'])
validations:
- expression: "size(variables.tooBig) == 0"
messageExpression: >-
'Memory-Limit über 512Mi: ' + variables.tooBig.join(', ')
Applying 1 policy rule(s) to 10 resource(s)...
policy max-memory -> resource shop/Pod/big failed:
1 - Memory-Limit über 512Mi: web=1Gi
pass: 3, fail: 1, warn: 0, error: 0, skip: 0
A string comparison ('1Gi' > '512Mi') would be wrong:
alphabetically, 1 is less than 5.
Save your progress? Sign up for free
Enterprise Note
- Pod Security Admission (PSA) in the namespace label is the first line, Kyverno the second: finer exceptions, better messages, reports (Day 13).
- Allowed registries in production: only the internal registry or the
pull-through cache (Harbor, Artifactory, Quay), never
docker.iodirectly. - Add a link to an internal wiki page to your messages (“How to fix this”). That saves the platform team tickets.
- Do not build exceptions into every policy via
matchConditions; handle them centrally viaPolicyExceptionwith an expiration date and a ticket reference (Day 11). - Check policies with
kyverno applyagainst all manifests in the repo before every rollout; otherwise the first team notices the error in its deployment.