Day 6: Testing and Visualizing Policies
Goal: Treat NetworkPolicies like code: target matrix as a file, check script with a ✔/✘ table, static analysis, Hubble UI and a CI job with kind. · Duration: approx. 90 min · Prerequisite: Day 5
Briefing
- A policy is only finished when a test proves what it allows and what it forbids. The negative cases are the more important ones.
- The target matrix belongs in Git as a file
(
von nach soll: from, to, expected). A script compares it with reality and returns an exit code for CI. - Static analysis (
policy-assistant) needs no cluster and explains why. Runtime tests (conncheck.sh) prove that the CNI actually does it. - Hubble UI and editor.networkpolicy.io make policies and flows visible – good for reviews with teams that do not want to read YAML.
matrix.tsv ──► np-matrix.sh ──► conncheck.sh ──► kubectl exec ──► ✔/✘ + exit code
policies/ ──► policy-assistant (offline) ──► explain and probe tables
Hubble ──► UI (service map) / flows.json ──► editor.networkpolicy.io
Drill 1: Prepare the ground
Continue on cilium-lab. You clear away the Cilium
policies from Days 4 and 5, then comes a clean target state for
shop – one file per policy.
mkdir -p ~/guard-lab/tag06 ~/guard-lab/code/np-tests/policies
cd ~/guard-lab/code/np-tests
kubectl config use-context kind-cilium-lab
kubectl delete cnp --all -A
kubectl delete ccnp --all
cat > policies/10-default-deny.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
namespace: shop
spec:
podSelector: {}
policyTypes: [Ingress]
EOF
cat > policies/20-frontend-public.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: frontend-public
namespace: shop
spec:
podSelector:
matchLabels: {app: frontend}
policyTypes: [Ingress]
ingress:
- ports:
- {port: 8080, protocol: TCP}
EOF
cat > policies/30-api-from-frontend.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: api-from-frontend
namespace: shop
spec:
podSelector:
matchLabels: {app: api}
policyTypes: [Ingress]
ingress:
- from:
- podSelector:
matchLabels: {app: frontend}
ports:
- {port: 8080, protocol: TCP}
- from:
- namespaceSelector:
matchLabels: {purpose: monitoring}
ports:
- {port: 9113, protocol: TCP}
EOF
cat > policies/40-db-from-api.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: db-from-api
namespace: shop
spec:
podSelector:
matchLabels: {app: db}
policyTypes: [Ingress]
ingress:
- from:
- podSelector:
matchLabels: {app: api}
ports:
- {port: 5432, protocol: TCP}
EOF
kubectl apply -f policies/Expected (abridged, may differ depending on version):
ciliumnetworkpolicy.cilium.io "api-l4" deleted
...
ciliumclusterwidenetworkpolicy.cilium.io "tenant-deny" deleted
networkpolicy.networking.k8s.io/default-deny created
networkpolicy.networking.k8s.io/frontend-public created
networkpolicy.networking.k8s.io/api-from-frontend created
networkpolicy.networking.k8s.io/db-from-api created
Why explicit policyTypes, even though the API server
would add them? Offline tools (Drill 4) only see the file, not the
default added by the server.
Save your progress? Sign up for free
Drill 2: Target matrix and check script
The matrix is a plain text file: source, target, expectation. Think of the negative cases first – what must never work?
cat > matrix.tsv <<'EOF'
# von nach soll
default/client http://frontend.shop allow
default/client http://api.shop deny
default/client tcp://db.shop:5432 deny
shop/frontend http://api.shop allow
shop/frontend tcp://db.shop:5432 deny
shop/api tcp://db.shop:5432 allow
monitoring/scraper http://api.shop:9113/metrics allow
monitoring/scraper http://api.shop deny
tenant-a/api http://api.shop deny
EOF
cat > np-matrix.sh <<'EOF'
#!/usr/bin/env bash
# np-matrix.sh - test matrix for NetworkPolicies: from -> to, expected vs. actual
#
# np-matrix.sh [--md] <matrix.tsv>
#
# matrix.tsv lines "<ns>/<pod|app> <url> allow|deny", # = comment
# --md output as a Markdown table (e.g. for $GITHUB_STEP_SUMMARY)
#
# Exit 0 = everything as expected, 1 = at least one deviation, 2 = usage error.
# CC points to conncheck.sh (default: ~/guard-lab/code/demo/conncheck.sh).
set -uo pipefail
md=false
[[ ${1:-} == --md ]] && { md=true; shift; }
file=${1:-}
cc=${CC:-$HOME/guard-lab/code/demo/conncheck.sh}
if [[ -z $file || ! -r $file ]]; then
echo "Aufruf: $0 [--md] <matrix.tsv>" >&2
exit 2
fi
if $md; then
echo "| von | nach | soll | ist |"
echo "|---|---|---|---|"
else
printf '%-22s %-34s %-6s %s\n' VON NACH SOLL IST
fi
ok=0 bad=0
while read -r src url want _; do
[[ -z ${src:-} || $src == \#* ]] && continue
"$cc" "$src" "$url" </dev/null >/dev/null 2>&1
case $? in
0) got=allow ;;
1) got=deny ;;
*) got=error ;;
esac
if [[ $got == "$want" ]]; then mark="✔"; ok=$((ok + 1))
else mark="✘"; bad=$((bad + 1)); fi
if $md; then
echo "| \`$src\` | \`$url\` | $want | $mark $got |"
else
printf '%-22s %-34s %-6s %s %s\n' "$src" "$url" "$want" "$mark" "$got"
fi
done < "$file"
echo
echo "$((ok + bad)) Tests: $ok ✔, $bad ✘"
[[ $bad -eq 0 ]]
EOF
chmod +x np-matrix.sh
./np-matrix.sh matrix.tsv; echo "rc=$?"Expected (abridged, may differ depending on version):
VON NACH SOLL IST
default/client http://frontend.shop allow ✔ allow
default/client http://api.shop deny ✔ deny
default/client tcp://db.shop:5432 deny ✔ deny
shop/frontend http://api.shop allow ✔ allow
shop/frontend tcp://db.shop:5432 deny ✔ deny
shop/api tcp://db.shop:5432 allow ✔ allow
monitoring/scraper http://api.shop:9113/metrics allow ✔ allow
monitoring/scraper http://api.shop deny ✔ deny
tenant-a/api http://api.shop deny ✔ deny
9 Tests: 9 ✔, 0 ✘
rc=0
Two details: </dev/null on the
conncheck.sh call prevents a command in the loop from
eating the remaining lines of the matrix from standard input. And
error (exit 2, e.g. source Pod missing) always counts as a
deviation – a test that does not run is red.
Save your progress? Sign up for free
Drill 3: Provoke a regression
A colleague “cleans up labels” and renames frontend to
web in the selector.
sed 's/app: frontend}/app: web}/' policies/30-api-from-frontend.yaml \
| kubectl apply -f -
./np-matrix.sh matrix.tsv > ~/guard-lab/tag06/run.txt; echo "rc=$?"
grep -E '✘|Tests' ~/guard-lab/tag06/run.txtExpected (abridged, may differ depending on version):
networkpolicy.networking.k8s.io/api-from-frontend configured
rc=1
shop/frontend http://api.shop allow ✘ deny
9 Tests: 8 ✔, 1 ✘
Exactly one failure, exactly in the right place. The frontend itself
keeps running (/ returns 200), only /api/…
fails – without a matrix, this is often only noticed in production. Back
to the target state:
kubectl apply -f policies/30-api-from-frontend.yaml
./np-matrix.sh matrix.tsv | tail -1Expected (abridged, may differ depending on version):
networkpolicy.networking.k8s.io/api-from-frontend configured
9 Tests: 9 ✔, 0 ✘
Save your progress? Sign up for free
Drill 4: Static analysis with policy-assistant
policy-assistant from the SIG project network-policy-api
is the successor of cyclonus (unmaintained since 2024). It
reads policies from files and simulates connections – entirely without a
cluster. There is a release binary only for amd64; on Apple Silicon it
runs via Rosetta.
cd /tmp
curl -sLO https://github.com/kubernetes-sigs/network-policy-api/releases/download/\
v0.0.1-policy-assistant/policy-assistant_darwin_amd64.tar.gz
tar xzf policy-assistant_darwin_amd64.tar.gz
PA=/tmp/policy-assistant_darwin_amd64/policy-assistant
cd ~/guard-lab/code/np-tests
$PA analyze -v warn --mode explain --policy-path policies/ | cut -c1-86 | head -9Expected:
explained policies:
+---------+-------------------+-------------------------------+-----------------------
| TYPE | SUBJECT | SOURCE RULES | PEER
+---------+-------------------+-------------------------------+-----------------------
| Ingress | Namespace: | [NPv1] shop/api-from-frontend | Namespace:
| | shop | | shop
| | Pod: | | Pod:
| | app = api | | app = frontend
+ + + +-----------------------
Warning: policy-assistant only reads the first YAML document per file and requires
policyTypes. That is why there is one policy per file (Drill 1).
For the simulation, you describe Pods and namespaces in a JSON file – a small model of the demo app:
cat > probe.json <<'EOF'
{
"Probes": [
{"Protocol": "TCP", "Port": 8080},
{"Protocol": "TCP", "Port": 5432}
],
"Resources": {
"Namespaces": {
"shop": {"kubernetes.io/metadata.name": "shop", "guard-lab": "demo"},
"monitoring": {"kubernetes.io/metadata.name": "monitoring",
"purpose": "monitoring"},
"default": {"kubernetes.io/metadata.name": "default"}
},
"Pods": [
{"Namespace": "shop", "Name": "frontend", "IP": "10.244.1.10",
"Labels": {"app": "frontend", "tier": "web"},
"Containers": [{"Name": "nginx", "Port": 8080, "PortName": "http",
"Protocol": "TCP"}]},
{"Namespace": "shop", "Name": "api", "IP": "10.244.1.11",
"Labels": {"app": "api", "tier": "api"},
"Containers": [{"Name": "nginx", "Port": 8080, "PortName": "http",
"Protocol": "TCP"}]},
{"Namespace": "shop", "Name": "db", "IP": "10.244.1.12",
"Labels": {"app": "db", "tier": "data"},
"Containers": [{"Name": "nginx", "Port": 5432, "PortName": "postgres",
"Protocol": "TCP"}]},
{"Namespace": "default", "Name": "client", "IP": "10.244.2.20",
"Labels": {"app": "client"},
"Containers": [{"Name": "curl", "Port": 80, "PortName": "none",
"Protocol": "TCP"}]}
]
}
}
EOF
$PA analyze -v warn --mode probe --policy-path policies/ --probe-path probe.json \
| grep -A8 CombinedExpected:
Combined:
+----------------+----------------+----------+---------+---------------+
| | DEFAULT/CLIENT | SHOP/API | SHOP/DB | SHOP/FRONTEND |
+----------------+----------------+----------+---------+---------------+
| default/client | N | X | N | . |
| shop/api | N | # | N | . |
| shop/db | N | X | N | . |
| shop/frontend | N | . | N | # |
+----------------+----------------+----------+---------+---------------+
--
Combined:
...
| default/client | N | N | X | N |
| shop/api | N | N | . | N |
| shop/frontend | N | N | X | N |
Rows are sources, columns are targets; one table per probe (8080,
then 5432). . = allowed, X = blocked,
N = the target does not listen on the port, #
= the Pod itself. Same result as in Drill 2 – just in milliseconds and
without a cluster. Try the regression from Drill 3 offline: The cell
shop/frontend → SHOP/API turns into X.
Tip: policy-assistant knows NetworkPolicy and ANP/BANP v1alpha1, but not yet ClusterNetworkPolicy v1alpha2 and no Cilium types. For those: runtime test.
Save your progress? Sign up for free
Drill 5: Hubble UI – the service map
Back in the cluster. You installed the UI along with everything else on Day 4.
cilium hubble ui --open-browser=false &
sleep 3
for i in 1 2 3; do ~/guard-lab/code/np-tests/np-matrix.sh \
~/guard-lab/code/np-tests/matrix.tsv >/dev/null; doneThe command sets up a port forward to the Service
hubble-ui (default port 12000). Open
http://localhost:12000 and select the namespace
shop at the top. You see the boxes frontend,
api, db and the incoming arrows from
client, scraper and tenant-a/api.
Dropped flows are marked red; in the flow table below it says
dropped with the reason Policy denied.
Clicking a flow shows source and destination labels – exactly the labels
your selectors have to match.
kill %1Security: The UI shows all flows of all namespaces. In production, never publish it without authentication (ingress with OIDC) and never via
LoadBalancer.
Save your progress? Sign up for free
Drill 6: editor.networkpolicy.io
The policy editor from the Cilium community runs in the browser and understands NetworkPolicy and CiliumNetworkPolicy. It shows a policy as a picture: Pod in the middle, allowed sources and targets on the left and right, default deny as a red area.
cat ~/guard-lab/code/np-tests/policies/30-api-from-frontend.yaml | pbcopy- Open
https://editor.networkpolicy.io, choose “Upload” or the YAML field, and paste. - Check: Ingress shows
app=frontendon 8080 andpurpose=monitoringon 9113. - Egress is open (green), because the policy only governs
Ingress. - Make changes in the picture and download the YAML again.
The editor can also generate policy suggestions from Hubble flows:
hubble observe -P --namespace shop --last 200 -o json > ~/guard-lab/tag06/flows.json
wc -l ~/guard-lab/tag06/flows.jsonExpected (abridged, may differ depending on version):
200 /Users/<du>/guard-lab/tag06/flows.json
Security: Flows contain IPs, names and URLs. You do not upload real customer flows to a public web tool. For production: generate suggestions locally and check them in a review – a generated allow-all-observed is not a security architecture.
Save your progress? Sign up for free
Drill 7: CI job with kind
What runs locally also runs in the pipeline: a fresh kind cluster
(kindnet enforces NetworkPolicies), demo app, policies, matrix.
Assumption: Your repo contains code/demo/ and
code/np-tests/ like ~/guard-lab/code/.
mkdir -p ~/guard-lab/code/np-tests/ci && cd ~/guard-lab/code/np-tests/ci
cat > github-np-tests.yaml <<'EOF'
# Day 6: GitHub Actions - copy to .github/workflows/np-tests.yaml
name: np-tests
on:
pull_request:
paths: ["code/**"]
workflow_dispatch:
jobs:
matrix:
runs-on: ubuntu-24.04
timeout-minutes: 20
defaults:
run:
shell: bash # = bash -eo pipefail: tee does not swallow the rc
steps:
- uses: actions/checkout@v7
- uses: helm/[email protected] # kindnet enforces NetworkPolicies
with:
version: v0.33.0
cluster_name: np-ci
- name: Demo-App ausrollen
run: |
kubectl apply -f code/demo/
kubectl wait --for=condition=Ready pod --all -A --timeout=300s
- name: Policies anwenden
run: kubectl apply -f code/np-tests/policies/
- name: Testmatrix
env:
CC: ${{ github.workspace }}/code/demo/conncheck.sh
run: |
sleep 5
code/np-tests/np-matrix.sh --md code/np-tests/matrix.tsv \
| tee -a "$GITHUB_STEP_SUMMARY"
EOFCopied to .github/workflows/np-tests.yaml, the job runs
on every pull request that changes code/. --md
writes the table into the job summary – reviewers see ✔/✘ directly in
the PR. shell: bash is needed so that pipefail
applies and tee does not swallow the exit code of
np-matrix.sh.
Check the workflow locally with actionlint before you
push:
go run github.com/rhysd/actionlint/cmd/actionlint@latest github-np-tests.yaml \
&& echo "✔ actionlint"Expected:
✔ actionlint
In GitLab, kind runs in Docker-in-Docker. The API server then does
not sit on the job’s 127.0.0.1, but in the service
container docker – hence the separate kind
configuration:
cat > kind-ci.yaml <<'EOF'
# Day 6: kind in GitLab CI (Docker-in-Docker) - API server via the service "docker"
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
apiServerAddress: "0.0.0.0" # only acceptable in a throwaway CI job
apiServerPort: 6443
kubeadmConfigPatches:
- |
kind: ClusterConfiguration
apiServer:
certSANs: ["docker"]
EOF
cat > gitlab-np-tests.yaml <<'EOF'
# Day 6: GitLab CI - add as a job to .gitlab-ci.yml (runner with privileged dind)
np-tests:
image: docker:29.1
services:
- name: docker:29.1-dind
alias: docker
variables:
DOCKER_HOST: tcp://docker:2375
DOCKER_TLS_CERTDIR: ""
KUBECTL_URL: https://dl.k8s.io/release/v1.37.0/bin/linux/amd64/kubectl
rules:
- changes: ["code/**/*"]
before_script:
- apk add --no-cache bash curl
- curl -sLo /usr/local/bin/kind https://kind.sigs.k8s.io/dl/v0.33.0/kind-linux-amd64
- curl -sLo /usr/local/bin/kubectl "$KUBECTL_URL"
- chmod +x /usr/local/bin/kind /usr/local/bin/kubectl
script:
- kind create cluster --name np-ci --config code/np-tests/ci/kind-ci.yaml
- sed -i 's#https://0.0.0.0:6443#https://docker:6443#' ~/.kube/config
- kubectl apply -f code/demo/
- kubectl wait --for=condition=Ready pod --all -A --timeout=300s
- kubectl apply -f code/np-tests/policies/
- sleep 5
- CC=$PWD/code/demo/conncheck.sh code/np-tests/np-matrix.sh code/np-tests/matrix.tsv
EOFYou add the job to your .gitlab-ci.yml. The runner must
run privileged (for docker:dind) – that
belongs on a dedicated runner, not on the shared one. The GitLab job has
not been tested live; the GitHub workflow has been checked with
actionlint.
Tip: If you want to test Cilium policies in CI, install Cilium into the kind cluster via Helm as on Day 4. Plan for 3–5 minutes of additional runtime.
Save your progress? Sign up for free
Checkpoint
cd ~/guard-lab/code/np-tests
PA=${PA:-/tmp/policy-assistant_darwin_amd64/policy-assistant}
[ "$(ls policies/*.yaml | wc -l)" -ge 4 ] \
&& echo "✔ Policies einzeln abgelegt" || echo "✘ policies/ unvollständig"
grep -q deny matrix.tsv && echo "✔ Matrix mit Negativ-Fällen" \
|| echo "✘ keine deny-Zeilen in der Matrix"
./np-matrix.sh matrix.tsv >/dev/null && echo "✔ Matrix grün" || echo "✘ Matrix rot"
./np-matrix.sh >/dev/null 2>&1; [ $? -eq 2 ] \
&& echo "✔ Aufruffehler = Exit 2" || echo "✘ Exit-Code bei Aufruffehler"
$PA analyze -v warn --mode probe --policy-path policies/ --probe-path probe.json \
| grep -q 'shop/frontend | N | .' \
&& echo "✔ policy-assistant: frontend -> api erlaubt" || echo "✘ policy-assistant"Challenge
Add an egress case to the matrix: The database must not reach the
internet (shop/db https://example.com deny). Let the matrix
turn red and make it green again with a new policy file, without
breaking any other lines.
Hint: An egress policy for db without
any allowed targets is enough – db itself does not open any
connections.
Solution
cd ~/guard-lab/code/np-tests
echo "shop/db https://example.com deny" >> matrix.tsv
./np-matrix.sh matrix.tsv | tail -1
cat > policies/50-db-no-egress.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: db-no-egress
namespace: shop
spec:
podSelector:
matchLabels: {app: db}
policyTypes: [Egress]
EOF
kubectl apply -f policies/50-db-no-egress.yaml
./np-matrix.sh matrix.tsv | tail -1Expected (abridged, may differ depending on version):
10 Tests: 9 ✔, 1 ✘
networkpolicy.networking.k8s.io/db-no-egress created
10 Tests: 10 ✔, 0 ✘
The db’s responses to the api keep flowing:
NetworkPolicies are stateful; return packets of an allowed connection do
not need an egress rule.
Save your progress? Sign up for free
Enterprise Note
- The matrix is a contract with the teams: Changes to it go through review like code. One file per tenant, shared platform cases in a separate one.
- Runtime tests belong in the pipeline after every CNI upgrade and
every platform rollout (smoke test namespace with a
clientPod in the real cluster). - Static analysis before the merge, runtime test after the deploy – both, not one.
- Export Hubble flows to the SIEM and alert on
policy-verdict:none … DENIED: Those are the connections for which someone forgot a policy. - Clean up:
kind delete cluster --name cilium-lab; from Day 7 on, you continue withkyverno-lab.