Day 5: Cluster-Wide Rules
Goal: Set guardrails that no namespace admin can override: with
CiliumClusterwideNetworkPolicyand with the new Kubernetes APIClusterNetworkPolicy. · Duration: approx. 120 min · Prerequisite: Day 4
Briefing
- A NetworkPolicy belongs to the namespace. Whoever has permissions there can open everything. The platform needs rules above it (admin) and below it (baseline).
- For this, SIG Network built AdminNetworkPolicy and
BaselineAdminNetworkPolicy (v1alpha1). Since network-policy-api v0.2.0
(April 2026), both have merged into one type:
ClusterNetworkPolicy(v1alpha2) withtier: AdminorBaseline. - Actions:
Accept(formerlyAllow),Deny,Pass.priority0–1000, lower = evaluated earlier. - Cilium does not implement the upstream API (yet). There, the tool is
called
CiliumClusterwideNetworkPolicy(CCNP), with explicit deny rules.
packet ─► Admin tier ──Accept/Deny──► done
│ Pass or no match
▼
NetworkPolicy (namespace) ──match──► Allow / default deny
│ Pod not covered by any NetworkPolicy
▼
Baseline tier ──Accept/Deny──► done (otherwise: allowed)
Drill 1: Who can do what?
On cilium-lab from Day 4 – which policy types does the
cluster know?
mkdir -p ~/guard-lab/tag05 && cd ~/guard-lab/tag05
kubectl config use-context kind-cilium-lab
kubectl api-resources | grep -i networkpolicExpected (abridged, may differ depending on version):
ciliumclusterwidenetworkpolicies ccnp cilium.io/v2 false CiliumClusterwide…
ciliumnetworkpolicies cnp cilium.io/v2 true CiliumNetworkPolicy
networkpolicies netpol networking.k8s.io/v1 true NetworkPolicy
No clusternetworkpolicies: Cilium does not ship the CRD
and has no controller for it. Creating a CRD by hand does not help –
then the policy is just YAML in etcd. State of the implementations
(according to network-policy-api, 2026):
| Implementation | ANP/BANP v1alpha1 | ClusterNetworkPolicy v1alpha2 |
|---|---|---|
| kube-network-policies | yes | yes (separate image …-npa-v1alpha2) |
| Calico | 3.29–3.31 | from 3.32 |
| OVN-Kubernetes, Antrea | yes | not yet |
| Kube-OVN | yes | experimental |
| Cilium | no (tracking issue) | no – CCNP instead |
Warning: The abbreviation
cnphas two meanings: in Cilium it stands forCiliumNetworkPolicy, upstream forClusterNetworkPolicy. Always write the full name in your team.
Save your progress? Sign up for free
Drill 2: CCNP – separate tenants hard
First, a team opens its namespace wide. Then the platform draws a line that no namespace policy can override.
cat > open-tenant-a.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-all-ingress
namespace: tenant-a
spec:
podSelector: {}
ingress:
- {}
EOF
kubectl apply -f open-tenant-a.yaml
cc tenant-b/api http://api.tenant-aExpected (abridged, may differ depending on version):
networkpolicy.networking.k8s.io/allow-all-ingress created
✔ erlaubt tenant-b/api -> http://api.tenant-a (HTTP 200)
Now the platform rule. A CCNP has no namespace; specs
bundles several rules in one object.
cat > ~/guard-lab/code/cilium/ccnp-tenant-deny.yaml <<'EOF'
apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
name: tenant-deny
specs: # several rules in one object
- endpointSelector:
matchLabels:
k8s:io.kubernetes.pod.namespace: tenant-a
enableDefaultDeny:
ingress: false # only deny, do not close everything else
ingressDeny:
- fromEndpoints:
- matchExpressions:
- {key: k8s:io.cilium.k8s.namespace.labels.tenant, operator: Exists}
- {key: k8s:io.kubernetes.pod.namespace, operator: NotIn, values: [tenant-a]}
- endpointSelector:
matchLabels:
k8s:io.kubernetes.pod.namespace: tenant-b
enableDefaultDeny:
ingress: false
ingressDeny:
- fromEndpoints:
- matchExpressions:
- {key: k8s:io.cilium.k8s.namespace.labels.tenant, operator: Exists}
- {key: k8s:io.kubernetes.pod.namespace, operator: NotIn, values: [tenant-b]}
EOF
kubectl apply -f ~/guard-lab/code/cilium/ccnp-tenant-deny.yaml
cc tenant-b/api http://api.tenant-a
cc shop/frontend http://api.tenant-a
cc tenant-a/api http://api.tenant-bExpected (abridged, may differ depending on version):
ciliumclusterwidenetworkpolicy.cilium.io/tenant-deny created
✘ blockiert tenant-b/api -> http://api.tenant-a (Timeout)
✔ erlaubt shop/frontend -> http://api.tenant-a (HTTP 200)
✘ blockiert tenant-a/api -> http://api.tenant-b (Timeout)
In Cilium, deny always wins over allow – no matter which policy it
comes from. The namespace admin of tenant-a cannot lift the
block. shop carries no tenant label and still
gets through.
Why enableDefaultDeny: false? Without that field, every
policy with ingress rules – even pure deny rules – switches the selected
Pods to default deny. Then shop/frontend would suddenly be
locked out too – immediately visible in tenant-b, which has
no NetworkPolicy of its own.
Save your progress? Sign up for free
Drill 3: CCNP – block cloud metadata
A classic for every platform: No Pod may reach the cloud’s metadata
service (169.254.169.254); node credentials live there.
cat > ~/guard-lab/code/cilium/ccnp-metadata-deny.yaml <<'EOF'
apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
name: deny-cloud-metadata
spec:
endpointSelector: {} # all Pods in the cluster
enableDefaultDeny:
egress: false
egressDeny:
- toCIDR:
- 169.254.169.254/32
EOF
kubectl apply -f ~/guard-lab/code/cilium/ccnp-metadata-deny.yaml
kubectl get ccnp
cc default/client http://169.254.169.254/latest/meta-data/
hubble observe -P --from-pod default/client --to-ip 169.254.169.254 --last 1Expected (abridged, may differ depending on version):
ciliumclusterwidenetworkpolicy.cilium.io/deny-cloud-metadata created
NAME VALID
deny-cloud-metadata True
tenant-deny True
✘ blockiert default/client -> http://169.254.169.254/latest/meta-data/ (Timeout)
... default/client:43120 (ID:23412) -> 169.254.169.254:80 (world) Policy denied by
denylist DROPPED (TCP Flags: SYN)
In kind there is no service there, so a timeout would happen even
without a policy. Only Hubble provides the proof:
Policy denied by denylist means a deny rule kicked in.
Save your progress? Sign up for free
Drill 4: np-lab with ClusterNetworkPolicy
For the upstream API you need an implementation. The leanest one is
kube-network-policies from SIG Network – the same engine
that is inside kindnet, but with ClusterNetworkPolicy. So that kindnet
does not enforce NetworkPolicies in parallel, you swap its image for an
older version without a policy engine (that is also how the project’s CI
does it).
kind create cluster --config ~/guard-lab/kind/np-lab.yaml
kubectl -n kube-system set image ds kindnet \
kindnet-cni=docker.io/kindest/kindnetd:v20230809-80a64d96
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/\
network-policy-api/v0.2.0/config/crd/standard/\
policy.networking.k8s.io_clusternetworkpolicies.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/\
kube-network-policies/v1.1.2/install-cnp.yaml
kubectl -n kube-system rollout status ds kindnet
kubectl -n kube-system rollout status ds kube-network-policiesExpected (abridged, may differ depending on version):
daemonset.apps/kindnet image updated
customresourcedefinition.apiextensions.k8s.io/clusternetworkpolicies.policy… created
daemonset.apps/kube-network-policies created
daemon set "kindnet" successfully rolled out
daemon set "kube-network-policies" successfully rolled out
Roll out the demo app and prove that NetworkPolicies still take
effect – now through kube-network-policies:
kubectl apply -f ~/guard-lab/code/demo/
kubectl wait --for=condition=Ready pod --all -A --timeout=300s
kubectl apply -f open-tenant-a.yaml
kubectl -n tenant-b create -f - <<'EOF'
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: {name: deny-all}
spec: {podSelector: {}, policyTypes: [Ingress]}
EOF
cc default/client http://api.tenant-b
cc tenant-b/api http://api.tenant-aExpected (abridged, may differ depending on version):
networkpolicy.networking.k8s.io/allow-all-ingress created
networkpolicy.networking.k8s.io/deny-all created
✘ blockiert default/client -> http://api.tenant-b (Timeout)
✔ erlaubt tenant-b/api -> http://api.tenant-a (HTTP 200)
Same starting point as in Drill 2: tenant-a is open,
tenant-b is sealed.
Save your progress? Sign up for free
Drill 5: Admin tier – deny beats every NetworkPolicy
cat > ~/guard-lab/code/cilium/cnp-tenant-deny.yaml <<'EOF'
apiVersion: policy.networking.k8s.io/v1alpha2
kind: ClusterNetworkPolicy
metadata:
name: tenant-a-isolation
spec:
tier: Admin
priority: 10
subject:
namespaces:
matchLabels: {tenant: a}
ingress:
- name: deny-other-tenants
action: Deny
from:
- namespaces:
matchExpressions:
- {key: tenant, operator: Exists}
- {key: tenant, operator: NotIn, values: [a]}
---
apiVersion: policy.networking.k8s.io/v1alpha2
kind: ClusterNetworkPolicy
metadata:
name: tenant-b-isolation
spec:
tier: Admin
priority: 10
subject:
namespaces:
matchLabels: {tenant: b}
ingress:
- name: deny-other-tenants
action: Deny
from:
- namespaces:
matchExpressions:
- {key: tenant, operator: Exists}
- {key: tenant, operator: NotIn, values: [b]}
EOF
kubectl apply -f ~/guard-lab/code/cilium/cnp-tenant-deny.yaml
kubectl get clusternetworkpolicy
cc tenant-b/api http://api.tenant-a
cc shop/frontend http://api.tenant-aExpected (abridged, may differ depending on version):
clusternetworkpolicy.policy.networking.k8s.io/tenant-a-isolation created
clusternetworkpolicy.policy.networking.k8s.io/tenant-b-isolation created
NAME TIER PRIORITY AGE
tenant-a-isolation Admin 10 2s
tenant-b-isolation Admin 10 2s
✘ blockiert tenant-b/api -> http://api.tenant-a (Timeout)
✔ erlaubt shop/frontend -> http://api.tenant-a (HTTP 200)
The admin tier is evaluated before any NetworkPolicy. For
tenant-b, allow-all-ingress no longer gets a
say at all. For all other sources, the namespace’s NetworkPolicy still
decides – a ClusterNetworkPolicy does not switch on default deny.
Save your progress? Sign up for free
Drill 6: Accept and Pass – monitoring and delegation
The platform guarantees monitoring access to port 9113 – even in
namespaces whose team has locked everything down (like
tenant-b).
kubectl -n monitoring logs deploy/scraper --tail=3
cat > ~/guard-lab/code/cilium/cnp-monitoring.yaml <<'EOF'
apiVersion: policy.networking.k8s.io/v1alpha2
kind: ClusterNetworkPolicy
metadata:
name: monitoring-accept
spec:
tier: Admin
priority: 20
subject:
namespaces:
matchLabels: {guard-lab: demo}
ingress:
- name: accept-scraper-metrics
action: Accept
from:
- namespaces:
matchLabels: {purpose: monitoring}
protocols:
- tcp:
destinationPort:
number: 9113
EOF
kubectl apply -f ~/guard-lab/code/cilium/cnp-monitoring.yaml
sleep 12; kubectl -n monitoring logs deploy/scraper --tail=3Expected (abridged, may differ depending on version):
ok shop
ok tenant-a
FAIL tenant-b
clusternetworkpolicy.policy.networking.k8s.io/monitoring-accept created
ok shop
ok tenant-a
ok tenant-b
Accept ends the evaluation: The deny-all
policy in tenant-b never sees the packet. In v1alpha2,
protocols replaces the old ports field from
v1alpha1.
Now the shop team wants to decide for itself who may
scrape. Pass skips the rest of the admin tier and hands the
decision on to the NetworkPolicies.
cat > ~/guard-lab/code/cilium/cnp-pass-shop.yaml <<'EOF'
apiVersion: policy.networking.k8s.io/v1alpha2
kind: ClusterNetworkPolicy
metadata:
name: shop-delegate-monitoring
spec:
tier: Admin
priority: 15 # lower = evaluated earlier: before monitoring-accept
subject:
namespaces:
matchLabels: {kubernetes.io/metadata.name: shop}
ingress:
- name: pass-monitoring
action: Pass
from:
- namespaces:
matchLabels: {purpose: monitoring}
EOF
kubectl apply -f ~/guard-lab/code/cilium/cnp-pass-shop.yaml
kubectl -n shop create -f - <<'EOF'
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: {name: deny-all}
spec: {podSelector: {}, policyTypes: [Ingress]}
EOF
cc monitoring/scraper http://api.shop:9113/metrics
cc monitoring/scraper http://api.tenant-b:9113/metricsExpected (abridged, may differ depending on version):
clusternetworkpolicy.policy.networking.k8s.io/shop-delegate-monitoring created
networkpolicy.networking.k8s.io/deny-all created
✘ blockiert monitoring/scraper -> http://api.shop:9113/metrics (Timeout)
✔ erlaubt monitoring/scraper -> http://api.tenant-b:9113/metrics (HTTP 200)
Same starting point (team deny-all), different result:
In shop, Pass with priority 15 took effect
before the Accept with priority 20, so the team’s
NetworkPolicy decides.
Save your progress? Sign up for free
Drill 7: Baseline tier – a default deny that teams override
Baseline is the counterpart: rules that only apply when no NetworkPolicy covers the Pod. Ideal for a cluster-wide default deny. First a clean slate:
kubectl delete netpol -n shop --all
kubectl delete netpol -n tenant-a --all
kubectl delete netpol -n tenant-b --all
cat > ~/guard-lab/code/cilium/cnp-baseline.yaml <<'EOF'
apiVersion: policy.networking.k8s.io/v1alpha2
kind: ClusterNetworkPolicy
metadata:
name: baseline-deny
spec:
tier: Baseline
priority: 50
subject:
namespaces:
matchLabels: {guard-lab: demo}
ingress:
- name: deny-all-pods
action: Deny
from:
- namespaces: {}
EOF
kubectl apply -f ~/guard-lab/code/cilium/cnp-baseline.yaml
cc default/client http://frontend.shop
cc monitoring/scraper http://api.tenant-a:9113/metricsExpected (abridged, may differ depending on version):
clusternetworkpolicy.policy.networking.k8s.io/baseline-deny created
✘ blockiert default/client -> http://frontend.shop (Timeout)
✔ erlaubt monitoring/scraper -> http://api.tenant-a:9113/metrics (HTTP 200)
Monitoring gets through because monitoring-accept in the
admin tier decides first. Now the shop team opens its
frontend with a perfectly ordinary NetworkPolicy:
cat <<'EOF' | kubectl apply -f -
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: frontend-public
namespace: shop
spec:
podSelector:
matchLabels: {app: frontend}
ingress:
- ports:
- {port: http}
EOF
cc default/client http://frontend.shop
cc default/client http://api.shopExpected (abridged, may differ depending on version):
networkpolicy.networking.k8s.io/frontend-public created
✔ erlaubt default/client -> http://frontend.shop (HTTP 200)
✘ blockiert default/client -> http://api.shop (Timeout)
As soon as a NetworkPolicy covers the Pod, the baseline tier is out
of the game for it. api has no policy of its own, so
baseline-deny still applies to it.
Warning: Baseline is not a safety net against teams, it is a default. Anything that must never be open belongs in the admin tier.
Save your progress? Sign up for free
Drill 8: Comparison – Calico GlobalNetworkPolicy
Calico has had cluster-wide rules for years, with its own tiers. The same tenant separation as in Drill 5 looks like this (read only, Calico does not run in the lab):
cat > ~/guard-lab/code/cilium/calico-gnp.yaml <<'EOF'
apiVersion: projectcalico.org/v3
kind: Tier
metadata:
name: platform
spec:
order: 100 # before tier "default" (that is where the NetworkPolicies live)
defaultAction: Pass
---
apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
name: platform.tenant-a-isolation # prefix = tier name (older Calico versions)
spec:
tier: platform
order: 10 # lower = evaluated earlier
namespaceSelector: tenant == 'a'
types:
- Ingress
ingress:
- action: Deny
source:
namespaceSelector: has(tenant) && tenant != 'a'
- action: Pass # rest: on to the next tier (NetworkPolicies)
EOF| Concept | ClusterNetworkPolicy | CCNP (Cilium) | GlobalNetworkPolicy (Calico) |
|---|---|---|---|
| Precedence | tier + priority |
deny before allow | tier order + order |
| Delegation | action: Pass |
– | action: Pass |
| Selectors | label selector | labels with prefix | expressions (has(x) && …) |
| Extras | networks, FQDN (exp.) |
L7, toFQDNs, entities |
Log, host endpoints, HTTP |
Calico from 3.32 on additionally implements
ClusterNetworkPolicy. If you are starting fresh today,
write guardrails preferably in the upstream API and use CNI types only
for their extras.
Save your progress? Sign up for free
Checkpoint
On np-lab:
cc=~/guard-lab/code/demo/conncheck.sh
kubectl get crd clusternetworkpolicies.policy.networking.k8s.io >/dev/null 2>&1 \
&& echo "✔ CRD ClusterNetworkPolicy" || echo "✘ CRD fehlt"
[ "$(kubectl get clusternetworkpolicy --no-headers | wc -l)" -ge 4 ] \
&& echo "✔ mindestens 4 ClusterNetworkPolicies" || echo "✘ Policies fehlen"
while read -r src url want; do
"$cc" "$src" "$url" "$want" >/dev/null \
&& echo "✔ $src -> $url ($want)" || echo "✘ $src -> $url (erwartet: $want)"
done <<'EOF'
tenant-b/api http://api.tenant-a deny
shop/frontend http://api.tenant-a allow
monitoring/scraper http://api.tenant-b:9113/metrics allow
default/client http://frontend.shop allow
default/client http://api.shop deny
EOFChallenge
Rebuild Drill 3 with the upstream API: No Pod in a demo namespace may
reach 169.254.169.254 – and no team can reopen it via
NetworkPolicy.
Hint: Egress rules use to, and for IP
ranges there is the peer networks.
Solution
cat > ~/guard-lab/code/cilium/cnp-metadata-deny.yaml <<'EOF'
apiVersion: policy.networking.k8s.io/v1alpha2
kind: ClusterNetworkPolicy
metadata:
name: deny-cloud-metadata
spec:
tier: Admin
priority: 5
subject:
namespaces:
matchLabels: {guard-lab: demo}
egress:
- name: deny-metadata
action: Deny
to:
- networks: ["169.254.169.254/32"]
EOF
kubectl apply -f ~/guard-lab/code/cilium/cnp-metadata-deny.yaml
cc tenant-a/api http://169.254.169.254/
cc tenant-a/api http://api.tenant-aExpected (abridged, may differ depending on version):
clusternetworkpolicy.policy.networking.k8s.io/deny-cloud-metadata created
✘ blockiert tenant-a/api -> http://169.254.169.254/ (Timeout)
✔ erlaubt tenant-a/api -> http://api.tenant-a (HTTP 200)
As in Drill 3, the timeout alone proves nothing, because in kind nobody answers there. Run the cross-check without the policy before you trust the result.
Save your progress? Sign up for free
Enterprise Note
- ClusterNetworkPolicy is
v1alpha2: Fields may still change. In production, use it only with an implementation that passes the conformance tests, and with a pinned CRD version. - Assign priorities in bands (0–99 security, 100–199 platform services, …) and document them in Git. Equal priority for overlapping rules is undefined.
- On Cilium clusters, use CCNP with
enableDefaultDeny: falsefor pure guardrails – otherwise you lock out entire namespaces. - RBAC governs who may create cluster-wide policies: only the platform group, never namespace admins. Kyverno can check this as well (Day 13).
- Clean up:
kind delete cluster --name np-lab; you will needcilium-labon Day 6.