Skip to content
Hands-on IT IT Drills

Day 13: Falco – alerts and faults

Goal: You trace a Falco alert back to node, Pod and container, silence a false positive with a narrow exception and recognize when Falco itself is broken or no longer delivers Kubernetes metadata. · Duration: approx. 60 min · Prerequisite: Day 12

Briefing

  • Falco reads every node’s system calls via eBPF and compares them with rules. It blocks nothing, it only reports – as a JSON line in the Falco Pod’s log.
  • Faults with Falco are operational questions: Where does the alert come from? Why the same one every night? Why does Falco not start after a rule change? Why is the Pod missing?
  • The container ID comes from the kernel, Pod and namespace from the container runtime.
Fault Symptom Where the answer is
Alert after exec -it Terminal shell in container output_fields, crictl
False positive from a Job the same alert every 10 min Rule, exceptions
Broken rules file one Falco Pod CrashLoopBackOff Startup log, LOAD_ERR_…
No metadata k8s_pod_name=<NA> Socket list, crictl.yaml

Drill 1: Install Falco

Falco runs as a DaemonSet, one Pod per node. modern_ebpf needs no kernel module; json_output turns every alert into a JSON line that you evaluate with jq.

# on cp1
mkdir -p ~/debug/tag13 && cd ~/debug/tag13
helm repo add falcosecurity https://falcosecurity.github.io/charts
helm repo update falcosecurity
cat > falco-values.yaml <<'EOF'
tty: true
driver:
  kind: modern_ebpf
falco:
  json_output: true
EOF
helm upgrade --install falco falcosecurity/falco -n falco --create-namespace \
  --version 9.2.2 -f falco-values.yaml --wait --timeout 5m | grep -E 'STATUS|REV'
kubectl -n falco get pods -o wide | cut -c1-72

Expected (abridged):

STATUS: deployed
REVISION: 1
NAME          READY   STATUS    RESTARTS   AGE   IP            NODE
falco-2cb48   2/2     Running   0          22s   10.42.2.105   worker2
falco-79l4k   2/2     Running   0          22s   10.42.0.131   cp1
falco-w6t4z   2/2     Running   0          22s   10.42.1.96    worker1

Chart 9.2.2 ships Falco 0.45.2. Whether Falco really works is in the startup log:

# on cp1
kubectl -n falco logs ds/falco -c falco \
  | grep -E 'Falco version|rules from|schema|k3s|BPF probe' | cut -c27-114

Expected:

Found 3 pods, using pod/falco-2cb48
Falco version: 0.45.2 (x86_64)
   /etc/falco/falco.yaml | schema validation: ok
[libs]: container: * enabled container runtime socket at '/host/run/k3s/containerd/conta
Loading rules from:
   /etc/falco/falco_rules.yaml | schema validation: ok
Opening 'syscall' source with modern BPF probe.

Three lines matter: every rules file with schema validation: ok, the container runtime’s socket and modern BPF probe. cut -c27- cuts off the timestamp; logs ds/falco reads only one of the three Pods and tells you which one.

Talos: There, the same DaemonSet first fails on Pod Security baseline (privileged, hostPath) – Day 19.

Save your progress? Sign up for free

Drill 2: Who opened the shell?

Break it: A Pod on worker1 in which someone opens a shell with a terminal:

# on cp1
kubectl create namespace tag13
cat > web.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata: {name: web, namespace: tag13}
spec:
  replicas: 1
  selector: {matchLabels: {app: web}}
  template:
    metadata: {labels: {app: web}}
    spec:
      nodeSelector: {kubernetes.io/hostname: worker1}
      containers:
      - name: web
        image: busybox:1.37
        command: [sleep, infinity]
EOF
kubectl apply -f web.yaml
kubectl -n tag13 rollout status deploy/web
kubectl -n tag13 exec -it deploy/web -- sh -c 'id; exit'

Expected:

namespace/tag13 created
deployment.apps/web created
deployment "web" successfully rolled out
uid=0(root) gid=0(root) groups=0(root),10(wheel)

Symptom: A Falco Pod’s log contains a long JSON line. The readable part is in output:

# on cp1
kubectl -n falco logs -l app.kubernetes.io/name=falco -c falco --tail=-1 \
  | grep 'Terminal shell' | jq -r .output | fold -w 86

Expected (abridged):

10:28:26.553927374: Notice A shell was spawned in a container with an attached termina
l | evt_type=execve user=root user_uid=0 user_loginuid=-1 process=sh proc_exepath=/bin
/sh parent=containerd-shim command=sh -c id; exit terminal=34816 exe_flags=EXE_WRITABL
E|EXE_LOWER_LAYER container_id=03479ff2b911 container_name=web container_image_reposit
ory=docker.io/library/busybox container_image_tag=1.37 k8s_pod_name=web-6f88f4fd9c-l5r
b4 k8s_ns_name=tag13

Investigate: With jq, you pull the fields out of output_fields. fromjson? skips the text lines from startup:

# on cp1
kubectl -n falco logs -l app.kubernetes.io/name=falco -c falco --tail=-1 \
  | jq -cR 'fromjson? | select(.rule=="Terminal shell in container")
      | {hostname, ns: .output_fields["k8s.ns.name"],
         pod: .output_fields["k8s.pod.name"], id: .output_fields["container.id"],
         cmd: .output_fields["proc.cmdline"]}'

Expected (line wrapped):

{"hostname":"worker1","ns":"tag13","pod":"web-6f88f4fd9c-l5rb4","id":"03479ff2b911",
  "cmd":"sh -c id; exit"}

hostname is the node, container.id the first twelve characters of the container ID. It is the reliable anchor, even when Pod names are missing. Take the container ID from your output – once via the API, once directly from the node’s runtime:

# on cp1
J='{range .items[*]}{.metadata.namespace} {.metadata.name} '
J="$J"'{.status.containerStatuses[*].containerID}{"\n"}{end}'
kubectl get pods -A -o jsonpath="$J" | grep 03479ff2b911 | cut -c1-60
ssh -n worker1 sudo -i crictl inspect 03479ff2b911 | jq .status.labels

Expected:

tag13 web-6f88f4fd9c-l5rb4 containerd://03479ff2b911ba953f53
{
  "io.kubernetes.container.name": "web",
  "io.kubernetes.pod.name": "web-6f88f4fd9c-l5rb4",
  "io.kubernetes.pod.namespace": "tag13",
  "io.kubernetes.pod.uid": "3ece3899-938c-4195-a6eb-590282063b47"
}

These exact container labels are what Falco reads through the runtime socket – keep that in mind for Drill 5.

Cause: No defect: the rule Terminal shell in container reports every shell with a terminal that enters a container as a new process – which is exactly what kubectl exec -it does. user_loginuid=-1 means “no login”. Falco does not know who issued the command; only the API server’s audit log has that (off by default in RKE2).

Fix: None needed – but know the limits of the rule.

Verify: Without -it, Falco stays quiet:

# on cp1
kubectl -n tag13 exec deploy/web -- id
sleep 5
kubectl -n falco logs -l app.kubernetes.io/name=falco -c falco --tail=-1 \
  | grep -c 'Terminal shell'

Expected:

uid=0(root) gid=0(root) groups=0(root),10(wheel)
1

Still exactly one hit – the one from exec -it.

Save your progress? Sign up for free

Drill 3: The false positive every ten minutes

Break it: A cleanup Job (aufraeumer, ‘cleaner’) deletes completed Pods every ten minutes. To do so, it talks to the Kubernetes API with kubectl:

# on cp1
cat > aufraeumer.yaml <<'EOF'
apiVersion: v1
kind: ServiceAccount
metadata: {name: aufraeumer, namespace: tag13}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata: {name: aufraeumer, namespace: tag13}
rules:
- apiGroups: [""]
  resources: [pods]
  verbs: [get, list, delete]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata: {name: aufraeumer, namespace: tag13}
subjects:
- {kind: ServiceAccount, name: aufraeumer, namespace: tag13}
roleRef: {apiGroup: rbac.authorization.k8s.io, kind: Role, name: aufraeumer}
---
apiVersion: batch/v1
kind: CronJob
metadata: {name: aufraeumer, namespace: tag13}
spec:
  schedule: "*/10 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccountName: aufraeumer
          restartPolicy: Never
          containers:
          - name: kubectl
            image: registry.k8s.io/kubectl:v1.37.2
            args: [delete, pods, -n, tag13, --field-selector=status.phase==Succeeded]
EOF
kubectl apply -f aufraeumer.yaml
kubectl -n tag13 create job --from=cronjob/aufraeumer aufraeumer-1
kubectl -n tag13 wait --for=condition=complete job/aufraeumer-1 --timeout=90s

Expected (abridged):

cronjob.batch/aufraeumer created
job.batch/aufraeumer-1 created
job.batch/aufraeumer-1 condition met

create job --from=cronjob/… starts the run immediately instead of waiting for the next tenth minute.

Symptom:

# on cp1
kubectl -n falco logs -l app.kubernetes.io/name=falco -c falco --tail=-1 \
  | grep 'Contact K8S' | jq -r .output | fold -w 86

Expected (abridged):

10:28:55.224711757: Notice Unexpected connection to K8s API Server from container | co
nnection=10.42.0.132:40924->10.43.0.1:443 lport=40924 rport=443 fd_type=ipv4 fd_proto=
tcp evt_type=connect user=root user_uid=0 user_loginuid=-1 process=kubectl proc_exepat
h=/bin/kubectl parent=containerd-shim command=kubectl delete pods -n tag13 --field-sel
ector=status.phase==Succeeded terminal=0 container_id=6eaeb4308ebe container_name=kube
ctl container_image_repository=registry.k8s.io/kubectl container_image_tag=v1.37.2 k8s
_pod_name=aufraeumer-1-cvmcn k8s_ns_name=tag13

From now on, six times an hour. Anyone who sees this for a week stops reading Falco alerts.

Investigate: Which condition triggered? The rule, as Falco loaded it, is in the Pod:

# on cp1
kubectl -n falco exec ds/falco -c falco -- \
  grep -A14 'rule: Contact K8S API Server From Container' /etc/falco/falco_rules.yaml \
  | sed -n '1p;/condition/,$p'
kubectl -n falco exec ds/falco -c falco -- \
  grep -A1 'macro: k8s_api_server' /etc/falco/falco_rules.yaml

Expected:

- rule: Contact K8S API Server From Container
  condition: >
    evt.type=connect
    and (fd.typechar=4 or fd.typechar=6)
    and container
    and k8s_api_server
    and not k8s_containers
    and not user_known_contact_k8s_api_server_activities
- macro: k8s_api_server
  condition: (fd.sip.name="kubernetes.default.svc.cluster.local")

k8s_containers is a fixed list of known images plus kube-system; our Job is not on it.

Cause: The rule is deliberately noisy: a container that talks to the API could be an attacker with a stolen token. Our Job, however, does so legitimately and regularly – the noise hides real hits of the same rule.

Fix: Do not disable the rule; add a narrow exception instead: this namespace, this image, this process – that is Drill 4.

Save your progress? Sign up for free

Drill 4: The exception takes Falco down

Break it: The first attempt at an exception – fields named, values forgotten:

# on cp1
cat > ausnahme.yaml <<'EOF'
customRules:
  tag13-ausnahmen.yaml: |-
    - rule: Contact K8S API Server From Container
      exceptions:
      - name: tag13_aufraeumer
        fields: [k8s.ns.name, container.image.repository, proc.name]
        comps: [=, =, =]
      override:
        exceptions: append
EOF
helm upgrade falco falcosecurity/falco -n falco --version 9.2.2 \
  -f falco-values.yaml -f ausnahme.yaml --wait --timeout 90s 2>&1 | tail -2

Expected (after 90 s, line wrapped):

Error: UPGRADE FAILED: resource DaemonSet/falco/falco not ready. status: InProgress,
  message: Updated: 1/3
context deadline exceeded

Symptom:

# on cp1
kubectl -n falco get pods
helm -n falco history falco -o json | jq -r '.[] | "\(.revision) \(.status)"'

Expected:

NAME          READY   STATUS             RESTARTS      AGE
falco-2cb48   2/2     Running            0             2m50s
falco-79l4k   2/2     Running            0             2m50s
falco-sjnrl   1/2     CrashLoopBackOff   3 (34s ago)   87s
1 deployed
2 failed

Investigate: Take the Pod name from your output:

# on cp1
kubectl -n falco logs falco-sjnrl -c falco \
  | grep -E 'ausnahmen.yaml \||^Error|LOAD_ERR' | sed 's/^.*[0-9]\{4\}: //'
kubectl -n falco get ds falco -o jsonpath='{.spec.updateStrategy}{"\n"}'

Expected (lines wrapped):

   /etc/falco/rules.d/tag13-ausnahmen.yaml | schema validation: ok
Error: /etc/falco/rules.d/tag13-ausnahmen.yaml: Invalid
LOAD_ERR_VALIDATE (Error validating rule/macro/list/exception objects): Rule
  exception must have values property with a list of values
{"rollingUpdate":{"maxSurge":0,"maxUnavailable":1},"type":"RollingUpdate"}

Cause: Every exception needs values – a list of tuples matching fields and comps. schema validation: ok is misleading: the schema only checks the file’s shape; the rule engine rejects it afterwards and Falco exits. The DaemonSet replaces only one Pod at a time (maxUnavailable: 1): two nodes keep running with the old rules, one node is blind – and Helm only reports a timeout.

Fix:

# on cp1
cat > ausnahme.yaml <<'EOF'
customRules:
  tag13-ausnahmen.yaml: |-
    - rule: Contact K8S API Server From Container
      exceptions:
      - name: tag13_aufraeumer
        fields: [k8s.ns.name, container.image.repository, proc.name]
        comps: [=, =, =]
        values:
        - [tag13, registry.k8s.io/kubectl, kubectl]
      override:
        exceptions: append
EOF
helm upgrade falco falcosecurity/falco -n falco --version 9.2.2 \
  -f falco-values.yaml -f ausnahme.yaml --wait --timeout 5m | grep -E 'STATUS|REV'

Expected:

STATUS: deployed
REVISION: 3

override: exceptions: append appends the exception to the existing rule without touching its condition or output. The chart places customRules in /etc/falco/rules.d/, which Falco loads after the default rules.

Verify: A new run of the Job stays quiet, while the rule itself is still alive:

# on cp1
kubectl -n falco logs ds/falco -c falco | grep rules.d | cut -c27-
kubectl -n tag13 create job --from=cronjob/aufraeumer aufraeumer-2
kubectl -n tag13 wait --for=condition=complete job/aufraeumer-2 --timeout=90s
kubectl -n tag13 run apitest --image=busybox:1.37 --restart=Never -- \
  wget -T 2 -q -O /dev/null https://kubernetes.default.svc
sleep 15
kubectl -n falco logs -l app.kubernetes.io/name=falco -c falco --tail=-1 \
  | jq -rR 'fromjson? | [.rule, .output_fields["k8s.pod.name"]] | @tsv'

Expected:

Found 3 pods, using pod/falco-pw5bs
   /etc/falco/rules.d/tag13-ausnahmen.yaml | schema validation: ok
job.batch/aufraeumer-2 created
job.batch/aufraeumer-2 condition met
pod/apitest created
Contact K8S API Server From Container   apitest

By the way, the alerts from Drills 2 and 3 are gone: every Helm upgrade restarts the Falco Pods, and their log starts from scratch. That is why, in production, alerts belong in a central system, not in the Pod log.

Save your progress? Sign up for free

Drill 5: Alerts without a Pod

Break it: Someone “standardizes” the Falco values for all clusters and sets the CRI sockets to containerd’s default path:

# on cp1
cat > sockets.yaml <<'EOF'
collectors:
  containerEngine:
    engines:
      cri:
        enabled: true
        sockets: [/run/containerd/containerd.sock]
EOF
helm upgrade falco falcosecurity/falco -n falco --version 9.2.2 -f falco-values.yaml \
  -f ausnahme.yaml -f sockets.yaml --wait --timeout 5m | grep STATUS
kubectl -n tag13 exec -it deploy/web -- sh -c exit

Expected:

STATUS: deployed

Symptom:

# on cp1
kubectl -n falco logs -l app.kubernetes.io/name=falco -c falco --tail=-1 \
  | grep 'Terminal shell' | jq -r .output | grep -o 'container_id.*'

Expected (line wrapped):

container_id=03479ff2b911 container_name=<NA> container_image_repository=<NA>
  container_image_tag=<NA> k8s_pod_name=<NA> k8s_ns_name=<NA>

Falco is running, all Pods are Running, the alerts arrive. Only Pod, namespace and image are missing – and the exception from Drill 4 compares k8s.ns.name. A new cleanup run shows up again:

# on cp1
kubectl -n tag13 create job --from=cronjob/aufraeumer aufraeumer-3
kubectl -n tag13 wait --for=condition=complete job/aufraeumer-3 --timeout=90s
sleep 5
kubectl -n falco logs -l app.kubernetes.io/name=falco -c falco --tail=-1 \
  | jq -rR 'fromjson? | [.rule, .output_fields["container.id"]] | @tsv'

Expected (abridged):

job.batch/aufraeumer-3 created
job.batch/aufraeumer-3 condition met
Contact K8S API Server From Container   616776102f5b
Terminal shell in container 03479ff2b911

Investigate: The container ID comes from the kernel; everything else is queried from the runtime by the container plugin. Which sockets is Falco looking for, and where is the real one?

# on cp1
kubectl -n falco logs ds/falco -c falco | grep 'runtime socket' | cut -c27-114
ssh -n worker1 sudo grep runtime-endpoint /var/lib/rancher/rke2/agent/etc/crictl.yaml
ssh -n worker1 sudo ls /run/containerd/containerd.sock \
  /run/k3s/containerd/containerd.sock

Expected (abridged):

Found 3 pods, using pod/falco-pqzlp
[libs]: container: * enabled container runtime socket at '/host/run/podman/podman.sock'
[libs]: container: * enabled container runtime socket at '/host/var/run/docker.sock'
[libs]: container: * enabled container runtime socket at '/host/run/containerd/contain
[libs]: container: * enabled container runtime socket at '/host/run/host-containerd/co
runtime-endpoint: unix:///run/k3s/containerd/containerd.sock
ls: cannot access '/run/containerd/containerd.sock': No such file or directory
/run/k3s/containerd/containerd.sock

Cause: RKE2 ships its own containerd, whose socket lives under /run/k3s/containerd/. The values replace the chart’s list completely (lists are replaced with -f, not merged), so Falco only queries paths where no runtime answers. Without a runtime, all container and Pod fields stay empty: <NA>. Incidentally, the chart only mounts the directories of the configured sockets – /run/k3s/containerd is now missing from the volumes as well.

Fix: Drop the override; the chart’s list already contains the RKE2 path:

# on cp1
helm upgrade falco falcosecurity/falco -n falco --version 9.2.2 \
  -f falco-values.yaml -f ausnahme.yaml --wait --timeout 5m | grep STATUS
kubectl -n tag13 exec -it deploy/web -- sh -c exit

Expected:

STATUS: deployed

Verify:

# on cp1
kubectl -n falco get ds falco \
  -o jsonpath='{range .spec.template.spec.volumes[*]}{.hostPath.path}{"\n"}{end}' \
  | grep k3s
kubectl -n falco logs -l app.kubernetes.io/name=falco -c falco --tail=-1 \
  | grep 'Terminal shell' | jq -r .output | grep -o 'k8s_pod_name.*'

Expected:

/run/k3s/containerd
k8s_pod_name=web-6f88f4fd9c-l5rb4 k8s_ns_name=tag13

Talos: There, /run/containerd/containerd.sock is the correct path for the kubelet’s Pods – the same path that breaks everything on RKE2 (Day 19).

Save your progress? Sign up for free

Exercise: Speed Drills

  1. Count all Falco alerts in the cluster by node and rule. Target time: 60 s
  2. Show which rules files the Falco Pod on cp1 has loaded. Target time: 30 s
  3. For the web Pod’s container ID (from an alert), find namespace and Pod via the API. Target time: 45 s

Solution

# on cp1
kubectl -n falco logs -l app.kubernetes.io/name=falco -c falco --tail=-1 \
  | jq -rR 'fromjson? | [.hostname, .rule] | @tsv' | sort | uniq -c
P=$(kubectl -n falco get pod -l app.kubernetes.io/name=falco \
  --field-selector spec.nodeName=cp1 -o name)
kubectl -n falco logs $P -c falco | grep 'schema validation' | cut -c27-
J='{range .items[*]}{.metadata.namespace} {.metadata.name} '
J="$J"'{.status.containerStatuses[*].containerID}{"\n"}{end}'
kubectl get pods -A -o jsonpath="$J" | grep 03479ff2b911 | cut -c1-60

Expected:

      1 worker1 Terminal shell in container
   /etc/falco/falco.yaml | schema validation: ok
   /etc/falco/falco_rules.yaml | schema validation: ok
   /etc/falco/rules.d/tag13-ausnahmen.yaml | schema validation: ok
tag13 web-6f88f4fd9c-l5rb4 containerd://03479ff2b911ba953f53

Save your progress? Sign up for free

Checkpoint

# on cp1
kubectl -n falco get ds falco | cut -c1-60
kubectl -n falco logs ds/falco -c falco | grep -c 'modern BPF probe'

Expected:

NAME    DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE
falco   3         3         3       3            3
Found 3 pods, using pod/falco-nqw96
1

Challenge

A hardening check pruefer (‘checker’) verifies every night whether the image has an entry for root in /etc/shadow. Falco reports it as Read sensitive file untrusted. Silence exactly this alert – other readers of /etc/shadow should still stand out, and the exception from Drill 4 must be kept.

# on cp1
cat > pruefer.yaml <<'EOF'
apiVersion: batch/v1
kind: CronJob
metadata: {name: pruefer, namespace: tag13}
spec:
  schedule: "0 3 * * *"
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: Never
          containers:
          - name: pruefer
            image: busybox:1.37
            command: [grep, -c, root, /etc/shadow]
EOF
kubectl apply -f pruefer.yaml
kubectl -n tag13 create job --from=cronjob/pruefer pruefer-1

Hint: The same key under customRules in two -f files means the second file wins. So the second exception belongs either in the same rules file or under a new key.

Solution

First wait until the Job has run and Falco has written the alert, then look at the alert’s fields and build the exception from exactly these fields:

# on cp1
kubectl -n tag13 wait --for=condition=complete job/pruefer-1 --timeout=60s
sleep 15
kubectl -n falco logs -l app.kubernetes.io/name=falco -c falco --tail=-1 \
  | jq -cR 'fromjson? | select(.rule=="Read sensitive file untrusted")
      | .output_fields | {"k8s.ns.name", "container.name", "proc.name", "fd.name"}'

Expected:

job.batch/pruefer-1 condition met
{"k8s.ns.name":"tag13","container.name":"pruefer","proc.name":"grep",
  "fd.name":"/etc/shadow"}
# on cp1
cat >> ausnahme.yaml <<'EOF'
    - rule: Read sensitive file untrusted
      exceptions:
      - name: tag13_pruefer
        fields: [k8s.ns.name, container.name, proc.name, fd.name]
        comps: [=, =, =, =]
        values:
        - [tag13, pruefer, grep, /etc/shadow]
      override:
        exceptions: append
EOF
helm upgrade falco falcosecurity/falco -n falco --version 9.2.2 \
  -f falco-values.yaml -f ausnahme.yaml --wait --timeout 5m | grep STATUS
kubectl -n tag13 create job --from=cronjob/pruefer pruefer-2
kubectl -n tag13 wait --for=condition=complete job/pruefer-2 --timeout=60s
kubectl -n tag13 run leser --image=busybox:1.37 --restart=Never -- cat /etc/shadow
sleep 15
kubectl -n falco logs -l app.kubernetes.io/name=falco -c falco --tail=-1 \
  | jq -rR 'fromjson? | [.rule, .output_fields["k8s.pod.name"]] | @tsv'

Expected:

STATUS: deployed
job.batch/pruefer-2 created
job.batch/pruefer-2 condition met
pod/leser created
Read sensitive file untrusted   leser

pruefer-2 stays quiet, leser (‘reader’) still stands out. The exception is tied to the container name, the process and the file – a cat /etc/shadow in the same container would still be reported.

Save your progress? Sign up for free

Cleanup

Falco is not needed later and costs memory on every node:

# on cp1
helm uninstall falco -n falco
kubectl delete namespace falco tag13
helm list -A | grep -c falco

Expected:

release "falco" uninstalled
namespace "falco" deleted
namespace "tag13" deleted
0

Interview Note

  • Where does a Falco alert come from? hostname = node, container.id from the kernel, Pod and namespace via the container runtime; only the API server’s audit log knows the human behind it.
  • False positive – what now? Never disable the rule; build a narrow exceptions list with override: exceptions: append from the alert’s fields.
  • <NA> in every alert? Falco cannot reach the runtime – check the socket list against the real path (crictl.yaml); on RKE2 it is /run/k3s/containerd/containerd.sock.

Enterprise Note

  • Collect alerts centrally (Falcosidekick, SIEM), not in the Pod log: every restart wipes it, and a blind node only stands out if someone monitors the number of reporting nodes.
  • Check rule changes in advance: falco --dry-run or a test Pod with the new rules file in CI; the schema alone does not catch missing values.
  • Version and review exceptions like code – every exception is a deliberate security hole.
  • Falco for attack detection and hardening: “CKS hands-on” and “Kubernetes attack and defense hands-on”.