iac.htora.dev · security templates

Home/For responders/Kubernetes for defenders

Kubernetes for defenders

Most Kubernetes compromises follow a handful of paths. Each one leaves traces in a different place, so knowing the path tells you where to look.

1A pod is compromiseda vulnerable app, a bad image, a stolen token2It reaches the node or the APIthrough host access or its service account3It gains more rightsa new role binding, another workload's identity4It reaches the cloud accountthrough the node's role or a workload identity
The usual shape of a cluster compromise. The last step is why a Kubernetes incident is also a cloud incident.

Four layers, three kinds of evidence

LayerWhere its evidence is
Container and nodeRuntime detection that watches system calls, such as Falco, plus the node's own disk and memory
Cluster APIThe Kubernetes audit log: every request, who made it, and what it touched
Cloud accountCloudTrail or Cloud Audit Logs, where the node's role and workload identities show up

Attack paths and what they leave

PathWhat shows itWhat stops it
Escape through a privileged pod or a host mountAudit log: a pod created with privileged, hostPath, or hostPID. Runtime: nsenter or a mount from inside a containerAn admission policy refusing those settings
A stolen service account tokenRuntime: the token file read by a shell or curl. Audit log: that account calling from an unusual addressNot mounting tokens where they are not needed, and narrow roles
Exec into a running podAudit log: a request on the pods/exec subresource, with the user. kubectl 1.31 and later sends it as a get, older clients as a create, so match the subresourceGranting pods/exec to few people, and alerting on it in production
The node's cloud credentialsCloud audit log: the node's role used from somewhere it should not beOn EKS, IMDSv2 with a hop limit of 1, which keeps pods off the metadata service
Borrowing a workload identityAudit log: a pod created with a service account bound to a cloud role. Cloud audit log: that role assumedLimiting who may create pods in that namespace, and trust conditions naming the exact service account
Bypassing admission controlAudit log: changes to webhook configurations, or pods created in an exempt namespaceSecurity policies set to fail closed, and alerts on any change to the webhooks
Escalating through role bindingsAudit log: a new cluster role binding, especially to cluster-adminFew people with the bind and escalate verbs, and an alert on every new binding

The workload identity row matters more than it looks. Anyone who can create a pod using a service account bound to a cloud role effectively has that role. Reviewing who can create pods is reviewing cloud access.

Keeping a compromised pod for investigation

Deleting the pod deletes the evidence. Keep it running and cut it off:

  1. Take it out of its controller. Remove the label its deployment selects on. The deployment starts a replacement, so the service keeps running, and leaves this pod alone.
  2. Cut its network. Add a quarantine label and a network policy that denies all traffic to and from pods with that label. This only works if the cluster's network plugin enforces network policies. Check that before an incident.
  3. Cordon the node, so nothing new is scheduled next to it.
  4. Capture. Save the pod's full definition and its logs, then snapshot the node's disks. The container evidence template does this on GKE, EKS, AKS, and OKE. Container forensics covers checkpointing a single container, and Containers on GKE and EKS and Containers on AKS and OKE what changes when you cannot reach the node. Once the evidence is safe, Clearing an attacker out of a cluster covers what outlives the pod.

Avoid exec into the pod if you can. It changes the evidence, and whoever compromised it may be watching.

Where the cluster logs are

EKS sends control plane logs, including the audit log, to CloudWatch Logs only when you switch them on, per log type.

GKE records administrative actions by default. Reads, such as someone listing secrets, are Data Access logs and are off by default.

The cluster audit logs template keeps both, turns five attack paths into alerts, and at Standard sends every entry to the SIEM.

Registry 0.23.0. Built 2026-11-17.

Made by Habibullah Tora. Source on GitHub. Code under the MIT licence, writing under CC BY 4.0.