Home/Refreshers/Container forensics: refresher
Container forensics
Where a container keeps its evidence, when each part disappears, and the order to capture it in.
| Evidence | Gone when |
|---|---|
| Pod definition | The pod is deleted |
| Writable layer and memory | The container restarts |
| Logs on the node | They rotate, or the pod leaves the node |
| emptyDir volumes | The pod leaves the node |
| Image, by digest | Rarely |
In this order
| Record the pod and its image digests | kubectl get pod <pod> -o json > pod.json |
| Save logs, current and previous | kubectl logs <pod> --all-containers --previous |
| Keep the node | kubectl annotate node <node> cluster-autoscaler.kubernetes.io/scale-down-disabled=true; kubectl cordon <node> |
| Snapshot the node's disks, then quarantine | kubectl label pod <pod> quarantine=true |
Watch out
- exec and kubectl cp write into the writable layer you are preserving.
- A checkpoint archive contains every memory page, secrets included.
- A tag can move to other code. Record the digest.
Check it worked
The pod's definition, logs, and hashes are stored, and the node still exists an hour later.