Home/Refreshers/Checkpoints and node memory: refresher
Checkpoints and node memory
Which nodes let you checkpoint a container or capture their memory, and the commands for each on EKS and GKE.
| Nodes | Checkpoint | Node memory |
|---|---|---|
| EKS, Amazon Linux 2023 | After installing CRIU | AVML through SSM |
| EKS, Bottlerocket | No | AVML from the admin container, untested |
| GKE, Container-Optimized OS | Pod snapshots, GKE Sandbox only | AVML from a privileged pod |
| GKE Autopilot | Pod snapshots, GKE Sandbox only | No |
Commands
| Checkpoint, from the node | curl -sk -X POST -H "Authorization: Bearer $TOKEN" https://localhost:10250/checkpoint/<ns>/<pod>/<container> |
| Node memory, through a pipe to S3 | mkfifo p; aws s3 cp - s3://<bucket>/<case>/<node>.lime < p & ./avml acquire p |
| Container-Optimized OS kernel | gcloud storage cp gs://cos-tools/<BUILD_ID>/vmlinux . |
| Check the kernel banner | vol -f <image> banners.Banners |
Watch out
- Grant create on nodes/checkpoint. nodes/proxy reaches exec in every container.
- Never lower kernel lockdown to capture. It needs a reboot, which loses the memory.
- AVML refuses /dev/stdout. Write into a named pipe.
- Both kinds of image contain secrets in clear.
Check it worked
The image's kernel banner matches uname -r from the node, and its hash matches the one taken while streaming.