Home/For responders/Incident response on AWS
Incident response on AWS
AWS keeps 90 days of changes to your account without being asked, and nothing about reads of your data unless you set that up. What you can reconstruct depends on the trail you built before the incident.
What AWS records, and what it leaves out
| Record | On by default | Kept |
|---|---|---|
| Event history | Yes, with no trail: management events, per Region | 90 days |
| A trail or event data store | Management events once you create one. Data events, such as S3 object reads, only for the resource types you add, at extra cost | As long as you keep the files, or 366 days by default in CloudTrail Lake, extendable to 3,653 |
| Network activity events | No. Calls made through your VPC endpoints, including those the endpoint policy refused, once you add them per service | As for the trail |
Management events answer "who changed this": a new access key, a new role, a changed trust policy. Data events answer "who read this". With S3 data events off, copying every object out of a bucket leaves no CloudTrail record. Switch them on for the buckets that contain sensitive data before you need them. The audit log template keeps the trail where nobody in the account can delete it.
Reading an identity out of CloudTrail
- The key tells you the kind of credential.
userIdentity.accessKeyIdstartingAKIAis a long-term IAM user key.ASIAis a temporary key from STS, such as a role session. - A temporary key leads back to where it was issued. The
AssumeRoleevent records the new key inresponseElements.credentials.accessKeyId. Search for that key to find the session's start, and who started it. - A role session names its role.
userIdentity.sessionContext.sessionIssueris the role.sourceIdentity, when your roles require it, names the person, stays the same through every role chained after it, and cannot be changed by the session. - Identity Center users sign in through their own events.
CredentialChallenge,CredentialVerification, andUserAuthentication, tied together by oneAuthWorkflowID. Their work in an account then appears as anAssumedRoleon a role namedAWSReservedSSO_<permission set>_<id>, with the user's name as the session name.
aws cloudtrail lookup-events --max-results 50 \
--lookup-attributes AttributeKey=AccessKeyId,AttributeValue=<key>
aws cloudtrail lookup-events --start-time <date> \
--lookup-attributes AttributeKey=EventName,AttributeValue=CreateAccessKey
aws cloudtrail lookup-events \
--lookup-attributes AttributeKey=Username,AttributeValue=<user or session name>
Event history is per Region. Look in every Region the account uses, and in the ones it should not use.
Containment, by what was compromised
The containment refresher puts the first move for each kind of compromise on one printable page. The commands below follow the AWS CLI reference; replace each value in angle brackets.
| Compromised | First move | Watch out |
|---|---|---|
| An IAM user's access key | Deactivate the key. Deactivating can be undone; delete it once the investigation no longer needs it
aws iam update-access-key --user-name <user> \ --access-key-id <key> --status Inactive | A second key, a console password, and anything the key created, such as a new user or role |
| A role's sessions | Revoke active sessions on the role, in the console or with this inline policy. It denies every credential issued before the time you write in it
aws iam put-role-policy --role-name <role> \ --policy-name RevokeOlderSessions \ --policy-document file://revoke.json {"Version": "2012-10-17", "Statement": {"Effect": "Deny",
"Action": "*", "Resource": "*", "Condition": {"DateLessThan":
{"aws:TokenIssueTime": "<now, as 2026-09-01T12:00:00Z>"}}}} | It does not work on service-linked roles. Roles from Identity Center permission sets are revoked through Identity Center |
| An Identity Center user | Find the user's ID and add it to a deny policy in a service control policy prepared beforehand. Then, in the Identity Center console, delete their active sessions and disable the user
aws identitystore get-user-id --identity-store-id <d-id> \
--alternate-identifier '{"UniqueAttribute": {"AttributePath": "userName", "AttributeValue": "<user>"}}'
aws organizations update-policy --policy-id <p-id> \
--content file://deny.json | Role sessions already issued last up to the permission set's session length, at most 12 hours. Keep the deny in place for 12 hours. Delete the sessions before disabling the user; after that, the console can no longer stop them |
| An EC2 instance | Termination protection, snapshots, memory with your own tool, then a quarantine security group
aws ec2 modify-instance-attribute --instance-id <id> \ --disable-api-termination aws ec2 create-snapshots --copy-tags-from-source volume \ --instance-specification InstanceId=<id> aws ec2 modify-instance-attribute --instance-id <id> \ --groups <quarantine-sg> | Changing a security group does not interrupt connections already open. A network ACL does, for the whole subnet |
| An instance role's credentials, used elsewhere | Revoke the role's older sessions, with the same inline policy on the instance role
aws iam put-role-policy --role-name <instance role> \ --policy-name RevokeOlderSessions \ --policy-document file://revoke.json | GuardDuty reports this as InstanceCredentialExfiltration, OutsideAWS or InsideAWS |
A deny in a service control policy stops every principal in the account, including its root user, and never grants anything. It has no effect on the management account or on service-linked roles.
Saved queries
16 saved queries from four templates open in this cloud's console, named as the console lists them. The Standard and Hardened levels install them; the Quick start installs none. The CloudWatch ones are in the Logs Insights query language, and the Athena ones in SQL.
| Saved query | What it answers | Template that installs it | Where to open it |
|---|---|---|---|
ai-model-logging/tokens-by-caller | Callers ranked by input and output tokens, from the invocation records. This is the query AWS publishes. | Know who is using your AI models | CloudWatch console, Logs Insights, Queries, folder ai-model-logging |
ai-model-logging/calls-by-model-and-caller | Calls and tokens by model and principal. | Know who is using your AI models | CloudWatch console, Logs Insights, Queries, folder ai-model-logging |
ai-model-logging/one-caller | What one caller asked and what came back. Replace the name in the filter first. | Know who is using your AI models | CloudWatch console, Logs Insights, Queries, folder ai-model-logging |
ai-model-logging/refused-calls-by-caller | Refused Bedrock calls by principal and error code, as when someone tests stolen keys. Reads the trail, so it covers every Region. | Know who is using your AI models | CloudWatch console, Logs Insights, Queries, folder ai-model-logging |
ai-model-logging/bearer-token-calls | Bedrock calls made with a Bedrock API key. Reads the trail, so it covers every Region. | Know who is using your AI models | CloudWatch console, Logs Insights, Queries, folder ai-model-logging |
insider-data-theft/permission-changes | Who gave whom new IAM access, newest first. Run it first when an alarm names a person. | Notice data walking out the door | CloudWatch console, Logs Insights, Queries, folder insider-data-theft |
insider-data-theft/one-principal | Everything one principal did. Replace the name in the filter first. | Notice data walking out the door | CloudWatch console, Logs Insights, Queries, folder insider-data-theft |
insider-data-theft/changes-by-person | The changes that alarm, with who made each one, for reviewing a week at once. | Notice data walking out the door | CloudWatch console, Logs Insights, Queries, folder insider-data-theft |
insider-data-theft/decrypt-grants | KMS grants that let someone decrypt, made with no AWS service acting for the caller. | Notice data walking out the door | CloudWatch console, Logs Insights, Queries, folder insider-data-theft |
insider-data-theft/bulk-readers | Who read the most S3 objects, hour by hour. Only when sensitive_bucket_arns is set. | Notice data walking out the door | CloudWatch console, Logs Insights, Queries, folder insider-data-theft |
okta/one-actor | Everything one Okta user did, newest first. Replace the login first. | Catch account takeover at the identity provider Okta | CloudWatch console, Logs Insights, Queries, folder okta |
okta/admin-and-policy-changes | Changes to who can sign in, how, and with what rights. | Catch account takeover at the identity provider Okta | CloudWatch console, Logs Insights, Queries, folder okta |
okta/sign-ins-by-source | Sign-ins by user, address, and result, to spot a spray or a new address. | Catch account takeover at the identity provider Okta | CloudWatch console, Logs Insights, Queries, folder okta |
okta/factor-changes | Whose MFA factors changed, and who changed them. | Catch account takeover at the identity provider Okta | CloudWatch console, Logs Insights, Queries, folder okta |
create-flow-log-table | Creates the flows table over the flow log files. Run it once, before the other. | Keep network records long enough to be useful | Athena query editor, Saved queries, workgroup vpc-flow-logs |
one-host-one-day | Flows and bytes for one address on one day. Edit the date and address first. | Keep network records long enough to be useful | Athena query editor, Saved queries, workgroup vpc-flow-logs |
Forensics on an EC2 instance
- Keep it. AWS's incident response guide lists termination protection, shutdown behaviour set to stop, and DeleteOnTermination turned off on every volume. Termination protection alone does not stop an Auto Scaling group, or a shutdown from inside when shutdown behaviour is terminate, and Spot instances cannot take it.
- Disks.
aws ec2 create-snapshotssnapshots every volume of an instance at one point in time. The EBS direct APIs read a snapshot's blocks without creating a volume. - Memory. AWS has no managed memory capture. The guide says to capture it with a tool of your own before isolating or shutting the instance down. The evidence template runs AVML through Systems Manager when someone tags the instance.
- Where to look at it. A separate forensics account, with a workstation that cannot reach the internet. Snapshots under the default EBS key cannot be shared with it, so encrypt volumes with a key of your own.
Detection you can switch on
| Service | What it adds |
|---|---|
| GuardDuty | Reads CloudTrail management events, VPC flow logs, and DNS queries to AWS's own resolvers from its own copies, with nothing to switch on. Protection plans add S3 data events, EKS audit logs, runtime monitoring on EKS, ECS, and EC2, malware scans of volumes and new S3 objects, RDS logins, and Lambda network activity |
| GuardDuty Extended Threat Detection | On by default at no extra cost. Joins findings and events over 24 hours into one critical attack sequence, such as compromised credentials or a compromised EKS cluster |
| Amazon Detective | Up to a year of CloudTrail, flow log, EKS audit, and GuardDuty history, arranged for investigating one identity or resource |
| Security Hub | The service AWS now calls Security Hub CSPM checks configuration. The newer Security Hub, generally available since December 2025, correlates GuardDuty, Inspector, Security Hub CSPM, and Macie findings into exposures |
| AWS Security Incident Response | A paid service that triages GuardDuty and Security Hub CSPM findings, runs containment for EC2, S3, and IAM that you authorise in advance, and brings in AWS's incident responders |
Containers on AWS
EKS adds a cluster audit log that is off until you switch it on, nodes that Auto Scaling and Karpenter replace, and Fargate pods with no node at all. Containers on GKE and EKS covers each one, and the container evidence template captures a compromised pod.