Home/For responders/Stolen credentials running AI models
Stolen credentials running AI models
Stolen cloud credentials are used to run AI models on the victim's bill, and the access is resold. Researchers call it LLMjacking. The case is an identity case with three extra questions: what the model service recorded, what the calls cost, and what the prompts carried.
This lesson works one case on AWS from the first alarm to containment, then shows what changes on Google Cloud, Azure, and Oracle Cloud. Every alert and saved query named here comes from the Standard level of the AI model logging template.
What the published cases show
- Sysdig, May 2024. The report that named LLMjacking. Stolen AWS keys sent a model call built to fail, to learn whether the key worked. The same keys read the Bedrock logging configuration before running prompts.
- Sysdig, September 2024. Some attackers checked logging and left a logged account alone. Others deleted the logging configuration. Sysdig estimated over $100,000 a day for abuse of one top model.
- Permiso, October 2024. More than 75,000 successful model
calls in two days. Permiso found that the
oai-reverse-proxytool skips a key when invocation logging is on. - Sysdig, February 2026. An actor read the logging configuration, confirmed it was off, accepted a Marketplace agreement, and then called several models.
- Google Threat Intelligence, November 2026. The actor UNC6993 reached a model API with a misconfigured service account key and ran up over $100,000. Another actor made more than 500,000 calls in under eight hours.
- Microsoft's Storm-2139 filings, 2024 and 2025. Keys stolen from Azure OpenAI customers were used through a reverse proxy to generate prohibited content. The proxy logged content filter results to tune its bypasses.
Two habits repeat across them. Attackers look at logging before they spend, and they test a key cheaply before they use it. Both leave events you can alarm on. The research behind the template found no published case on Oracle Cloud.
A worked case on AWS
The account runs the Standard template under its default name,
ai-model-logging. The pipeline's roles are in
expected_principal_arns, and the application's roles are in
model_caller_role_arns.
The first alarm: logging checked
ai-model-logging-logging-checked emails. A principal outside the
expected roles called GetModelInvocationLoggingConfiguration. Two
ordinary causes exist: the console's logging settings page reads it, and so does
every terraform plan. Find the caller in the trail's log group. This
query is written for the lesson and is not one of the saved ones:
fields @timestamp, userIdentity.arn, userIdentity.accessKeyId, sourceIPAddress, userAgent, awsRegion
| filter eventSource = "bedrock.amazonaws.com" and eventName = "GetModelInvocationLoggingConfiguration"
| sort @timestamp desc
A person in the console, from an address you know, closes the alert. An IAM user's access key, from an address nobody recognises, opens a case. Start the decision log now, and treat the key as stolen until shown otherwise.
The second alarm: an unexpected caller
ai-model-logging-unexpected-caller emails next. A principal
outside model_caller_role_arns called a model or started a batch
job. The alarm reads the trail, so it sees every Region the trail records. Its
description names two saved queries. Run
ai-model-logging/refused-calls-by-caller first:
filter eventSource = "bedrock.amazonaws.com" and eventType = "AwsApiCall"
| filter errorCode like /AccessDenied/ or errorCode = "ValidationException"
| stats count(*) as refused, count_distinct(eventName) as operations by userIdentity.arn, errorCode
| sort refused desc
Sysdig saw stolen keys send a call with an invalid token limit. A
ValidationException meant the key could reach the model, and an
AccessDenied meant it could not. Many refusals from one principal,
across several operations, is that test.
Who is calling
The principal's ARN is in both alarms' events. A user ARN means a long-lived
access key. An assumed-role ARN means session credentials taken from wherever
that role runs. Incident response on AWS
covers reading each kind. A third kind is particular to Bedrock, the API key,
and ai-model-logging/bearer-token-calls finds its use:
fields @timestamp, userIdentity.arn, eventName, requestParameters.modelId, sourceIPAddress, userAgent, errorCode
| filter eventSource = "bedrock.amazonaws.com"
| filter @message like /callWithBearerToken.{0,3}true/
| sort @timestamp desc
- Long-term API keys leave a creation event. It is
CreateServiceSpecificCredentialon IAM, with the service namebedrock.amazonaws.com. Theapi-key-createdalarm watches it. The console makes these keys under IAM users whose names startBedrockAPIKey-. - Short-term API keys leave none. They are made on the client and never reach CloudTrail. Only their use shows.
What was asked, and how much
Bedrock's invocation record keeps the prompt and the response. Put the
principal's name into ai-model-logging/one-caller:
fields @timestamp, operation, modelId, input.inputTokenCount, output.outputTokenCount, @message
| filter identity.arn like /REPLACE-WITH-ROLE-OR-USER-NAME/
| sort @timestamp asc
@message is the whole record. Then measure the volume with
ai-model-logging/tokens-by-caller, which is AWS's own published
query, and split it by model with
ai-model-logging/calls-by-model-and-caller:
fields identity.arn as principal,
input.inputTokenCount as inTokens,
output.outputTokenCount as outTokens
| stats sum(inTokens) as totalInput,
sum(outTokens) as totalOutput,
count() as calls
by principal
| sort totalInput desc
stats count(*) as calls, sum(input.inputTokenCount) as inTokens, sum(output.outputTokenCount) as outTokens by modelId, identity.arn
| sort calls desc
- The record covers one Region. Invocation logging is set per
Region, and only for calls through
bedrock-runtime. Calls in other Regions appear in the trail queries and are missing from these three. - Cross-Region inference is recorded where the caller sent it.
Search the source Region.
inferenceRegionin CloudTrail says where the request ran. - Attribute on
identity.arnonly.requestMetadatais set by the caller.
What else the credential did
Search the trail for the same principal over the whole period the credential existed. The published cases give the event names to look for:
| Move | CloudTrail event names | Alarm in the template |
|---|---|---|
| Logging changed | PutModelInvocationLoggingConfiguration, DeleteModelInvocationLoggingConfiguration | logging-changed. A put can send the record elsewhere, so read the new destinations |
| Model access turned on | PutUseCaseForModelAccess, CreateFoundationModelAgreement, PutFoundationModelEntitlement, AcceptAgreementRequest | model-access |
| Guardrails weakened | UpdateGuardrail, DeleteGuardrail, PutEnforcedGuardrailConfiguration, DeleteEnforcedGuardrailConfiguration | guardrails-weakened. No published CloudTrail sample shows these names, so run the template's drill once |
| A key that outlasts the stolen one | CreateServiceSpecificCredential, CreateAccessKey, CreateUser | api-key-created for the first. Wiz saw an attacker create a user and a policy of its own |
| The record's key attacked | DisableKey, ScheduleKeyDeletion, PutKeyPolicy | record-key-changed |
| Looking around | ListFoundationModels, ListInferenceProfiles, ListAgents, ListKnowledgeBases, GetKnowledgeBase | None. Sysdig saw all five in one intrusion |
| Limits raised | RequestServiceQuotaIncrease, from Service Quotas | None. AWS's threat technique catalog lists it |
Most serverless models switch on at their first call. A quiet
model-access alarm says nothing about whether models were used.
Cut it off
- Disable the credential. For a user's key, run
aws iam update-access-key --status Inactive. For a role, revoke its active sessions, which denies every credential issued before now. - Deny model use for that principal. Deny
bedrock:InvokeModelandbedrock:InvokeModelWithResponseStream, which also cover the Converse calls, andbedrock:CreateModelInvocationJobfor batch jobs. AWS's compromised-key quarantine policy, AWSCompromisedKeyQuarantineV3, denies all three. It also denies bearer token calls, model agreements, entitlements, and the bedrock-mantle actions, so attaching it covers more. - Deny API keys if one was used. The action is
bedrock:CallWithBearerToken. Thebedrock-mantleendpoint has actions of its own, which a deny onbedrockactions does not name. - Do not rely on removing model access. AWS says deleting a model agreement does not block the model, because the next call creates access again. A deny on the invoke actions is what blocks it.
- Enforce the guardrail if the content matters. Set
enforce_guardrailto true and apply. The template's guardrail then screens every model call in the account and Region for prompt attacks and five kinds of harmful content. - Rotate what sat beside it. Wherever the key leaked from, every other secret stored there is suspect.
An enforced guardrail limits what a stolen credential can generate. Ordinary
prompts still run and still cost, so it follows the deny and does not replace
it. It can also block legitimate prompts. AWS says to check CloudTrail for
AccessDenied after turning it on.
Containment is confirmed in the log. Run
refused-calls-by-caller again and look for the principal's calls
failing with AccessDenied.
What each cloud records
| Cloud | Record of a model call | Names the caller by | Prompt and response text |
|---|---|---|---|
| AWS Bedrock | Model invocation logging, off by default and set per Region. CloudTrail also logs the four inference calls as management events | identity.arn in the record, userIdentity.arn in the trail | Kept. Up to 100 KB sits in the record, and larger bodies, images, and documents go to S3 |
| Google Cloud Vertex AI | Data Access audit logs, off by default | principalEmail, and serviceAccountKeyName when a key signed the call | Absent. Model Armor's logs keep the text of the Gemini calls it screens, in the projects you name |
| Azure OpenAI | The RequestResponse resource log, once a diagnostic setting exists | The caller's object ID for Entra sign-ins, and a caller IP with its last octet masked | Microsoft does not say. Named research and Elastic's documentation say it is absent |
| Oracle Cloud Generative AI | The Audit log only. Oracle does not document whether model calls reach it | The audit event's principal, if the call is recorded | No record, unless a Generative AI project is set to store them |
The log source planner lists each of these logs with its default and how long it is kept.
On Google Cloud
The same case starts with a different alert. AI models:
unexpected-caller names a principal outside model_callers
the first time they call a model. AI models: model-recon fires when
models or endpoints are listed, or a publisher model is looked up.
AI models: ai-api-enabled fires when the Vertex AI API is turned on in
a project.
Google Threat Intelligence names a service account key as one actor's way
in. The audit entry names the key that signed each call, and the saved query
AI models: calls made with a service account key lists them:
(protoPayload.serviceName="aiplatform.googleapis.com" AND protoPayload.methodName=~"[.]PredictionService[.](Predict|RawPredict|StreamRawPredict|DirectPredict|DirectRawPredict|StreamDirectPredict|StreamDirectRawPredict|StreamingPredict|ServerStreamingPredict|StreamingRawPredict|Explain|GenerateContent|StreamGenerateContent|ChatCompletions|EmbedContent)$") AND protoPayload.authenticationInfo.serviceAccountKeyName:*
AI models: one principal's model calls matches the same model calls for one principalEmail, with or without a key. The seven-day count runs in Log
Analytics:
SELECT
proto_payload.audit_log.authentication_info.principal_email AS principal,
proto_payload.audit_log.resource_name AS model,
COUNT(*) AS calls
FROM
`<security-project>.global.ai-model-logging._AllLogs`
WHERE
timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY)
AND proto_payload.audit_log.service_name = "aiplatform.googleapis.com"
AND SPLIT(proto_payload.audit_log.method_name, '.')[SAFE_OFFSET(4)] = "PredictionService"
AND SPLIT(proto_payload.audit_log.method_name, '.')[SAFE_OFFSET(5)] IN ("Predict", "RawPredict", "StreamRawPredict", "DirectPredict", "DirectRawPredict", "StreamDirectPredict", "StreamDirectRawPredict", "StreamingPredict", "ServerStreamingPredict", "StreamingRawPredict", "Explain", "GenerateContent", "StreamGenerateContent", "ChatCompletions", "EmbedContent")
GROUP BY principal, model
ORDER BY calls DESC
LIMIT 1000
- Set the scope before searching. The entries sit in the
security project's own buckets, which Logs Explorer does not search by default.
Choose Refine scope, then Scope by storage, and pick
ai-model-logging. - The entry carries no text. It names the caller, the model
in
resourceName, the caller's IP, and a user agent the caller supplies. Model Armor's entries, where it is on, sit in the workload project's_Defaultbucket. - A key for an expected account does not alert. A stolen
key for an account in
model_callerscalls as that account. The key query and the count are where its use shows. - Look for accounts made and deleted. Google Threat
Intelligence describes one actor creating and deleting short-lived service accounts
to dodge rate limits. Search for
CreateServiceAccountandCreateServiceAccountKey. Google also saw actors make extra API keys in victim projects, so search for the API Keys service'sCreateKeytoo.
Contain by disabling the key with gcloud iam service-accounts keys
disable, and by removing who may impersonate the account. Two settings in
the template narrow what any credential can do. allowed_models
sets an organisation policy that refuses every other model. Setting
model_armor_mode to inspect_and_block refuses the
Gemini calls Model Armor flags. Model Armor screens
generateContent only, and lets calls through when it is
unavailable.
On Azure
Azure's weak point is the account key, which works for whoever has it.
Storm-2139 used customer keys scraped from public sources. A key taken before
the template was applied fires no key alert, so its use shows as spend.
ai-tokens-<account> fires when an account passes
tokens_per_hour. Its description points to the saved query
ai-calls-by-caller:
AzureDiagnostics
| where TimeGenerated > ago(7d)
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES" and Category == "RequestResponse"
| extend P = parse_json(properties_s)
| extend ObjectId = tostring(P.objectId), Deployment = tostring(P.modelDeploymentName), Model = tostring(P.modelName)
| summarize Calls = count(), First = min(TimeGenerated), Last = max(TimeGenerated), Results = make_set(ResultSignature, 10), IPs = make_set(CallerIPAddress, 10) by Account = _ResourceId, ObjectId, Deployment, Model
| order by Calls desc
The saved search keeps it on one line.
Calls signed in through Entra carry the caller's object ID. Microsoft does
not document the object ID on a call signed with a key, so
ai-calls-without-object-id treats an empty value as a likely key,
to be checked:
AzureDiagnostics
| where TimeGenerated > ago(7d)
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES" and Category == "RequestResponse"
| extend P = parse_json(properties_s)
| extend ObjectId = tostring(P.objectId), Deployment = tostring(P.modelDeploymentName), Model = tostring(P.modelName)
| where isempty(ObjectId)
| summarize Calls = count(), First = min(TimeGenerated), Last = max(TimeGenerated), Operations = make_set(OperationName, 10) by Account = _ResourceId, CallerIPAddress, Deployment
| order by Calls desc
- The IP is partial. The last octet is masked, so it narrows a network and does not name a host.
- The deployment field is unverified. Microsoft does not
publish the fields inside
properties_s. If the Deployment column comes back empty, read one row and correct the key. - Expect a delay. A call can take up to two hours to become queryable.
- The activity log names who fetched a key.
ai-keys-fetchedreads it, and also fires on a regenerated key and on a connection's secrets being listed. A connection stores credentials for other services. - Content filters are the guardrail.
ai-filters-changedfires when a filter policy or blocklist is created, changed, or deleted, andai-deployment-changedwhen a model deployment is.
Contain by regenerating the account's keys, which fires
ai-keys-fetched for your own change. Then move callers to Entra
sign-in and turn key access off. Azure can keep accepting old keys for up to
several hours after that, so regenerate first. Setting
key_access_policy_effect to Deny refuses any new or
changed account that leaves keys on.
On Oracle Cloud
Oracle Cloud keeps the least. There is no record of prompts or responses,
and Oracle does not document whether model calls reach the Audit log. The
template saves a search for them, ai-inference-calls, and alarms
on none:
search "<tenancy-ocid>/_Audit_Include_Subcompartment" | where data.eventName='Chat' or data.eventName='EmbedText' or data.eventName='RerankText' or data.eventName='ApplyGuardrails' | sort by datetime desc
Make one test call today and check that it appears. If it does not, a case
here has no call record at all, and the changes around the calls are the
evidence. The four alarms cover those: ai-api-key-changed,
ai-endpoint-changed, ai-hosting-created, and
ai-logging-weakened. The search behind the key alarm:
search "<tenancy-ocid>/_Audit_Include_Subcompartment" | where data.eventName='CreateApiKey' or data.eventName='RenewApiKey' or data.eventName='SetApiKeyState' or data.eventName='UpdateApiKey' or data.eventName='DeleteApiKey' or data.eventName='ChangeApiKeyCompartment' | sort by datetime desc
- A Generative AI API key is a bearer secret. Oracle advises these keys for testing and early development, and IAM sign-in for production. What the audit event shows for a call made with one is not documented.
- Identity's signing keys share two event names.
CreateApiKeyandDeleteApiKeyalso fire for API signing keys, so read the event before reacting. - Only the home region is watched. Treat models, endpoints, and keys in other regions as unwatched, and search each region's audit log by hand.
- Alarms run about 15 minutes late. Audit usually records a call within that time.
Contain by switching the key off or deleting it in the console, and making a new one for the application. The provider has no resource for these keys, so no template can rotate them. Oracle's guidance is to rotate at once on suspicious activity.
Containment, by what was stolen
| Stolen | First move | Still open afterwards |
|---|---|---|
| An AWS access key | Deactivate it, then deny the two invoke actions and batch jobs on its user | Any user, key, or long-term Bedrock API key it created |
| An AWS role session | Revoke the role's active sessions | The place the session was taken from, until it is fixed |
| A Bedrock API key | Deny bedrock:CallWithBearerToken on the principal behind it | Short-term keys are made on the client. Their creation cannot be prevented, only their use |
| A Google Cloud service account key | Disable the key, and remove who may impersonate the account | Disabling the whole account breaks every workload that uses it |
| An Azure account key | Regenerate the keys, then turn key access off | Old keys can be accepted for up to several hours after key access is turned off |
| An Oracle Cloud Generative AI API key | Switch it off or delete it, and issue a new one by hand | Keys in regions other than the home region, which the template does not watch |
On every cloud, then look for what the credential created. Leading a cloud incident lists the audit events for new credentials, new identities, and changed trust.
What to preserve
- Each query with its context. Save the query text, the time range, and the full results together, hashed, in the locked bucket from the evidence template.
- The Bedrock record before it expires. The log group keeps
records for
retention_days, and the copy in S3 underinvocations/forcopy_retention_days. Raise both if the case could outlast them. - The record's key. Do not destroy the stack during a case. Destroy deletes the log group and schedules the record key for deletion, and without the key nothing in the kept bucket can be read.
- The audit trail for the principal. Export its events from every Region, project, or subscription. Azure keeps the activity log 90 days, and Oracle Cloud keeps Audit 365 days.
- Model Armor's entries. They sit in the workload project's
_Defaultbucket, which keeps 30 days by default. Copy them out. - The source of the leak. The repository, build log, or machine the credential was taken from is evidence too.
These records are sensitive in two ways. They contain whatever your own people pasted into a prompt. They can also contain what the attacker generated, and Microsoft's filings describe that as prohibited content. Agree with Legal who reads the attacker's prompts before analysts open them. The insider cases lesson covers custody.
Sizing the bill
| Cloud | What you can count | What is missing |
|---|---|---|
| AWS | Input and output tokens by principal and by model, from tokens-by-caller and calls-by-model-and-caller. Price each model's tokens at its rate | Regions without invocation logging. There, streaming calls leave a completion event in CloudTrail with token counts, and other calls leave only a count |
| Google Cloud | Calls by principal and model, from the seven-day count. Widen its window for a longer case | Token totals, which the saved count does not have. Price the calls from the project's billing data |
| Azure | Tokens per account from the TotalTokens metric, and calls by caller from ai-calls-by-caller | The metric has no client IP or application, so separate the attacker's share by time and by call counts |
| Oracle Cloud | The invoice. On-demand use is billed for each inference call | Any call record, unless your test call showed up in Audit |
Count CloudTrail model calls on eventType
AwsApiCall. A streaming call also writes a completion event, and
counting both doubles it.
Check for capacity bought as well as calls made. On Oracle Cloud a hosting
cluster for a pretrained or fine-tuned model commits to 744 unit-hours, and
ai-hosting-created watches for one. On AWS, look for batch jobs
with CreateModelInvocationJob.
Published figures give a sense of scale. Sysdig's first report put the worst case at $46,080 a day across four Regions. Google Threat Intelligence reports some victims reaching $1 million in total.
Sizing the data exposure
Bill theft and data exposure are separate findings. Answer each of these before the report says no data was involved:
- What the attacker sent and received. On Bedrock,
one-callershows it. On the other three clouds the call record has no text, so say that plainly. - What the credential could reach through the model service. Sysdig saw an actor list agents and knowledge bases. Calls that query a knowledge base or invoke an agent are CloudTrail data events, off by default. Without them, nothing shows what was retrieved.
- Whether the credential could read the record itself. The record contains every user's prompts. If the stolen principal could read the log group, the bucket, or the workspace, treat those prompts as exposed.
- Other secrets returned. On Azure, listing a connection's secrets returns credentials for other services. Rotate each one.
- Prompt text stored elsewhere. On Oracle Cloud, a Generative AI project can be set to store responses and conversations.
Afterwards
| Finding | The change |
|---|---|
| Nobody could say who was allowed to call models | Name them: model_caller_role_arns on AWS, model_callers on Google Cloud |
| The spend was noticed on the invoice | Set tokens_per_hour on Azure from a week of normal use |
| A long-lived key leaked from a pipeline | Cloud access without stored keys |
| Account keys were still accepted | key_access_policy_effect set to Deny, after a test subscription |
| Any model could be called | allowed_models on Google Cloud, and a deny on the invoke actions for everyone but the application roles on AWS |
| The plan role set off logging-checked every day | Add it to expected_principal_arns, and keep that list short |
The AI model logging refresher puts the template on one printable page.