iac.htora.dev · security templates

Home/For responders/Stolen credentials running AI models

Stolen credentials running AI models

Stolen cloud credentials are used to run AI models on the victim's bill, and the access is resold. Researchers call it LLMjacking. The case is an identity case with three extra questions: what the model service recorded, what the calls cost, and what the prompts carried.

1An alertlogging checked, an unexpected caller, or a token spike2Attributewhich key or role session, and from where3Scopemodels, Regions, and what else the credential did4Containthe credential first, then model access5Size itthe bill, and the data in the prompts
Preserve alongside every step. Export each query's results as you run it, with the query text and the time range.

This lesson works one case on AWS from the first alarm to containment, then shows what changes on Google Cloud, Azure, and Oracle Cloud. Every alert and saved query named here comes from the Standard level of the AI model logging template.

What the published cases show

  • Sysdig, May 2024. The report that named LLMjacking. Stolen AWS keys sent a model call built to fail, to learn whether the key worked. The same keys read the Bedrock logging configuration before running prompts.
  • Sysdig, September 2024. Some attackers checked logging and left a logged account alone. Others deleted the logging configuration. Sysdig estimated over $100,000 a day for abuse of one top model.
  • Permiso, October 2024. More than 75,000 successful model calls in two days. Permiso found that the oai-reverse-proxy tool skips a key when invocation logging is on.
  • Sysdig, February 2026. An actor read the logging configuration, confirmed it was off, accepted a Marketplace agreement, and then called several models.
  • Google Threat Intelligence, November 2026. The actor UNC6993 reached a model API with a misconfigured service account key and ran up over $100,000. Another actor made more than 500,000 calls in under eight hours.
  • Microsoft's Storm-2139 filings, 2024 and 2025. Keys stolen from Azure OpenAI customers were used through a reverse proxy to generate prohibited content. The proxy logged content filter results to tune its bypasses.

Two habits repeat across them. Attackers look at logging before they spend, and they test a key cheaply before they use it. Both leave events you can alarm on. The research behind the template found no published case on Oracle Cloud.

A worked case on AWS

The account runs the Standard template under its default name, ai-model-logging. The pipeline's roles are in expected_principal_arns, and the application's roles are in model_caller_role_arns.

The first alarm: logging checked

ai-model-logging-logging-checked emails. A principal outside the expected roles called GetModelInvocationLoggingConfiguration. Two ordinary causes exist: the console's logging settings page reads it, and so does every terraform plan. Find the caller in the trail's log group. This query is written for the lesson and is not one of the saved ones:

fields @timestamp, userIdentity.arn, userIdentity.accessKeyId, sourceIPAddress, userAgent, awsRegion
| filter eventSource = "bedrock.amazonaws.com" and eventName = "GetModelInvocationLoggingConfiguration"
| sort @timestamp desc

A person in the console, from an address you know, closes the alert. An IAM user's access key, from an address nobody recognises, opens a case. Start the decision log now, and treat the key as stolen until shown otherwise.

The second alarm: an unexpected caller

ai-model-logging-unexpected-caller emails next. A principal outside model_caller_role_arns called a model or started a batch job. The alarm reads the trail, so it sees every Region the trail records. Its description names two saved queries. Run ai-model-logging/refused-calls-by-caller first:

filter eventSource = "bedrock.amazonaws.com" and eventType = "AwsApiCall"
| filter errorCode like /AccessDenied/ or errorCode = "ValidationException"
| stats count(*) as refused, count_distinct(eventName) as operations by userIdentity.arn, errorCode
| sort refused desc

Sysdig saw stolen keys send a call with an invalid token limit. A ValidationException meant the key could reach the model, and an AccessDenied meant it could not. Many refusals from one principal, across several operations, is that test.

Who is calling

The principal's ARN is in both alarms' events. A user ARN means a long-lived access key. An assumed-role ARN means session credentials taken from wherever that role runs. Incident response on AWS covers reading each kind. A third kind is particular to Bedrock, the API key, and ai-model-logging/bearer-token-calls finds its use:

fields @timestamp, userIdentity.arn, eventName, requestParameters.modelId, sourceIPAddress, userAgent, errorCode
| filter eventSource = "bedrock.amazonaws.com"
| filter @message like /callWithBearerToken.{0,3}true/
| sort @timestamp desc
  • Long-term API keys leave a creation event. It is CreateServiceSpecificCredential on IAM, with the service name bedrock.amazonaws.com. The api-key-created alarm watches it. The console makes these keys under IAM users whose names start BedrockAPIKey-.
  • Short-term API keys leave none. They are made on the client and never reach CloudTrail. Only their use shows.

What was asked, and how much

Bedrock's invocation record keeps the prompt and the response. Put the principal's name into ai-model-logging/one-caller:

fields @timestamp, operation, modelId, input.inputTokenCount, output.outputTokenCount, @message
| filter identity.arn like /REPLACE-WITH-ROLE-OR-USER-NAME/
| sort @timestamp asc

@message is the whole record. Then measure the volume with ai-model-logging/tokens-by-caller, which is AWS's own published query, and split it by model with ai-model-logging/calls-by-model-and-caller:

fields identity.arn as principal,
       input.inputTokenCount as inTokens,
       output.outputTokenCount as outTokens
| stats sum(inTokens) as totalInput,
        sum(outTokens) as totalOutput,
        count() as calls
        by principal
| sort totalInput desc
stats count(*) as calls, sum(input.inputTokenCount) as inTokens, sum(output.outputTokenCount) as outTokens by modelId, identity.arn
| sort calls desc
  • The record covers one Region. Invocation logging is set per Region, and only for calls through bedrock-runtime. Calls in other Regions appear in the trail queries and are missing from these three.
  • Cross-Region inference is recorded where the caller sent it. Search the source Region. inferenceRegion in CloudTrail says where the request ran.
  • Attribute on identity.arn only. requestMetadata is set by the caller.

What else the credential did

Search the trail for the same principal over the whole period the credential existed. The published cases give the event names to look for:

MoveCloudTrail event namesAlarm in the template
Logging changedPutModelInvocationLoggingConfiguration, DeleteModelInvocationLoggingConfigurationlogging-changed. A put can send the record elsewhere, so read the new destinations
Model access turned onPutUseCaseForModelAccess, CreateFoundationModelAgreement, PutFoundationModelEntitlement, AcceptAgreementRequestmodel-access
Guardrails weakenedUpdateGuardrail, DeleteGuardrail, PutEnforcedGuardrailConfiguration, DeleteEnforcedGuardrailConfigurationguardrails-weakened. No published CloudTrail sample shows these names, so run the template's drill once
A key that outlasts the stolen oneCreateServiceSpecificCredential, CreateAccessKey, CreateUserapi-key-created for the first. Wiz saw an attacker create a user and a policy of its own
The record's key attackedDisableKey, ScheduleKeyDeletion, PutKeyPolicyrecord-key-changed
Looking aroundListFoundationModels, ListInferenceProfiles, ListAgents, ListKnowledgeBases, GetKnowledgeBaseNone. Sysdig saw all five in one intrusion
Limits raisedRequestServiceQuotaIncrease, from Service QuotasNone. AWS's threat technique catalog lists it

Most serverless models switch on at their first call. A quiet model-access alarm says nothing about whether models were used.

Cut it off

  1. Disable the credential. For a user's key, run aws iam update-access-key --status Inactive. For a role, revoke its active sessions, which denies every credential issued before now.
  2. Deny model use for that principal. Deny bedrock:InvokeModel and bedrock:InvokeModelWithResponseStream, which also cover the Converse calls, and bedrock:CreateModelInvocationJob for batch jobs. AWS's compromised-key quarantine policy, AWSCompromisedKeyQuarantineV3, denies all three. It also denies bearer token calls, model agreements, entitlements, and the bedrock-mantle actions, so attaching it covers more.
  3. Deny API keys if one was used. The action is bedrock:CallWithBearerToken. The bedrock-mantle endpoint has actions of its own, which a deny on bedrock actions does not name.
  4. Do not rely on removing model access. AWS says deleting a model agreement does not block the model, because the next call creates access again. A deny on the invoke actions is what blocks it.
  5. Enforce the guardrail if the content matters. Set enforce_guardrail to true and apply. The template's guardrail then screens every model call in the account and Region for prompt attacks and five kinds of harmful content.
  6. Rotate what sat beside it. Wherever the key leaked from, every other secret stored there is suspect.

An enforced guardrail limits what a stolen credential can generate. Ordinary prompts still run and still cost, so it follows the deny and does not replace it. It can also block legitimate prompts. AWS says to check CloudTrail for AccessDenied after turning it on.

Containment is confirmed in the log. Run refused-calls-by-caller again and look for the principal's calls failing with AccessDenied.

What each cloud records

CloudRecord of a model callNames the caller byPrompt and response text
AWS BedrockModel invocation logging, off by default and set per Region. CloudTrail also logs the four inference calls as management eventsidentity.arn in the record, userIdentity.arn in the trailKept. Up to 100 KB sits in the record, and larger bodies, images, and documents go to S3
Google Cloud Vertex AIData Access audit logs, off by defaultprincipalEmail, and serviceAccountKeyName when a key signed the callAbsent. Model Armor's logs keep the text of the Gemini calls it screens, in the projects you name
Azure OpenAIThe RequestResponse resource log, once a diagnostic setting existsThe caller's object ID for Entra sign-ins, and a caller IP with its last octet maskedMicrosoft does not say. Named research and Elastic's documentation say it is absent
Oracle Cloud Generative AIThe Audit log only. Oracle does not document whether model calls reach itThe audit event's principal, if the call is recordedNo record, unless a Generative AI project is set to store them

The log source planner lists each of these logs with its default and how long it is kept.

On Google Cloud

The same case starts with a different alert. AI models: unexpected-caller names a principal outside model_callers the first time they call a model. AI models: model-recon fires when models or endpoints are listed, or a publisher model is looked up. AI models: ai-api-enabled fires when the Vertex AI API is turned on in a project.

Google Threat Intelligence names a service account key as one actor's way in. The audit entry names the key that signed each call, and the saved query AI models: calls made with a service account key lists them:

(protoPayload.serviceName="aiplatform.googleapis.com" AND protoPayload.methodName=~"[.]PredictionService[.](Predict|RawPredict|StreamRawPredict|DirectPredict|DirectRawPredict|StreamDirectPredict|StreamDirectRawPredict|StreamingPredict|ServerStreamingPredict|StreamingRawPredict|Explain|GenerateContent|StreamGenerateContent|ChatCompletions|EmbedContent)$") AND protoPayload.authenticationInfo.serviceAccountKeyName:*

AI models: one principal's model calls matches the same model calls for one principalEmail, with or without a key. The seven-day count runs in Log Analytics:

SELECT
  proto_payload.audit_log.authentication_info.principal_email AS principal,
  proto_payload.audit_log.resource_name AS model,
  COUNT(*) AS calls
FROM
  `<security-project>.global.ai-model-logging._AllLogs`
WHERE
  timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY)
  AND proto_payload.audit_log.service_name = "aiplatform.googleapis.com"
  AND SPLIT(proto_payload.audit_log.method_name, '.')[SAFE_OFFSET(4)] = "PredictionService"
  AND SPLIT(proto_payload.audit_log.method_name, '.')[SAFE_OFFSET(5)] IN ("Predict", "RawPredict", "StreamRawPredict", "DirectPredict", "DirectRawPredict", "StreamDirectPredict", "StreamDirectRawPredict", "StreamingPredict", "ServerStreamingPredict", "StreamingRawPredict", "Explain", "GenerateContent", "StreamGenerateContent", "ChatCompletions", "EmbedContent")
GROUP BY principal, model
ORDER BY calls DESC
LIMIT 1000
  • Set the scope before searching. The entries sit in the security project's own buckets, which Logs Explorer does not search by default. Choose Refine scope, then Scope by storage, and pick ai-model-logging.
  • The entry carries no text. It names the caller, the model in resourceName, the caller's IP, and a user agent the caller supplies. Model Armor's entries, where it is on, sit in the workload project's _Default bucket.
  • A key for an expected account does not alert. A stolen key for an account in model_callers calls as that account. The key query and the count are where its use shows.
  • Look for accounts made and deleted. Google Threat Intelligence describes one actor creating and deleting short-lived service accounts to dodge rate limits. Search for CreateServiceAccount and CreateServiceAccountKey. Google also saw actors make extra API keys in victim projects, so search for the API Keys service's CreateKey too.

Contain by disabling the key with gcloud iam service-accounts keys disable, and by removing who may impersonate the account. Two settings in the template narrow what any credential can do. allowed_models sets an organisation policy that refuses every other model. Setting model_armor_mode to inspect_and_block refuses the Gemini calls Model Armor flags. Model Armor screens generateContent only, and lets calls through when it is unavailable.

On Azure

Azure's weak point is the account key, which works for whoever has it. Storm-2139 used customer keys scraped from public sources. A key taken before the template was applied fires no key alert, so its use shows as spend. ai-tokens-<account> fires when an account passes tokens_per_hour. Its description points to the saved query ai-calls-by-caller:

AzureDiagnostics
| where TimeGenerated > ago(7d)
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES" and Category == "RequestResponse"
| extend P = parse_json(properties_s)
| extend ObjectId = tostring(P.objectId), Deployment = tostring(P.modelDeploymentName), Model = tostring(P.modelName)
| summarize Calls = count(), First = min(TimeGenerated), Last = max(TimeGenerated), Results = make_set(ResultSignature, 10), IPs = make_set(CallerIPAddress, 10) by Account = _ResourceId, ObjectId, Deployment, Model
| order by Calls desc

The saved search keeps it on one line.

Calls signed in through Entra carry the caller's object ID. Microsoft does not document the object ID on a call signed with a key, so ai-calls-without-object-id treats an empty value as a likely key, to be checked:

AzureDiagnostics
| where TimeGenerated > ago(7d)
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES" and Category == "RequestResponse"
| extend P = parse_json(properties_s)
| extend ObjectId = tostring(P.objectId), Deployment = tostring(P.modelDeploymentName), Model = tostring(P.modelName)
| where isempty(ObjectId)
| summarize Calls = count(), First = min(TimeGenerated), Last = max(TimeGenerated), Operations = make_set(OperationName, 10) by Account = _ResourceId, CallerIPAddress, Deployment
| order by Calls desc
  • The IP is partial. The last octet is masked, so it narrows a network and does not name a host.
  • The deployment field is unverified. Microsoft does not publish the fields inside properties_s. If the Deployment column comes back empty, read one row and correct the key.
  • Expect a delay. A call can take up to two hours to become queryable.
  • The activity log names who fetched a key. ai-keys-fetched reads it, and also fires on a regenerated key and on a connection's secrets being listed. A connection stores credentials for other services.
  • Content filters are the guardrail. ai-filters-changed fires when a filter policy or blocklist is created, changed, or deleted, and ai-deployment-changed when a model deployment is.

Contain by regenerating the account's keys, which fires ai-keys-fetched for your own change. Then move callers to Entra sign-in and turn key access off. Azure can keep accepting old keys for up to several hours after that, so regenerate first. Setting key_access_policy_effect to Deny refuses any new or changed account that leaves keys on.

On Oracle Cloud

Oracle Cloud keeps the least. There is no record of prompts or responses, and Oracle does not document whether model calls reach the Audit log. The template saves a search for them, ai-inference-calls, and alarms on none:

search "<tenancy-ocid>/_Audit_Include_Subcompartment" | where data.eventName='Chat' or data.eventName='EmbedText' or data.eventName='RerankText' or data.eventName='ApplyGuardrails' | sort by datetime desc

Make one test call today and check that it appears. If it does not, a case here has no call record at all, and the changes around the calls are the evidence. The four alarms cover those: ai-api-key-changed, ai-endpoint-changed, ai-hosting-created, and ai-logging-weakened. The search behind the key alarm:

search "<tenancy-ocid>/_Audit_Include_Subcompartment" | where data.eventName='CreateApiKey' or data.eventName='RenewApiKey' or data.eventName='SetApiKeyState' or data.eventName='UpdateApiKey' or data.eventName='DeleteApiKey' or data.eventName='ChangeApiKeyCompartment' | sort by datetime desc
  • A Generative AI API key is a bearer secret. Oracle advises these keys for testing and early development, and IAM sign-in for production. What the audit event shows for a call made with one is not documented.
  • Identity's signing keys share two event names. CreateApiKey and DeleteApiKey also fire for API signing keys, so read the event before reacting.
  • Only the home region is watched. Treat models, endpoints, and keys in other regions as unwatched, and search each region's audit log by hand.
  • Alarms run about 15 minutes late. Audit usually records a call within that time.

Contain by switching the key off or deleting it in the console, and making a new one for the application. The provider has no resource for these keys, so no template can rotate them. Oracle's guidance is to rotate at once on suspicious activity.

Containment, by what was stolen

StolenFirst moveStill open afterwards
An AWS access keyDeactivate it, then deny the two invoke actions and batch jobs on its userAny user, key, or long-term Bedrock API key it created
An AWS role sessionRevoke the role's active sessionsThe place the session was taken from, until it is fixed
A Bedrock API keyDeny bedrock:CallWithBearerToken on the principal behind itShort-term keys are made on the client. Their creation cannot be prevented, only their use
A Google Cloud service account keyDisable the key, and remove who may impersonate the accountDisabling the whole account breaks every workload that uses it
An Azure account keyRegenerate the keys, then turn key access offOld keys can be accepted for up to several hours after key access is turned off
An Oracle Cloud Generative AI API keySwitch it off or delete it, and issue a new one by handKeys in regions other than the home region, which the template does not watch

On every cloud, then look for what the credential created. Leading a cloud incident lists the audit events for new credentials, new identities, and changed trust.

What to preserve

  • Each query with its context. Save the query text, the time range, and the full results together, hashed, in the locked bucket from the evidence template.
  • The Bedrock record before it expires. The log group keeps records for retention_days, and the copy in S3 under invocations/ for copy_retention_days. Raise both if the case could outlast them.
  • The record's key. Do not destroy the stack during a case. Destroy deletes the log group and schedules the record key for deletion, and without the key nothing in the kept bucket can be read.
  • The audit trail for the principal. Export its events from every Region, project, or subscription. Azure keeps the activity log 90 days, and Oracle Cloud keeps Audit 365 days.
  • Model Armor's entries. They sit in the workload project's _Default bucket, which keeps 30 days by default. Copy them out.
  • The source of the leak. The repository, build log, or machine the credential was taken from is evidence too.

These records are sensitive in two ways. They contain whatever your own people pasted into a prompt. They can also contain what the attacker generated, and Microsoft's filings describe that as prohibited content. Agree with Legal who reads the attacker's prompts before analysts open them. The insider cases lesson covers custody.

Sizing the bill

CloudWhat you can countWhat is missing
AWSInput and output tokens by principal and by model, from tokens-by-caller and calls-by-model-and-caller. Price each model's tokens at its rateRegions without invocation logging. There, streaming calls leave a completion event in CloudTrail with token counts, and other calls leave only a count
Google CloudCalls by principal and model, from the seven-day count. Widen its window for a longer caseToken totals, which the saved count does not have. Price the calls from the project's billing data
AzureTokens per account from the TotalTokens metric, and calls by caller from ai-calls-by-callerThe metric has no client IP or application, so separate the attacker's share by time and by call counts
Oracle CloudThe invoice. On-demand use is billed for each inference callAny call record, unless your test call showed up in Audit

Count CloudTrail model calls on eventType AwsApiCall. A streaming call also writes a completion event, and counting both doubles it.

Check for capacity bought as well as calls made. On Oracle Cloud a hosting cluster for a pretrained or fine-tuned model commits to 744 unit-hours, and ai-hosting-created watches for one. On AWS, look for batch jobs with CreateModelInvocationJob.

Published figures give a sense of scale. Sysdig's first report put the worst case at $46,080 a day across four Regions. Google Threat Intelligence reports some victims reaching $1 million in total.

Sizing the data exposure

Bill theft and data exposure are separate findings. Answer each of these before the report says no data was involved:

  • What the attacker sent and received. On Bedrock, one-caller shows it. On the other three clouds the call record has no text, so say that plainly.
  • What the credential could reach through the model service. Sysdig saw an actor list agents and knowledge bases. Calls that query a knowledge base or invoke an agent are CloudTrail data events, off by default. Without them, nothing shows what was retrieved.
  • Whether the credential could read the record itself. The record contains every user's prompts. If the stolen principal could read the log group, the bucket, or the workspace, treat those prompts as exposed.
  • Other secrets returned. On Azure, listing a connection's secrets returns credentials for other services. Rotate each one.
  • Prompt text stored elsewhere. On Oracle Cloud, a Generative AI project can be set to store responses and conversations.

Afterwards

FindingThe change
Nobody could say who was allowed to call modelsName them: model_caller_role_arns on AWS, model_callers on Google Cloud
The spend was noticed on the invoiceSet tokens_per_hour on Azure from a week of normal use
A long-lived key leaked from a pipelineCloud access without stored keys
Account keys were still acceptedkey_access_policy_effect set to Deny, after a test subscription
Any model could be calledallowed_models on Google Cloud, and a deny on the invoke actions for everyone but the application roles on AWS
The plan role set off logging-checked every dayAdd it to expected_principal_arns, and keep that list short

The AI model logging refresher puts the template on one printable page.

Registry 0.23.0. Built 2026-11-17.

Made by Habibullah Tora. Source on GitHub. Code under the MIT licence, writing under CC BY 4.0.