Home/Templates/Know who is using your AI models
Know who is using your AI models
Model call records on Bedrock, Vertex AI, Azure OpenAI, and OCI Generative AI, with alerts when logging or guardrails are turned off, or stolen keys go looking for models.
Terraform
Why bother
Stolen cloud credentials are used to run AI models on the victim's bill, and the access is resold. Researchers have watched attackers check whether prompts are logged before using a key, and delete the logging when it was on. A prompt can also carry data out: a customer file pasted into a chat is a copy that left. These templates keep a record of model calls and alert on the moves around them.
In a test account or project, run the drill in the verify output of the template you applied. Each drill changes a setting and puts it back, and the matching alert reaches the security inbox within minutes.
How it works
Set it up
Pick your platform, then open the level you want. Each level is complete on its own. Read the comments in the files as you go. Anything with a real consequence is explained on the line where it happens.
Terraform. Bedrock model invocation logging to CloudWatch Logs and S3, CloudWatch metric filters on the audit trail, and a Bedrock guardrail
What you need first
- A multi-region CloudTrail trail delivering management events, read and write, to CloudWatch Logs, such as the Standard audit log template builds. Apply in that log group's Region.
- An email address, or for Standard an SNS topic, that reaches the security team.
- For Standard, Terraform 1.9 or later, AWS provider 6.70.0 or later, the ARNs of the roles that run Terraform plans, and the ARNs of the roles your applications call models with.
What it creates
- Model invocation logging for this Region: every call's prompt and response in a CloudWatch log group, with images, documents, and bodies over 100 KB in an S3 bucket
- A delivery role limited to one log stream, and a bucket policy, both scoped to Bedrock acting for this account and Region
- Five metric filters and alarms on the trail's log group: logging changed, logging checked, model access turned on or a Marketplace offer accepted, guardrails changed, and a Bedrock API key created
- Alarms on Bedrock's delivery failure metrics, one per destination
- Named pipeline and plan roles left out of the five alarms (Standard)
- A full copy of every record in the bucket with its own expiry (Standard)
- A KMS key for the log group and bucket, and a second key for the alert topic, so an alarm about the record key can still be delivered (Standard)
- An alarm when the record key is disabled, scheduled for deletion, or given a new policy (Standard)
- An alarm on model calls and batch jobs in any Region by a principal outside the roles you name (Standard)
- A guardrail with a prompt attack filter and five content filters, a published version, and optional enforcement on every call in the account and Region (Standard)
- Saved Logs Insights queries for tokens by caller, calls by model and caller, one caller's prompts and responses, refused calls by caller, and calls made with an API key (Standard)
See the 10 alerts this creates
Saved queries this installs
Standard and Hardened install these five, ready for the first hour of a case. Quick start installs none.
| Saved query | What it answers | Where to open it |
|---|---|---|
ai-model-logging/tokens-by-caller | Callers ranked by input and output tokens, from the invocation records. This is the query AWS publishes. | CloudWatch console, Logs Insights, Queries, folder ai-model-logging |
ai-model-logging/calls-by-model-and-caller | Calls and tokens by model and principal. | CloudWatch console, Logs Insights, Queries, folder ai-model-logging |
ai-model-logging/one-caller | What one caller asked and what came back. Replace the name in the filter first. | CloudWatch console, Logs Insights, Queries, folder ai-model-logging |
ai-model-logging/refused-calls-by-caller | Refused Bedrock calls by principal and error code, as when someone tests stolen keys. Reads the trail, so it covers every Region. | CloudWatch console, Logs Insights, Queries, folder ai-model-logging |
ai-model-logging/bearer-token-calls | Bedrock calls made with a Bedrock API key. Reads the trail, so it covers every Region. | CloudWatch console, Logs Insights, Queries, folder ai-model-logging |
The code
Quick startReady
trail_log_group = "your-cloudtrail-log-group"main.tf, line 27alert_email = "security-alerts@example.com"main.tf, line 28
aws sts get-caller-identity
Shows the account and role you are signed in as. Check it before every apply.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform output verify
Prints the commands that prove it worked.terraform destroy
When you have finished. If it refuses, How to undo it below explains why.
Formatting passedValid not runLint passed
Security scan: 35 checks passed, and six rules flagged, each with a written reason:
- S3 event notifications
- S3 cross-region replication
- S3 encrypted with KMS
- Log group encrypted with KMS
- S3 server access logging
- SNS topic encrypted
Last run 2026-11-06 with OpenTofu v1.12.6.
Open main.tf as plain text
StandardReady
trail_log_group. CloudWatch log group a multi-region CloudTrail trail delivers management events to, read and write, such as the Standard audit log template builds. logging-checked watches a read event.
aws sts get-caller-identity
Shows the account and role you are signed in as. Check it before every apply.# terraform.tfvars trail_log_group = ""
Create this file beside the others and fill in your values. Terraform reads it on its own.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform output verify
Prints the commands that prove it worked.
Formatting passedValid not runLint passed
Security scan: 85 checks passed, and six rules flagged, each with a written reason:
- S3 event notifications
- IAM permissions management without constraints
- IAM write access without constraints
- S3 cross-region replication
- S3 server access logging
- IAM resource of *
Last run 2026-11-06 with OpenTofu v1.12.6.
Open guardrail.tf as plain text
Open logging.tf as plain text
Open main.tf as plain text
Open outputs.tf as plain text
Open queries.tf as plain text
Open variables.tf as plain text
Open versions.tf as plain text
HardenedReady
trail_log_group. CloudWatch log group a multi-region CloudTrail trail delivers management events to, read and write, such as the Standard audit log template builds. logging-checked watches a read event.
aws sts get-caller-identity
Shows the account and role you are signed in as. Check it before every apply.# terraform.tfvars trail_log_group = ""
Create this file beside the others and fill in your values. Terraform reads it on its own.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform output verify
Prints the commands that prove it worked.
Formatting passedValid not runLint passed
Security scan: 79 checks passed, and none flagged.
Last run 2026-11-06 with OpenTofu v1.12.6.
Open .checkov.yml as plain text
Open guard.yml as plain text
Open main.tf as plain text
Open outputs.tf as plain text
Open variables.tf as plain text
Open versions.tf as plain text
How to check it worked
Run aws bedrock get-model-invocation-logging-configuration. The logging-checked alarm goes to ALARM and emails within about ten minutes.
Invocation logging covers the Region it is set in, and only calls through bedrock-runtime. Calls in other Regions, and calls on bedrock-mantle, are not recorded. The trail alarms read every Region's events, so Standard's unexpected-caller alarm still sees bedrock-runtime model calls made elsewhere.
Terraform reads the logging configuration on every plan, the same call logging-checked watches. In the quick start, expect that alarm after each plan. In Standard, name the roles that plan, including the daily drift check's role, in expected_principal_arns.
No published CloudTrail sample shows UpdateGuardrail, DeleteGuardrail, PutEnforcedGuardrailConfiguration, or DeleteEnforcedGuardrailConfiguration. The guardrails-weakened filter uses the API's operation names, so run Standard's drill once to prove it fires.
AWS documents no key path for large data objects. The bucket policy lets Bedrock write anywhere in the bucket for that reason, and Standard's large data expiry assumes they land under their prefix. Check the bucket after the first image call.
The provider has no flag for audio, which the API accepts. Audio logging is whatever the API applies when the setting is absent, and AWS documents no default.
Most serverless models switch on at their first call, so a stolen key can use them without any call the model-access alarm watches. Standard's unexpected-caller alarm catches that first model call once model_caller_role_arns names your applications. The Marketplace call in model-access fires for any offer accepted.
An enforced guardrail applies to every model call in the account and Region, including calls that name no guardrail. Legitimate prompts can be blocked, so enforce_guardrail is off by default. Turn it on after testing, then check CloudTrail for AccessDenied, as AWS advises.
The records carry full prompts and responses. Keep read access to the log group and bucket tight. requestMetadata is set by the caller, so attribute calls on identity.arn only.
How to undo it
terraform destroy calls DeleteModelInvocationLoggingConfiguration, the call logging-changed watches, so tell the alert list first. It removes the alarms, the log group with its records, and in Standard the queries and the guardrail. The bucket is kept until you empty it, because it contains evidence. In Standard, destroy also schedules the record key for deletion, and once its 30-day window ends, nothing in the kept bucket can be read. Cancel the deletion with aws kms cancel-key-deletion, or copy the objects out first.
What it costs
CloudWatch Logs charges to ingest and keep every record, and S3 charges to store the large data and, in Standard, the copy. Both grow with prompt and response size. AWS does not say whether vended-log rates apply to these records. Each alarm costs cents a month, and each of Standard's two KMS keys a dollar a month. An enforced guardrail is billed on the safeguards it runs for every call.
Terraform. Azure OpenAI and AI Services resource logs and the subscription's activity log in Log Analytics, with scheduled and native alerts, a token alert per account, and the built-in key access policy
What you need first
- One or more existing Azure OpenAI or AI Services accounts in the watched subscription, given by full resource ID.
- Rights to add diagnostic settings on the accounts and the subscription, and to create a Log Analytics workspace, alert rules, and an action group. Standard also assigns roles on the workspace, a policy on the subscription, and a lock on the resource group, which needs Owner or User Access Administrator.
- An existing resource group for them, kept apart from workloads, because Standard locks everything in it against deletes.
- The Microsoft.Insights and Microsoft.OperationalInsights resource providers registered, because the azurerm provider registers none from version 5.0. Register Microsoft.PolicyInsights too, to read the key policy's compliance results (Standard).
What it creates
- A Log Analytics workspace receiving each account's Audit and RequestResponse logs and its metrics, and the subscription's activity log
- Five alerts on the activity log. Four watch every AI account in the subscription. They cover a key fetched or regenerated or a connection's secrets listed, a content filter or blocklist changed, and a model deployment made, changed, or removed. The fourth is an account changed, deleted, or purged, or its Defender for AI setting changed.
- The fifth covers a diagnostic setting, workspace, alert rule, action group, or policy assignment changed or deleted. It also covers a workspace purged, table settings changed, a lock or resource group deleted, and an alert processing rule or policy exemption written.
- Three native activity log alerts, which Azure evaluates without the workspace: a diagnostic setting deleted, a diagnostic setting written, and a log search rule disabled by Azure
- A cannot-delete lock on the security resource group, made last so it covers everything there, and a native alert on the rules' role assignments deleted (Standard)
- Named callers left out of the alerts, while the workspace keeps their entries, and an identity for each rule with the Log Analytics Reader role (Standard)
- An alert per account when its TotalTokens metric passes a set number in an hour, because stolen access shows up as spend (Standard)
- The built-in policy that reports AI accounts and AI Search services still accepting keys, or with Deny refuses new or changed ones (Standard)
- A saved query behind every scheduled alert, plus inference calls by caller object ID and deployment, and calls with no object ID, under the category AI model logging (Standard)
See the 10 alerts this creates
Saved queries this installs
Standard and Hardened install these seven, ready for the first hour of a case. Quick start installs none.
| Saved query | What it answers | Where to open it |
|---|---|---|
ai-keys-fetched | The search behind the keys-fetched alert, over the last seven days. | Log Analytics workspace, Logs, Queries, category AI model logging |
ai-filters-changed | The search behind the filters-changed alert, over the last seven days. | Log Analytics workspace, Logs, Queries, category AI model logging |
ai-deployment-changed | The search behind the deployment-changed alert, over the last seven days. | Log Analytics workspace, Logs, Queries, category AI model logging |
ai-account-changed | The search behind the account-changed alert, over the last seven days. | Log Analytics workspace, Logs, Queries, category AI model logging |
ai-logging-weakened | The search behind the logging-weakened alert, over the last seven days. | Log Analytics workspace, Logs, Queries, category AI model logging |
ai-calls-by-caller | Inference calls in the last seven days, by account, caller object ID, and deployment. | Log Analytics workspace, Logs, Queries, category AI model logging |
ai-calls-without-object-id | Inference calls in the last seven days with no caller object ID. An empty value suggests a key and needs checking. | Log Analytics workspace, Logs, Queries, category AI model logging |
The code
Quick startReady
subscription_id = "00000000-0000-0000-0000-000000000000"main.tf, line 31resource_group_name = "rg-security"main.tf, line 32alert_email = "security-alerts@example.com"main.tf, line 33main.tf, line 35
az login
Signs the command line tool in. Terraform uses the same sign-in.export ARM_SUBSCRIPTION_ID=$(az account show --query id -o tsv)
Tells the provider which subscription. It stopped guessing in version 4.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform destroy
When you have finished. If it refuses, How to undo it below explains why.
Formatting passedValid not runLint passed
Security scan: zero checks passed, and none flagged.
Last run 2026-11-06 with OpenTofu v1.12.6.
Open main.tf as plain text
StandardReady
subscription_id. Subscription whose activity log is watched, as a GUID. The workspace, alerts, and key policy go in it too.resource_group_name. An existing resource group for the workspace, alerts, and saved queries. Keep it apart from workloads: Standard locks it, so nothing in it can be deleted.ai_account_ids. Azure OpenAI or AI Services accounts in subscription_id whose calls are recorded, by full resource ID. Each gets its own diagnostic setting and token alert.alert_emails. Addresses that receive the alerts. Prefer an on-call address over a person.
az login
Signs the command line tool in. Terraform uses the same sign-in.export ARM_SUBSCRIPTION_ID=$(az account show --query id -o tsv)
Tells the provider which subscription. It stopped guessing in version 4.# terraform.tfvars subscription_id = "" resource_group_name = "" ai_account_ids = "" alert_emails = ""
Create this file beside the others and fill in your values. Terraform reads it on its own.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform output verify
Prints the commands that prove it worked.
Formatting passedValid not runLint passed
Security scan: zero checks passed, and none flagged.
Last run 2026-11-06 with OpenTofu v1.12.6.
Open main.tf as plain text
Open outputs.tf as plain text
Open policy.tf as plain text
Open queries.tf as plain text
Open tokens.tf as plain text
Open variables.tf as plain text
Open versions.tf as plain text
HardenedReady
subscription_id. Subscription whose activity log is watched, as a GUID. The workspace, alerts, and key policy go in it too.resource_group_name. An existing resource group for the workspace, alerts, and saved queries. Keep it apart from workloads: Standard locks it, so nothing in it can be deleted.ai_account_ids. Azure OpenAI or AI Services accounts in subscription_id whose calls are recorded, by full resource ID. Each gets its own diagnostic setting and token alert.alert_emails. Addresses that receive the alerts. Prefer an on-call address over a person.
az login
Signs the command line tool in. Terraform uses the same sign-in.export ARM_SUBSCRIPTION_ID=$(az account show --query id -o tsv)
Tells the provider which subscription. It stopped guessing in version 4.# terraform.tfvars subscription_id = "" resource_group_name = "" ai_account_ids = "" alert_emails = ""
Create this file beside the others and fill in your values. Terraform reads it on its own.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform output verify
Prints the commands that prove it worked.
Formatting passedValid not runLint passed
Security scan: zero checks passed, and none flagged.
Last run 2026-11-06 with OpenTofu v1.12.6.
Open .checkov.yml as plain text
Open guard.yml as plain text
Open main.tf as plain text
Open outputs.tf as plain text
Open variables.tf as plain text
Open versions.tf as plain text
How to check it worked
Wait up to 90 minutes after the first apply, while the exports start. Then run az cognitiveservices account keys list on a test account. The keys-fetched alert emails once the entry reaches the workspace.
RequestResponse has a row per inference call with the operation, status, duration, a caller IP with its last octet masked, and, for Entra sign-ins, the caller's object ID. Microsoft does not say whether it carries prompt or completion text; named research and Elastic's docs say it does not. Recording the text needs API Management's GatewayLlmLogs, which this template leaves out.
Microsoft does not document the object ID on a call signed with a key. The saved query of calls with no object ID treats an empty value as a likely key, to be checked.
Microsoft does not publish the fields inside properties_s. The saved queries read objectId and modelName as named research found them. The deployment field, modelDeploymentName, is inferred from Elastic's field names and is unverified. If the Deployment column comes back empty, read one row and correct it.
A key works for whoever has it. Storm-2139 reached Azure OpenAI with customer keys scraped from public sources and resold the access. A key taken before this template was applied fires no key alert; the token alert and the saved queries are where its use shows.
Every alert, the native ones included, emails through one action group. In the quick start, deleting or disabling that group, or deleting its resource group, silences them all. Standard's lock stops the deletes, but whoever can edit the group can still disable it or change its addresses.
Standard's lock blocks Terraform too. A change that replaces or removes a resource in the group fails while it is on, such as a saved query whose text or tags change. Remove the lock with terraform destroy -target=azurerm_management_lock.security, then apply, and the lock returns. Its removal fires the logging alert.
A cannot-delete lock does not stop a workspace purge, which the logging alert watches. Removing the rules' read access needs the lock lifted first, and the rule-access alert reports it without the workspace.
The rule-access alert matches the rules' role assignments by ID. Microsoft documents a role assignment's activity log entry under its full ID, scope included. That an assignment on a workspace is recorded the same way is inferred; the Standard verify output tests it.
The diagnostic-setting-written alert fires on every diagnostic setting created or changed in the subscription, policy remediation included, because a rewrite can drop a category or send logs elsewhere without a delete.
The logging alert fires on any policy exemption written in the subscription, because an exemption takes a scope out of the key policy's evaluation, Deny included. Expect it when teams exempt their own resources from other policies.
The key policy covers every Cognitive Services account and Azure AI Search service in the subscription. Deny refuses a new or changed one that leaves keys on, and reports those that already do. The policy's own description warns that Azure OpenAI Studio needs keys.
After keys are turned off on an account, Azure can keep accepting old keys for up to several hours, until its cache refreshes.
Azure checks the token alert's metric name when it creates the alert, so a mistake fails at apply. Set tokens_per_hour from a week of normal use; one million is a starting point.
Named research in 2023 found the account's Audit log recorded only key listings, without naming who listed them. The activity log copy of the same listing names the caller, and the keys-fetched alert reads that.
An account must sit in the watched subscription, because its key, filter, deployment, and account changes are written to that subscription's activity log. The plan refuses an account anywhere else.
Whether calls from the Foundry portal playground, Foundry project endpoints, batch jobs, the Agent Service, or failed sign-ins appear in RequestResponse is not documented. Test each path you use.
An export can take up to 90 minutes to start after it is made, and a call up to two hours to become queryable, so run the drills after that. Each resource allows five diagnostic settings, and this template uses one on each account and one on the subscription.
The AzureActivity table does not exist until the first event arrives, so the rules skip Azure's query check at creation. Azure disables a rule whose query fails for a week, and the rule-disabled alert reports that without the workspace.
How to undo it
In Standard, remove the lock first with terraform destroy -target=azurerm_management_lock.security, or module.standard.azurerm_management_lock.security in Hardened. Then terraform destroy removes the alerts, the saved queries, the diagnostic settings, the key policy assignment, and the workspace, and the call records with it. The accounts are untouched, and Azure keeps the activity log for 90 days.
What it costs
Ingesting the activity log is free, with 90 days kept free, and the native alerts cost nothing. RequestResponse, Audit, and the exported AllMetrics are billed per GB at the Analytics rate. Microsoft advises skipping the metrics export unless log queries need it; the token alert reads metrics without it. RequestResponse and Audit cannot move to the Basic plan, and a busy account writes a row per call. Longer retention and each scheduled or metric alert rule are billed.
Terraform. Data Access audit logs for Vertex AI, organisation log sinks into the security project, log-based alerts, and Model Armor floor settings
What you need first
- An organisation, and rights to set its audit configuration, create organisation sinks, and grant roles in a security project.
- The Cloud Logging and Cloud Monitoring APIs on in the security project.
- For the model list (Standard), the Organization Policy Administrator role, which can only be granted at organisation level.
- For Model Armor (Standard), Vertex AI already on in each named project, and rights there to turn on an API, grant roles, and change Model Armor floor settings.
What it creates
- Data Access audit logs for Vertex AI in every project in the organisation, for admin reads, data reads, and data writes
- An organisation sink copying model calls into one log bucket kept 400 days, matched by the 15 inference methods in Google's API definitions, each entry naming the caller and the model
- A second sink and bucket carrying only logging changes, so cutting either feed reaches the alert through the other
- Four log-based alerts. Logging weakened: a Data Access setting, log sink, bucket, exclusion, logging setting, or sink write grant changed. Guardrails weakened: a Model Armor floor setting or template, or the allowed-models policy, changed or deleted. Also the Vertex AI API turned on, and models or endpoints listed.
- Named service accounts left out of the three configuration change alerts, while the buckets keep their entries, and retention you set (Standard)
- An alert naming any principal outside a list you give who calls a model (Standard)
- An organisation policy on vertexai.allowedModels naming the models and features that may be used (Standard)
- Model Armor floor settings in named projects that screen Gemini generateContent calls for prompt injection, jailbreaks, and four content categories, and log what they find (Standard)
- Shared saved queries behind every alert, plus one principal's model calls, calls made with a service account key, and a seven-day count of calls by principal and model (Standard)
See the five alerts this creates
Saved queries this installs
Standard and Hardened install these eight, ready for the first hour of a case. Quick start installs none.
| Saved query | What it answers | Where to open it |
|---|---|---|
AI models: logging-weakened | The events the logging-weakened alert matches. | Logs Explorer, Saved queries, Shared, in the security project |
AI models: guardrails-weakened | The events the guardrails-weakened alert matches. | Logs Explorer, Saved queries, Shared, in the security project |
AI models: ai-api-enabled | The events the ai-api-enabled alert matches. | Logs Explorer, Saved queries, Shared, in the security project |
AI models: model-recon | The events the model-recon alert matches. | Logs Explorer, Saved queries, Shared, in the security project |
AI models: one principal's model calls | Model calls by one principal. Replace the email first. | Logs Explorer, Saved queries, Shared, in the security project |
AI models: calls made with a service account key | Model calls signed with a service account key. | Logs Explorer, Saved queries, Shared, in the security project |
AI models: unexpected-caller | Model calls by anyone outside the callers you listed. Only when model_callers is set. | Logs Explorer, Saved queries, Shared, in the security project |
AI models: calls by principal and model, last seven days | Who called which model, and how often, over the last seven days. | Log Analytics, Saved queries, in the security project |
The code
Quick startReady
org_id = "123456789012"main.tf, line 25security_project = "your-security-project"main.tf, line 26alert_email = "security-alerts@example.com"main.tf, line 27
gcloud auth application-default login
The sign-in Terraform reads. A plain gcloud auth login is not enough.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform destroy
When you have finished. If it refuses, How to undo it below explains why.
Formatting passedValid not runLint passed
Security scan: 12 checks passed, and none flagged.
Last run 2026-11-06 with OpenTofu v1.12.6.
Open main.tf as plain text
StandardReady
org_id. Numeric organisation ID.security_project. Project that contains the log bucket, alerts, and saved queries. Keep it separate from workloads.alert_emails. Addresses that receive the alerts. Prefer an on-call address over a person.
gcloud auth application-default login
The sign-in Terraform reads. A plain gcloud auth login is not enough.# terraform.tfvars org_id = "" security_project = "" alert_emails = ""
Create this file beside the others and fill in your values. Terraform reads it on its own.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform output verify
Prints the commands that prove it worked.
Formatting passedValid not runLint passed
Security scan: 17 checks passed, and none flagged.
Last run 2026-11-06 with OpenTofu v1.12.6.
Open guardrails.tf as plain text
Open main.tf as plain text
Open outputs.tf as plain text
Open queries.tf as plain text
Open variables.tf as plain text
Open versions.tf as plain text
HardenedReady
org_id. Numeric organisation ID.security_project. Project that contains the log bucket, alerts, and saved queries. Keep it separate from workloads.alert_emails. Addresses that receive the alerts. Prefer an on-call address over a person.
gcloud auth application-default login
The sign-in Terraform reads. A plain gcloud auth login is not enough.# terraform.tfvars org_id = "" security_project = "" alert_emails = ""
Create this file beside the others and fill in your values. Terraform reads it on its own.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform output verify
Prints the commands that prove it worked.
Formatting passedValid not runLint passed
Security scan: 17 checks passed, and none flagged.
Last run 2026-11-06 with OpenTofu v1.12.6.
Open .checkov.yml as plain text
Open guard.yml as plain text
Open main.tf as plain text
Open outputs.tf as plain text
Open variables.tf as plain text
Open versions.tf as plain text
How to check it worked
In a test project with Vertex AI on, run gcloud ai models list --region=us-central1. The model-recon alert emails within minutes.
Each model call entry names the caller and the model, and carries no prompt or response text. Model Armor's platform logs are the one way here to keep Gemini prompt text with Terraform. Request-response logging for Google and partner models has no Terraform resource, so a change to it never shows in a plan.
Data Access logs are off by default. The audit setting is authoritative for Vertex AI at the organisation, so remove any other setting for that service there first, or the two will undo each other.
The record matches the 15 inference methods in Google's v1 and v1beta1 PredictionService definitions, by the end of the name. The names come from those definitions, and a published log sample confirms only Predict, GenerateContent, StreamGenerateContent, and StreamRawPredict.
Google's table of which Vertex AI method is which audit type could not be read. A published test shows StreamRawPredict as DATA_READ. All three types are on, so each call is recorded whichever type it is.
Cutting the record's feed is itself a logging change. A second sink carries those changes into a bucket of its own. Deleting or disabling either sink, deleting either bucket, or taking either write grant away reaches the alert through the other. A single policy write that removes both grants at once may reach neither.
The audit setting goes on after the alerts exist, so the first apply should fire logging-weakened within minutes. If it does not, the sinks' write grants were still taking effect; run the recon drill. In Standard it stays quiet if the identity applying it is an expected principal. Adding a Model Armor project should fire guardrails-weakened the same way.
Guardrails-weakened also watches the allowed-models policy. No published sample shows how Organization Policy writes are logged, so that part assumes the service orgpolicy.googleapis.com with the constraint in the resource name. Change the policy in a test organisation once before relying on it.
The ai-api-enabled alert looks for aiplatform.googleapis.com anywhere in the entry. Google's query language page describes that search and does not say whether sinks and metrics apply it the same way, so run the API drill once.
The model-recon alert fires on every model or endpoint listing. It is noisy in projects that use Vertex AI, and is there for the projects that never should.
The unexpected-caller alert names a principal the first time they call a model. A stolen key for an account on the list calls as that account and does not alert. The saved query for key calls shows which calls used a service account key.
Model Armor screens generateContent only, and does not screen uploaded documents or files. It fails open: when Model Armor is unavailable, the call goes through unscreened. Its platform logs carry the full prompt and response into that project's _Default bucket, so restrict who can read them.
Terraform cannot delete a Model Armor floor setting. Destroy leaves it in force, so turn its enforcement off by hand to stop screening.
Name each model version in allowed_models. In one report, a version-less ID did not cover that model's numbered versions, and calls to them failed.
The entries sit in the security project's own log buckets, which Logs Explorer does not search by default. Choose Refine scope, then Scope by storage, and pick ai-model-logging, plus ai-model-logging-watch for logging changes, before running a saved query.
How to undo it
terraform destroy removes the audit setting, sinks, metrics, alerts, queries, and model policy, and the log buckets are deleted after their pending-deletion period. Floor settings stay in force until you turn their enforcement off.
What it costs
Model calls are Data Access logs, billed by volume. Each one is stored in its project's _Default bucket and again in the security bucket, and Cloud Logging charges ingestion at both. The rate is $0.50 per GiB beyond 50 GiB free per project each month, with 30 days kept. Admin reads, such as model listings, add to the volume. Log Analytics adds no charge. Model Armor usage can be charged on the volume it screens, and its platform logs can be large.
Terraform. The Audit service counted into metrics by caller, with alarms on Generative AI endpoints, API keys, new hosting, and logging changes
What you need first
- Rights to create connectors, alarms, a notification topic, and policies in the tenancy root, and for Standard saved searches there.
- The tenancy's home region. IAM resources can only be made there, so the template is applied there.
What it creates
- Four connectors reading the audit log of every compartment, counting matching events into four metrics with the caller as a dimension
- Four alarms. The first is a model endpoint, Agents endpoint, or routing profile created, changed, moved, or deleted. Endpoints carry the guardrails, and a routing profile decides which models and regions on-demand calls may use.
- The second alarm is a Generative AI API key created, renewed, switched on or off, changed, moved, or deleted.
- The third alarm is a dedicated AI cluster, custom model, or imported model created. The fourth is a log, connector, alarm, alarm suppression, notification topic, or subscription changed, moved, or deleted.
- Named principals left out of each alarm, while the metrics keep them, and one notification per caller (Standard)
- A saved search behind every alarm, and one for model calls in the audit log, with no alarm on it (Standard)
See the four alerts this creates
Saved queries this installs
Standard and Hardened install these five, ready for the first hour of a case. Quick start installs none.
| Saved query | What it answers | Where to open it |
|---|---|---|
ai-endpoint-changed | The search behind the endpoint-changed alarm. | Logging, Saved searches, in the tenancy |
ai-api-key-changed | The search behind the api-key-changed alarm. | Logging, Saved searches, in the tenancy |
ai-hosting-created | The search behind the hosting-created alarm. | Logging, Saved searches, in the tenancy |
ai-logging-weakened | The search behind the logging-weakened alarm. | Logging, Saved searches, in the tenancy |
ai-inference-calls | Model calls, if Audit records them. Oracle does not document whether it does, so check one test call appears first. | Logging, Saved searches, in the tenancy |
The code
Quick startReady
tenancy_ocid = "ocid1.tenancy.oc1..aaaaaaaa"main.tf, line 38region = "us-ashburn-1"main.tf, line 39alert_email = "security-alerts@example.com"main.tf, line 40
oci os ns get
Prints your Object Storage namespace, which proves ~/.oci/config works.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform destroy
When you have finished. If it refuses, How to undo it below explains why.
Formatting passedValid passedLint passed
Security scan: one check passed, and none flagged.
Last run 2026-11-12 with OpenTofu v1.12.6 against oracle/oci 9.8.0.
Open main.tf as plain text
StandardReady
tenancy_ocid. The tenancy. The connectors read its audit log, every compartment included, and the alarms live in its root.region. The tenancy's home region, such as us-ashburn-1. IAM resources can only be made there, and the connectors read that region's audit log.alert_emails. Addresses that receive the alarms. Prefer an on-call address over a person. Each must confirm its subscription before mail arrives.
oci os ns get
Prints your Object Storage namespace, which proves ~/.oci/config works.# terraform.tfvars tenancy_ocid = "" region = "" alert_emails = ""
Create this file beside the others and fill in your values. Terraform reads it on its own.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform output verify
Prints the commands that prove it worked.
Formatting passedValid passedLint passed
Security scan: one check passed, and none flagged.
Last run 2026-11-12 with OpenTofu v1.12.6 against oracle/oci 9.8.0.
Open main.tf as plain text
Open outputs.tf as plain text
Open queries.tf as plain text
Open variables.tf as plain text
Open versions.tf as plain text
HardenedReady
tenancy_ocid. The tenancy. The connectors read its audit log, every compartment included, and the alarms live in its root.region. The tenancy's home region, such as us-ashburn-1. IAM resources can only be made there, and the connectors read that region's audit log.alert_emails. Addresses that receive the alarms. Prefer an on-call address over a person. Each must confirm its subscription before mail arrives.
oci os ns get
Prints your Object Storage namespace, which proves ~/.oci/config works.# terraform.tfvars tenancy_ocid = "" region = "" alert_emails = ""
Create this file beside the others and fill in your values. Terraform reads it on its own.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform output verify
Prints the commands that prove it worked.
Formatting passedValid passedLint passed
Security scan: one check passed, and none flagged.
Last run 2026-11-12 with OpenTofu v1.12.6 against oracle/oci 9.8.0.
Open .checkov.yml as plain text
Open guard.yml as plain text
Open main.tf as plain text
Open outputs.tf as plain text
Open variables.tf as plain text
Open versions.tf as plain text
How to check it worked
Confirm the subscription email first. Then, in a test compartment, create a Generative AI API key in the console, and delete it. The ai-api-key-changed alarm emails within about 15 minutes.
Oracle Cloud keeps no record of prompts or responses from model calls, unless a Generative AI project is set to store them. Generative AI is on neither Oracle's list of services with logs nor its list of services that send events. The audit log is the only record left, and it names the operation and the caller.
Oracle does not document whether Audit records model calls such as Chat, EmbedText, RerankText, and ApplyGuardrails. Standard saves a search for them and alarms on none. The Agents runtime also names its calls Chat, so the search would list both. Make one test call and check that it appears before relying on the search.
Audit event names are the API operations' own names, and the connectors match them exactly. Oracle publishes no sample Generative AI audit event, so run the verify drill before trusting the alarms. If one stays quiet, search the audit log for the call and read data.eventName.
Other services share some of these names. AI Language endpoints use the same four endpoint operations. Data Science, AI Language, AI Vision, and Document Understanding each name an operation CreateModel. Identity's API signing keys share CreateApiKey and DeleteApiKey. Expect alarms from those services too.
Endpoint guardrails exist only on dedicated AI clusters. On-demand calls are checked only when the application itself calls ApplyGuardrails, which does not block by default, so there is no setting to watch. Generative AI Agents endpoints carry guardrails of their own, and the endpoint alarm watches them too.
The provider does not model the prompt injection or PII settings on an endpoint, so a change to them never shows in a plan. The endpoint alarm fires on any endpoint change. Whether the audit event says which setting changed is not documented.
IAM resources can only be made in the home region, so the template is applied there and its connectors read that region's audit log. Oracle's log aggregation guide repeats collection in every subscribed region. Treat models, endpoints, and keys in other regions as unwatched.
Generative AI API keys are bearer secrets that call models, and the provider has no resource for them, so no template can make or rotate them. Oracle advises them for testing and early development, and IAM sign-in for production. The research behind this template found no published LLMjacking case on Oracle Cloud.
Audit usually records a call within 15 minutes, so every alarm here is about that late.
Connector Hub allows 20 connectors per region by default and five on a trial account. This template uses four.
Applying a change to this template's own alarms or connectors fires the logging-weakened alarm, unless Standard names the principal that applies it. That email is a test of it.
Deactivating the logging connector may stop it counting its own deactivation. Every connector is pinned active in Terraform, so the daily drift plan reports one switched off.
Deleting the notification topic or the connectors' policy stops all four alarms, and none of them can report it. The daily drift plan in Hardened is what notices.
Standard leaves one named principal out with != and several with !~ and a pipe between names, the forms Oracle documents. A name cannot contain a pipe or an asterisk, which Monitoring reads as patterns.
How to undo it
terraform destroy removes the connectors, alarms, topic, policy, and saved searches. The Audit service keeps recording, for 365 days.
What it costs
Connector Hub has no charge. Monitoring bills the custom metrics, and Notifications bills the emails, at standard rates, and a few changes a day make few of either. The cost the hosting alarm watches is larger: a hosting cluster for a pretrained or fine-tuned model commits to 744 unit-hours.
Good to know
- What each cloud records differs. Bedrock keeps the prompt and the response. Vertex AI audit logs and Azure's request logs name the caller and the model, without the text. Oracle Cloud records model work only in its Audit log.
- On Google Cloud, Standard can keep prompt text through Model Armor, which screens Gemini calls and logs what it screened.
- These logs contain whatever people paste into a prompt, so treat them as sensitive data. Give them few readers, and your own key where the platform offers one.
- A model API key leaves the weakest trail, because no person stands behind the call. The AWS, Azure, and Oracle Cloud templates alert when model keys are made or fetched. Azure's Standard level can refuse keys outright, and Google Cloud's saves a query for calls made with service account keys.
- These templates watch the cloud's own model service. Calls to a model vendor's own API, or to a model you host yourself, need logging of their own.