iac.htora.dev · security templates

Home/Templates/Know who is using your AI models

Know who is using your AI models

Model call records on Bedrock, Vertex AI, Azure OpenAI, and OCI Generative AI, with alerts when logging or guardrails are turned off, or stolen keys go looking for models.

Terraform

Why bother

Stolen cloud credentials are used to run AI models on the victim's bill, and the access is resold. Researchers have watched attackers check whether prompts are logged before using a key, and delete the logging when it was on. A prompt can also carry data out: a customer file pasted into a chat is a copy that left. These templates keep a record of model calls and alert on the moves around them.

How you know it worked

In a test account or project, run the drill in the verify output of the template you applied. Each drill changes a setting and puts it back, and the matching alert reaches the security inbox within minutes.

How it works

1Someone calls a modela person, a pipeline, or a stolen key2The call is recordedwith the caller and the model, and on AWS the prompt itself3Changes around it are watchedlogging, guardrails, keys, and new deployments4An alert names the moveand a saved query shows who did what else
Step by step, from the thing that happens to the result you want.

Set it up

Pick your platform, then open the level you want. Each level is complete on its own. Read the comments in the files as you go. Anything with a real consequence is explained on the line where it happens.

Terraform. Bedrock model invocation logging to CloudWatch Logs and S3, CloudWatch metric filters on the audit trail, and a Bedrock guardrail

What you need first

  • A multi-region CloudTrail trail delivering management events, read and write, to CloudWatch Logs, such as the Standard audit log template builds. Apply in that log group's Region.
  • An email address, or for Standard an SNS topic, that reaches the security team.
  • For Standard, Terraform 1.9 or later, AWS provider 6.70.0 or later, the ARNs of the roles that run Terraform plans, and the ARNs of the roles your applications call models with.

What it creates

  • Model invocation logging for this Region: every call's prompt and response in a CloudWatch log group, with images, documents, and bodies over 100 KB in an S3 bucket
  • A delivery role limited to one log stream, and a bucket policy, both scoped to Bedrock acting for this account and Region
  • Five metric filters and alarms on the trail's log group: logging changed, logging checked, model access turned on or a Marketplace offer accepted, guardrails changed, and a Bedrock API key created
  • Alarms on Bedrock's delivery failure metrics, one per destination
  • Named pipeline and plan roles left out of the five alarms (Standard)
  • A full copy of every record in the bucket with its own expiry (Standard)
  • A KMS key for the log group and bucket, and a second key for the alert topic, so an alarm about the record key can still be delivered (Standard)
  • An alarm when the record key is disabled, scheduled for deletion, or given a new policy (Standard)
  • An alarm on model calls and batch jobs in any Region by a principal outside the roles you name (Standard)
  • A guardrail with a prompt attack filter and five content filters, a published version, and optional enforcement on every call in the account and Region (Standard)
  • Saved Logs Insights queries for tokens by caller, calls by model and caller, one caller's prompts and responses, refused calls by caller, and calls made with an API key (Standard)

See the 10 alerts this creates

Saved queries this installs

Standard and Hardened install these five, ready for the first hour of a case. Quick start installs none.

Saved queryWhat it answersWhere to open it
ai-model-logging/tokens-by-callerCallers ranked by input and output tokens, from the invocation records. This is the query AWS publishes.CloudWatch console, Logs Insights, Queries, folder ai-model-logging
ai-model-logging/calls-by-model-and-callerCalls and tokens by model and principal.CloudWatch console, Logs Insights, Queries, folder ai-model-logging
ai-model-logging/one-callerWhat one caller asked and what came back. Replace the name in the filter first.CloudWatch console, Logs Insights, Queries, folder ai-model-logging
ai-model-logging/refused-calls-by-callerRefused Bedrock calls by principal and error code, as when someone tests stolen keys. Reads the trail, so it covers every Region.CloudWatch console, Logs Insights, Queries, folder ai-model-logging
ai-model-logging/bearer-token-callsBedrock calls made with a Bedrock API key. Reads the trail, so it covers every Region.CloudWatch console, Logs Insights, Queries, folder ai-model-logging

The code

Quick startReadyone file, 338 lines
One file. Invocation logging for this Region to CloudWatch Logs and S3, five alarms on the trail, and two delivery failure alarms.
Change these first
  • trail_log_group = "your-cloudtrail-log-group" main.tf, line 27
  • alert_email = "security-alerts@example.com" main.tf, line 28
Run it
  1. aws sts get-caller-identity
    Shows the account and role you are signed in as. Check it before every apply.
  2. terraform init
    Downloads the provider. Run it once per folder.
  3. terraform plan
    Shows what will be created. Changes nothing. Read it.
  4. terraform apply
    Shows the plan again and waits for you to type yes.
  5. terraform output verify
    Prints the commands that prove it worked.
  6. terraform destroy
    When you have finished. If it refuses, How to undo it below explains why.
Download main.tf
Checks

Formatting passedValid not runLint passed

Security scan: 35 checks passed, and six rules flagged, each with a written reason:

Last run 2026-11-06 with OpenTofu v1.12.6.

outcomes/ai-model-logging/aws/t0
main.tf
Open main.tf as plain text
StandardReadyseven files, 910 lines
Adds named exceptions, a full copy in S3, separate keys for the record and the alert topic, and alarms on the record key and on unexpected model callers. It also adds a guardrail with optional account-wide enforcement, and saved queries. Needs Terraform 1.9 and provider 6.70.0 or later.
Change these first
  • trail_log_group. CloudWatch log group a multi-region CloudTrail trail delivers management events to, read and write, such as the Standard audit log template builds. logging-checked watches a read event.
Run it
  1. aws sts get-caller-identity
    Shows the account and role you are signed in as. Check it before every apply.
  2. # terraform.tfvars
    trail_log_group = ""
    Create this file beside the others and fill in your values. Terraform reads it on its own.
  3. terraform init
    Downloads the provider. Run it once per folder.
  4. terraform plan
    Shows what will be created. Changes nothing. Read it.
  5. terraform apply
    Shows the plan again and waits for you to type yes.
  6. terraform output verify
    Prints the commands that prove it worked.
Download all seven files as a zip
Checks

Formatting passedValid not runLint passed

Security scan: 85 checks passed, and six rules flagged, each with a written reason:

Last run 2026-11-06 with OpenTofu v1.12.6.

outcomes/ai-model-logging/aws/t1
guardrail.tf
Open guardrail.tf as plain text
logging.tf
Open logging.tf as plain text
main.tf
Open main.tf as plain text
outputs.tf
Open outputs.tf as plain text
queries.tf
Open queries.tf as plain text
variables.tf
Open variables.tf as plain text
versions.tf
Open versions.tf as plain text
HardenedReadysix files, 343 lines
Standard as a module. Plan rules refuse the record, its delivery, its key, the enforced guardrail, or any detection but unexpected-caller deleted, and the topic removed while alarms use it. They refuse any delivery dropped, short retention, force_destroy, public access, and encryption other than KMS. They also refuse a key disabled or cut off from a service, the topic on the AWS managed key, and prompt attack filtering off. Alarms changed so one match no longer fires them, and filters changed beyond role exclusions, are refused too. A daily plan reports drift.
Change these first
  • trail_log_group. CloudWatch log group a multi-region CloudTrail trail delivers management events to, read and write, such as the Standard audit log template builds. logging-checked watches a read event.
Run it
  1. aws sts get-caller-identity
    Shows the account and role you are signed in as. Check it before every apply.
  2. # terraform.tfvars
    trail_log_group = ""
    Create this file beside the others and fill in your values. Terraform reads it on its own.
  3. terraform init
    Downloads the provider. Run it once per folder.
  4. terraform plan
    Shows what will be created. Changes nothing. Read it.
  5. terraform apply
    Shows the plan again and waits for you to type yes.
  6. terraform output verify
    Prints the commands that prove it worked.
Download all six files as a zip
Checks

Formatting passedValid not runLint passed

Security scan: 79 checks passed, and none flagged.

Last run 2026-11-06 with OpenTofu v1.12.6.

outcomes/ai-model-logging/aws/t2
.checkov.yml
Open .checkov.yml as plain text
guard.yml
Open guard.yml as plain text
main.tf
Open main.tf as plain text
outputs.tf
Open outputs.tf as plain text
variables.tf
Open variables.tf as plain text
versions.tf
Open versions.tf as plain text

How to check it worked

Run aws bedrock get-model-invocation-logging-configuration. The logging-checked alarm goes to ALARM and emails within about ten minutes.

What catches people

Invocation logging covers the Region it is set in, and only calls through bedrock-runtime. Calls in other Regions, and calls on bedrock-mantle, are not recorded. The trail alarms read every Region's events, so Standard's unexpected-caller alarm still sees bedrock-runtime model calls made elsewhere.

Terraform reads the logging configuration on every plan, the same call logging-checked watches. In the quick start, expect that alarm after each plan. In Standard, name the roles that plan, including the daily drift check's role, in expected_principal_arns.

No published CloudTrail sample shows UpdateGuardrail, DeleteGuardrail, PutEnforcedGuardrailConfiguration, or DeleteEnforcedGuardrailConfiguration. The guardrails-weakened filter uses the API's operation names, so run Standard's drill once to prove it fires.

AWS documents no key path for large data objects. The bucket policy lets Bedrock write anywhere in the bucket for that reason, and Standard's large data expiry assumes they land under their prefix. Check the bucket after the first image call.

The provider has no flag for audio, which the API accepts. Audio logging is whatever the API applies when the setting is absent, and AWS documents no default.

Most serverless models switch on at their first call, so a stolen key can use them without any call the model-access alarm watches. Standard's unexpected-caller alarm catches that first model call once model_caller_role_arns names your applications. The Marketplace call in model-access fires for any offer accepted.

An enforced guardrail applies to every model call in the account and Region, including calls that name no guardrail. Legitimate prompts can be blocked, so enforce_guardrail is off by default. Turn it on after testing, then check CloudTrail for AccessDenied, as AWS advises.

The records carry full prompts and responses. Keep read access to the log group and bucket tight. requestMetadata is set by the caller, so attribute calls on identity.arn only.

How to undo it

terraform destroy calls DeleteModelInvocationLoggingConfiguration, the call logging-changed watches, so tell the alert list first. It removes the alarms, the log group with its records, and in Standard the queries and the guardrail. The bucket is kept until you empty it, because it contains evidence. In Standard, destroy also schedules the record key for deletion, and once its 30-day window ends, nothing in the kept bucket can be read. Cancel the deletion with aws kms cancel-key-deletion, or copy the objects out first.

What it costs

CloudWatch Logs charges to ingest and keep every record, and S3 charges to store the large data and, in Standard, the copy. Both grow with prompt and response size. AWS does not say whether vended-log rates apply to these records. Each alarm costs cents a month, and each of Standard's two KMS keys a dollar a month. An enforced guardrail is billed on the safeguards it runs for every call.

Good to know

  • What each cloud records differs. Bedrock keeps the prompt and the response. Vertex AI audit logs and Azure's request logs name the caller and the model, without the text. Oracle Cloud records model work only in its Audit log.
  • On Google Cloud, Standard can keep prompt text through Model Armor, which screens Gemini calls and logs what it screened.
  • These logs contain whatever people paste into a prompt, so treat them as sensitive data. Give them few readers, and your own key where the platform offers one.
  • A model API key leaves the weakest trail, because no person stands behind the call. The AWS, Azure, and Oracle Cloud templates alert when model keys are made or fetched. Azure's Standard level can refuse keys outright, and Google Cloud's saves a query for calls made with service account keys.
  • These templates watch the cloud's own model service. Calls to a model vendor's own API, or to a model you host yourself, need logging of their own.

Registry 0.23.0. Built 2026-11-17.

Made by Habibullah Tora. Source on GitHub. Code under the MIT licence, writing under CC BY 4.0.