Home/Templates/Get cloud logs into your SIEM
Get cloud logs into your SIEM
A delivery pipe from cloud logging to the place your detections actually run.
Terraform
Why bother
Logs sitting in a cloud console are not next to your endpoint and identity data, so nothing correlates and the detection never fires. This wires up the delivery so events land where analysts already look.
Do one thing you can recognise. Search for it in the SIEM and find it.
How it works
Set it up
Pick your platform, then open the level you want. Each level is complete on its own. Read the comments in the files as you go. Anything with a real consequence is explained on the line where it happens.
Terraform. Kinesis Firehose to a Splunk HTTP event collector
What you need first
- A Splunk HTTP Event Collector token with indexer acknowledgement on, and the source type your add-on expects, such as aws:cloudtrail.
- A CloudWatch log group to send, such as the trail's log group from the audit log template.
- For Standard, an SNS topic for the delivery alarms.
What it creates
- A Firehose stream to Splunk that unpacks CloudWatch's compressed batches into single events
- A bucket for events Splunk refused, kept 30 days for resending
- A subscription from each log group to the stream, and the roles for both hops
- The stream under its own key, its errors logged, and alarms when delivery fails or falls 15 minutes behind (Standard)
See the two alerts this creates
The code
Quick startReady
hec_token. Splunk HTTP Event Collector token. Set it with TF_VAR_hec_token so it stays out of files. It is still stored in state.log_group = "your-trail-log-group"main.tf, line 20hec_endpoint = "https://http-inputs-yourstack.splunkcloud.com:443"main.tf, line 21
aws sts get-caller-identity
Shows the account and role you are signed in as. Check it before every apply.# terraform.tfvars hec_token = ""
Create this file beside the others and fill in your values. Terraform reads it on its own.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform destroy
When you have finished. If it refuses, How to undo it below explains why.
Formatting passedValid passedLint passed
Security scan: 56 checks passed, and six rules flagged, each with a written reason:
- S3 event notifications
- S3 cross-region replication
- S3 encrypted with KMS
- S3 server access logging
- S3 versioning on the SIEM failed-events bucket
- Firehose encrypted with a customer-managed key
Last run 2026-10-02 with OpenTofu v1.12.6 against hashicorp/aws 6.66.0.
Open main.tf as plain text
StandardReady
log_groups. CloudWatch log groups to send, such as the trail and the EKS audit log.hec_endpoint. Splunk HTTP Event Collector address, with port.hec_token. Splunk HTTP Event Collector token, with indexer acknowledgement on. Set it with TF_VAR_hec_token. It is still stored in state, so the state bucket must be locked down.alert_topic_arn. SNS topic for alarms when delivery fails or falls behind.
aws sts get-caller-identity
Shows the account and role you are signed in as. Check it before every apply.# terraform.tfvars log_groups = "" hec_endpoint = "" hec_token = "" alert_topic_arn = ""
Create this file beside the others and fill in your values. Terraform reads it on its own.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform output verify
Prints the commands that prove it worked.
Formatting passedValid passedLint passed
Security scan: 65 checks passed, and six rules flagged, each with a written reason:
- S3 event notifications
- S3 cross-region replication
- Log group encrypted with KMS
- S3 server access logging
- S3 versioning on the SIEM failed-events bucket
- CloudWatch log retention of a year
Last run 2026-10-02 with OpenTofu v1.12.6 against hashicorp/aws 6.66.0.
Open main.tf as plain text
Open outputs.tf as plain text
Open variables.tf as plain text
Open versions.tf as plain text
HardenedReady
log_groups. CloudWatch log groups to send, such as the trail and the EKS audit log.hec_endpoint. Splunk HTTP Event Collector address, with port.hec_token. Splunk HTTP Event Collector token, with indexer acknowledgement on. Set it with TF_VAR_hec_token. It is still stored in state, so the state bucket must be locked down.alert_topic_arn. SNS topic for alarms when delivery fails or falls behind.
aws sts get-caller-identity
Shows the account and role you are signed in as. Check it before every apply.# terraform.tfvars log_groups = "" hec_endpoint = "" hec_token = "" alert_topic_arn = ""
Create this file beside the others and fill in your values. Terraform reads it on its own.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform output verify
Prints the commands that prove it worked.
Formatting passedValid passedLint passed
Security scan: 65 checks passed, and none flagged.
Last run 2026-10-02 with OpenTofu v1.12.6 against hashicorp/aws 6.66.0.
Open .checkov.yml as plain text
Open guard.yml as plain text
Open main.tf as plain text
Open outputs.tf as plain text
Open variables.tf as plain text
Open versions.tf as plain text
How to check it worked
Do one recognisable thing, such as creating a bucket named siem-drill-<today>, and find it in Splunk. The failed events bucket stays empty when everything arrives.
Firehose will not deliver to a token without indexer acknowledgement. Turn it on in Splunk before the first apply.
The token ends up in Terraform state even when set through an environment variable. The state bucket needs the same protection as the token.
A log group takes at most two subscription filters. If it already has two, remove one or reuse an existing destination.
The stream sends raw events, so Splunk takes the source type from the token. Set it there, or searches come back under the wrong source type.
How to undo it
terraform destroy removes the stream, subscriptions, roles, and the failed events bucket with its contents. The log groups are left as they were.
What it costs
Firehose charges per gigabyte delivered. Splunk's licence counts the same data at the other end.
Terraform. Event Hubs, read by your SIEM's Azure input or a Sentinel workspace
What you need first
- A subscription, and Contributor on it.
- A SIEM input that reads Event Hubs, such as the Splunk Add-on for Microsoft Cloud Services.
- For Standard's Entra logs, Global Administrator or Security Administrator for whoever runs apply.
What it creates
- An Event Hubs namespace and a hub that keeps events for seven days
- The subscription's Activity Log streamed to it
- A consumer group and a listen-only key for the SIEM
- Chosen resources' logs, optional Entra sign-in and audit logs, and an optional Log Analytics workspace as a second destination (Standard)
- Automatic scaling, and alerts when events are refused for lack of capacity or stop arriving (Standard)
See the two alerts this creates
The code
Quick startReady
az login
Signs the command line tool in. Terraform uses the same sign-in.export ARM_SUBSCRIPTION_ID=$(az account show --query id -o tsv)
Tells the provider which subscription. It stopped guessing in version 4.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform destroy
When you have finished. If it refuses, How to undo it below explains why.
Formatting passedValid passedLint passed
Security scan: two checks passed, and none flagged.
Last run 2026-10-02 with OpenTofu v1.12.6 against hashicorp/azurerm 5.6.0.
Open main.tf as plain text
StandardReady
subscription_id. Subscription whose Activity Log is streamed, and where the hub lives.alert_emails. Addresses told when events are refused for lack of capacity, or stop arriving.
az login
Signs the command line tool in. Terraform uses the same sign-in.export ARM_SUBSCRIPTION_ID=$(az account show --query id -o tsv)
Tells the provider which subscription. It stopped guessing in version 4.# terraform.tfvars subscription_id = "" alert_emails = ""
Create this file beside the others and fill in your values. Terraform reads it on its own.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform output verify
Prints the commands that prove it worked.
Formatting passedValid passedLint passed
Security scan: two checks passed, and none flagged.
Last run 2026-10-02 with OpenTofu v1.12.6 against hashicorp/azurerm 5.6.0.
Open main.tf as plain text
Open outputs.tf as plain text
Open variables.tf as plain text
Open versions.tf as plain text
HardenedReady
subscription_id. Subscription whose Activity Log is streamed, and where the hub lives.alert_emails. Addresses told when events are refused for lack of capacity, or stop arriving.
az login
Signs the command line tool in. Terraform uses the same sign-in.export ARM_SUBSCRIPTION_ID=$(az account show --query id -o tsv)
Tells the provider which subscription. It stopped guessing in version 4.# terraform.tfvars subscription_id = "" alert_emails = ""
Create this file beside the others and fill in your values. Terraform reads it on its own.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform output verify
Prints the commands that prove it worked.
Formatting passedValid passedLint passed
Security scan: two checks passed, and none flagged.
Last run 2026-10-02 with OpenTofu v1.12.6 against hashicorp/azurerm 5.6.0.
Open .checkov.yml as plain text
Open guard.yml as plain text
Open main.tf as plain text
Open outputs.tf as plain text
Open variables.tf as plain text
Open versions.tf as plain text
How to check it worked
Change a tag on any resource. Within a few minutes the namespace's IncomingMessages metric rises, and the SIEM shows an event with operationName Microsoft.Resources/tags/write.
A diagnostic setting streams only to a hub in the resource's own region. Resources in other regions need a hub of their own.
The SIEM's key can read but not send or manage. Keep it that way, and rotate it in the SIEM when you rotate it here.
Events older than seven days are dropped. A SIEM outage longer than that loses them.
The quick start's single throughput unit takes in about 1 MB a second. A busy subscription refuses events beyond that; Standard scales and alerts.
How to undo it
terraform destroy removes the diagnostic settings, the hub, and the namespace. Events not yet read are lost.
What it costs
Event Hubs Standard bills per throughput unit hour and per million events taken in. Log Analytics, if used, bills per GB ingested.
Terraform. Pub/Sub to your collector
What you need first
- A project where you can create log sinks, Pub/Sub topics, and service accounts.
- A collector that pulls from Pub/Sub, such as the Splunk add-on for Google Cloud or Google Security Operations.
- For the Standard organisation-wide sink, rights to create sinks at organisation level.
What it creates
- A sink that sends every audit log in the project to a topic
- A subscription that keeps messages for seven days and never expires
- A collector service account allowed to read it
- An organisation-wide sink, exclusions, a dead-letter subscription, and an alert when the collector falls an hour behind (Standard)
- Encryption under your own key, when you give one (Standard)
See the one alert this creates
The code
Quick startReady
project_id = "your-project-id"main.tf, line 20
gcloud auth application-default login
The sign-in Terraform reads. A plain gcloud auth login is not enough.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform output verify
Prints the commands that prove it worked.terraform destroy
When you have finished. If it refuses, How to undo it below explains why.
Formatting passedValid passedLint passed
Security scan: two checks passed, and one rule flagged, each with a written reason:
Last run 2026-10-02 with OpenTofu v1.12.6 against hashicorp/google 8.4.0.
Open main.tf as plain text
StandardReady
project_id. Project that contains the topic and subscription.alert_emails. Addresses told when the collector falls more than an hour behind.
gcloud auth application-default login
The sign-in Terraform reads. A plain gcloud auth login is not enough.# terraform.tfvars project_id = "" alert_emails = ""
Create this file beside the others and fill in your values. Terraform reads it on its own.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform output verify
Prints the commands that prove it worked.
Formatting passedValid passedLint passed
Security scan: seven checks passed, and none flagged.
Last run 2026-10-02 with OpenTofu v1.12.6 against hashicorp/google 8.4.0.
Open main.tf as plain text
Open outputs.tf as plain text
Open variables.tf as plain text
Open versions.tf as plain text
HardenedReady
project_id. Project that contains the topic and subscription.alert_emails. Addresses told when the collector falls more than an hour behind.
gcloud auth application-default login
The sign-in Terraform reads. A plain gcloud auth login is not enough.# terraform.tfvars project_id = "" alert_emails = ""
Create this file beside the others and fill in your values. Terraform reads it on its own.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform output verify
Prints the commands that prove it worked.
Formatting passedValid passedLint passed
Security scan: seven checks passed, and none flagged.
Last run 2026-10-02 with OpenTofu v1.12.6 against hashicorp/google 8.4.0.
Open .checkov.yml as plain text
Open guard.yml as plain text
Open main.tf as plain text
Open outputs.tf as plain text
Open variables.tf as plain text
Open versions.tf as plain text
How to check it worked
Do one recognisable thing, such as creating a bucket named siem-drill-<today>, and find it in the SIEM. Before the collector is connected, terraform output verify shows a message waiting.
A subscription nobody reads for 31 days is deleted by default, and the feed ends with no error. Both levels switch that expiry off.
The sink writes as its own identity. Without publish rights on the topic, entries are dropped and the only sign is an error on the sink.
Many collectors ask for a service account key, which the guardrail template blocks. Prefer a collector that signs in through workload identity federation.
Every exclusion is a blind spot for detection. Keep them few, and name what each one leaves out.
How to undo it
terraform destroy removes the sink, topics, subscriptions, and service account. Messages still waiting are lost, so let the collector catch up first.
What it costs
Pub/Sub charges by the volume delivered. The sink itself is free.
Terraform. Connector Hub copying audit events to a stream the SIEM reads
What you need first
- Permission to manage streams, connectors, and policies in the tenancy.
- A group and user for the SIEM, with an API key or auth token as its input asks.
What it creates
- A stream that keeps events for seven days
- A connector copying every compartment's audit events into it
- A policy letting the connector write to the stream and the SIEM group read it
- A stream pool of its own with the Kafka API on, an optional key of your own, and more logs by name (Standard)
- Alarms when the connector fails to write or its newest event is more than 15 minutes old (Standard)
See the two alerts this creates
The code
Quick startReady
tenancy_ocid = "ocid1.tenancy.oc1..change-me"main.tf, line 21siem_group = "siem-readers"main.tf, line 22
oci os ns get
Prints your Object Storage namespace, which proves ~/.oci/config works.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform destroy
When you have finished. If it refuses, How to undo it below explains why.
Formatting passedValid passedLint passed
Security scan: one check passed, and none flagged.
Last run 2026-11-09 with OpenTofu v1.12.6 against oracle/oci 9.8.0.
Open main.tf as plain text
StandardReady
tenancy_ocid. The tenancy. Audit events are read from its root, which includes every compartment.region. Region identifier, such as us-ashburn-1. Audit events are regional, so each region you use needs its own copy of this.compartment_ocid. Compartment for the stream pool, stream, connector, and alarms.siem_group. Group the SIEM's user belongs to. It may read messages from this stream only, and list the streams and pools in the compartment.alert_email. Address told when the connector fails to write or falls behind. It must confirm the subscription before mail arrives.
oci os ns get
Prints your Object Storage namespace, which proves ~/.oci/config works.# terraform.tfvars tenancy_ocid = "" region = "" compartment_ocid = "" siem_group = "" alert_email = ""
Create this file beside the others and fill in your values. Terraform reads it on its own.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform output verify
Prints the commands that prove it worked.
Formatting passedValid passedLint passed
Security scan: one check passed, and none flagged.
Last run 2026-11-09 with OpenTofu v1.12.6 against oracle/oci 9.8.0.
Open main.tf as plain text
Open outputs.tf as plain text
Open variables.tf as plain text
Open versions.tf as plain text
HardenedReady
tenancy_ocid. The tenancy. Audit events are read from its root, which includes every compartment.region. Region identifier, such as us-ashburn-1. Audit events are regional, so each region you use needs its own copy of this.compartment_ocid. Compartment for the stream pool, stream, connector, and alarms.siem_group. Group the SIEM's user belongs to. It may read messages from this stream only, and list the streams and pools in the compartment.alert_email. Address told when the connector fails to write or falls behind. It must confirm the subscription before mail arrives.
oci os ns get
Prints your Object Storage namespace, which proves ~/.oci/config works.# terraform.tfvars tenancy_ocid = "" region = "" compartment_ocid = "" siem_group = "" alert_email = ""
Create this file beside the others and fill in your values. Terraform reads it on its own.terraform init
Downloads the provider. Run it once per folder.terraform plan
Shows what will be created. Changes nothing. Read it.terraform apply
Shows the plan again and waits for you to type yes.terraform output verify
Prints the commands that prove it worked.
Formatting passedValid passedLint passed
Security scan: one check passed, and none flagged.
Last run 2026-11-09 with OpenTofu v1.12.6 against oracle/oci 9.8.0.
Open .checkov.yml as plain text
Open guard.yml as plain text
Open main.tf as plain text
Open outputs.tf as plain text
Open variables.tf as plain text
Open versions.tf as plain text
How to check it worked
Change a tag on any resource. Within a few minutes the SIEM shows the event. In Standard, confirm the subscription email first, or the alarms send nothing. Standard prints commands to read the stream directly with terraform output verify.
Audit events are regional. Each region you use needs its own connector and stream.
Events older than seven days are dropped. A SIEM outage longer than that loses them.
Each partition takes in about 1 MB a second. A busy tenancy needs more partitions.
A stream pool with your own key fails to create unless the key's policy lets the Streaming service use it.
Deleting Standard's notification topic stops both alarms, and neither can report it. The daily drift plan in Hardened is what notices.
How to undo it
terraform destroy removes the connector, the stream, and the policy. Events not yet read are lost.
What it costs
Streaming bills per GB moved in or out and per GB stored. Connector Hub itself is free.
Good to know
- Watch the pipe as closely as what flows through it. A stopped pipe looks the same in the SIEM as a quiet day, so both Standard versions alert when delivery fails or falls behind.
- Each pipe keeps what it could not deliver: refused events in a bucket on AWS, and seven days of messages plus a dead-letter subscription on Google Cloud.