iac.htora.dev · security templates

Home/Templates/Get cloud logs into your SIEM

Get cloud logs into your SIEM

A delivery pipe from cloud logging to the place your detections actually run.

Terraform

Why bother

Logs sitting in a cloud console are not next to your endpoint and identity data, so nothing correlates and the detection never fires. This wires up the delivery so events land where analysts already look.

How you know it worked

Do one thing you can recognise. Search for it in the SIEM and find it.

How it works

1Cloud logs are writtenaudit, network, identity2A stream picks them upFirehose, Event Hub, Pub/Sub, or Connector Hub3The collectorSplunk, Sentinel, or your own4Searchable in one placenext to endpoint and identity data
Step by step, from the thing that happens to the result you want.

Set it up

Pick your platform, then open the level you want. Each level is complete on its own. Read the comments in the files as you go. Anything with a real consequence is explained on the line where it happens.

Terraform. Kinesis Firehose to a Splunk HTTP event collector

What you need first

  • A Splunk HTTP Event Collector token with indexer acknowledgement on, and the source type your add-on expects, such as aws:cloudtrail.
  • A CloudWatch log group to send, such as the trail's log group from the audit log template.
  • For Standard, an SNS topic for the delivery alarms.

What it creates

  • A Firehose stream to Splunk that unpacks CloudWatch's compressed batches into single events
  • A bucket for events Splunk refused, kept 30 days for resending
  • A subscription from each log group to the stream, and the roles for both hops
  • The stream under its own key, its errors logged, and alarms when delivery fails or falls 15 minutes behind (Standard)

See the two alerts this creates

The code

Quick startReadyone file, 211 lines
One file. One log group into Splunk, with refused events kept for resending.
Change these first
  • hec_token. Splunk HTTP Event Collector token. Set it with TF_VAR_hec_token so it stays out of files. It is still stored in state.
  • log_group = "your-trail-log-group" main.tf, line 20
  • hec_endpoint = "https://http-inputs-yourstack.splunkcloud.com:443" main.tf, line 21
Run it
  1. aws sts get-caller-identity
    Shows the account and role you are signed in as. Check it before every apply.
  2. # terraform.tfvars
    hec_token = ""
    Create this file beside the others and fill in your values. Terraform reads it on its own.
  3. terraform init
    Downloads the provider. Run it once per folder.
  4. terraform plan
    Shows what will be created. Changes nothing. Read it.
  5. terraform apply
    Shows the plan again and waits for you to type yes.
  6. terraform destroy
    When you have finished. If it refuses, How to undo it below explains why.
Download main.tf
Checks

Formatting passedValid passedLint passed

Security scan: 56 checks passed, and six rules flagged, each with a written reason:

Last run 2026-10-02 with OpenTofu v1.12.6 against hashicorp/aws 6.66.0.

outcomes/siem-ingest/aws/t0
main.tf
Open main.tf as plain text
StandardReadyfour files, 334 lines
Adds several log groups, the stream and refused events under their own key, the stream's error log, and alarms when delivery fails or falls behind.
Change these first
  • log_groups. CloudWatch log groups to send, such as the trail and the EKS audit log.
  • hec_endpoint. Splunk HTTP Event Collector address, with port.
  • hec_token. Splunk HTTP Event Collector token, with indexer acknowledgement on. Set it with TF_VAR_hec_token. It is still stored in state, so the state bucket must be locked down.
  • alert_topic_arn. SNS topic for alarms when delivery fails or falls behind.
Run it
  1. aws sts get-caller-identity
    Shows the account and role you are signed in as. Check it before every apply.
  2. # terraform.tfvars
    log_groups = ""
    hec_endpoint = ""
    hec_token = ""
    alert_topic_arn = ""
    Create this file beside the others and fill in your values. Terraform reads it on its own.
  3. terraform init
    Downloads the provider. Run it once per folder.
  4. terraform plan
    Shows what will be created. Changes nothing. Read it.
  5. terraform apply
    Shows the plan again and waits for you to type yes.
  6. terraform output verify
    Prints the commands that prove it worked.
Download all four files as a zip
Checks

Formatting passedValid passedLint passed

Security scan: 65 checks passed, and six rules flagged, each with a written reason:

Last run 2026-10-02 with OpenTofu v1.12.6 against hashicorp/aws 6.66.0.

outcomes/siem-ingest/aws/t1
main.tf
Open main.tf as plain text
outputs.tf
Open outputs.tf as plain text
variables.tf
Open variables.tf as plain text
versions.tf
Open versions.tf as plain text
HardenedReadysix files, 219 lines
Standard as a module. Plan rules refuse the stream, key, or backup bucket deleted, the stream without a customer key, refused events not backed up, error logging off, silent alarms, public access allowed, key rotation off. A daily plan reports drift, and the scan fails on any unexplained finding.
Change these first
  • log_groups. CloudWatch log groups to send, such as the trail and the EKS audit log.
  • hec_endpoint. Splunk HTTP Event Collector address, with port.
  • hec_token. Splunk HTTP Event Collector token, with indexer acknowledgement on. Set it with TF_VAR_hec_token. It is still stored in state, so the state bucket must be locked down.
  • alert_topic_arn. SNS topic for alarms when delivery fails or falls behind.
Run it
  1. aws sts get-caller-identity
    Shows the account and role you are signed in as. Check it before every apply.
  2. # terraform.tfvars
    log_groups = ""
    hec_endpoint = ""
    hec_token = ""
    alert_topic_arn = ""
    Create this file beside the others and fill in your values. Terraform reads it on its own.
  3. terraform init
    Downloads the provider. Run it once per folder.
  4. terraform plan
    Shows what will be created. Changes nothing. Read it.
  5. terraform apply
    Shows the plan again and waits for you to type yes.
  6. terraform output verify
    Prints the commands that prove it worked.
Download all six files as a zip
Checks

Formatting passedValid passedLint passed

Security scan: 65 checks passed, and none flagged.

Last run 2026-10-02 with OpenTofu v1.12.6 against hashicorp/aws 6.66.0.

outcomes/siem-ingest/aws/t2
.checkov.yml
Open .checkov.yml as plain text
guard.yml
Open guard.yml as plain text
main.tf
Open main.tf as plain text
outputs.tf
Open outputs.tf as plain text
variables.tf
Open variables.tf as plain text
versions.tf
Open versions.tf as plain text

How to check it worked

Do one recognisable thing, such as creating a bucket named siem-drill-<today>, and find it in Splunk. The failed events bucket stays empty when everything arrives.

What catches people

Firehose will not deliver to a token without indexer acknowledgement. Turn it on in Splunk before the first apply.

The token ends up in Terraform state even when set through an environment variable. The state bucket needs the same protection as the token.

A log group takes at most two subscription filters. If it already has two, remove one or reuse an existing destination.

The stream sends raw events, so Splunk takes the source type from the token. Set it there, or searches come back under the wrong source type.

How to undo it

terraform destroy removes the stream, subscriptions, roles, and the failed events bucket with its contents. The log groups are left as they were.

What it costs

Firehose charges per gigabyte delivered. Splunk's licence counts the same data at the other end.

Good to know

  • Watch the pipe as closely as what flows through it. A stopped pipe looks the same in the SIEM as a quiet day, so both Standard versions alert when delivery fails or falls behind.
  • Each pipe keeps what it could not deliver: refused events in a bucket on AWS, and seven days of messages plus a dead-letter subscription on Google Cloud.

Registry 0.23.0. Built 2026-11-17.

Made by Habibullah Tora. Source on GitHub. Code under the MIT licence, writing under CC BY 4.0.