Skip to main content

AWS S3

Write audit logs to an Amazon S3 bucket you own. Use this when your SIEM ingests from S3 rather than from an HTTP endpoint — Panther, for example, onboards an S3 bucket as a data transport log source.

Using Panther?

Panther supports two data transports. If you can create an HTTP log source, the Panther provider is simpler: no bucket, no IAM user, and Panther receives events directly. Choose S3 when your Panther setup is built around bucket ingestion, or when you also want a durable copy of the audit trail in storage you control.

Prerequisites​

  1. An S3 bucket in your AWS account to receive the logs.
  2. An IAM user with programmatic access (access key ID and secret access key) that can write to that bucket.
  3. Network egress allowed from your deployment to the S3 endpoint for the bucket's region on port 443.

Configuration fields​

FieldRequiredDescription
BucketYesThe destination bucket name (e.g., my-siem-logs). Do not include s3://.
RegionYesThe bucket's AWS region (e.g., us-east-1).
Access Key IDYesAccess key ID for the IAM user that writes the objects.
Secret Access KeyYesThe matching secret access key.
Key PrefixNoPrefix for every object written. Defaults to willow-audit-logs.
KMS Key IDNoSSE-KMS key ARN or ID. Leave empty to use the bucket's default encryption.
Anonymous ModeNoToggle to send only statistical data (see Anonymous Mode).

How it works​

Every audit log is written as its own S3 object, as soon as it is recorded. Each object contains a single newline-delimited JSON (NDJSON) record with content type application/x-ndjson, which is the format S3-based SIEM sources expect.

Objects are keyed by organization and hour:

<prefix>/org_id=<organization-uuid>/YYYY/MM/DD/HH/<epoch-ms>-<uuid>.json

For example:

willow-audit-logs/org_id=6f1c.../2026/08/04/17/1785772097813-9f0e....json

The hour partitioning lets you scope SIEM ingestion and lifecycle rules by time, and the random suffix keeps keys unique when several events land in the same millisecond.

The object body is the audit log record. In non-anonymous mode it includes a user: { id, name, email } object; in anonymous mode the data payload is stripped down to error metadata only.

Requests use a 5-second connection and request timeout, and are retried once on failure.

IAM policy​

Grant the least privilege needed to write objects under the prefix. Replace BUCKET and PREFIX with your values.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "WillowAuditLogWrite",
      "Effect": "Allow",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::BUCKET/PREFIX/*"
    }
  ]
}

Add s3:PutObjectAcl only if the bucket belongs to a different AWS account and uses ACLs to grant the bucket owner control.

If the bucket is encrypted with SSE-KMS, or you set a KMS Key ID, the IAM user also needs permission on the key:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "WillowAuditLogKms",
      "Effect": "Allow",
      "Action": ["kms:GenerateDataKey", "kms:Encrypt"],
      "Resource": "arn:aws:kms:REGION:ACCOUNT_ID:key/KEY_ID"
    }
  ]
}

Onboard the bucket into Panther​

  1. In Panther, go to Configure > Data Sources > Onboard a Log Source, and choose AWS S3.
  2. Point the source at your bucket and set the prefix to the same Key Prefix you configured in Willow.
  3. Complete the IAM role setup Panther generates so it can read from the bucket, and configure the S3 event notifications Panther asks for. Without notifications, Panther will not be told when new objects arrive.
  4. Define a custom log schema for the Willow audit log fields (id, org_id, action, created_at, user, data, and the tool-call metadata). Set the timestamp field to created_at.
  5. Send a test event from Willow using the Test button on the log provider, then confirm the object lands in the bucket and is classified by Panther.

The same approach works for any other SIEM that ingests NDJSON from S3.

Tips​

  • Use a dedicated bucket or prefix for audit logs so retention and access policies stay isolated.
  • One object is written per audit event. Add an S3 lifecycle rule to expire or transition old objects, and expect object counts to scale with tool-call volume.
  • Enable bucket versioning and access logging if your compliance program requires tamper evidence.
  • Rotate the IAM user's access key on your normal schedule; update the provider in Willow with the new key.

Troubleshooting​

  • AccessDenied: the IAM policy does not cover the bucket and prefix in use. Confirm the resource ARN ends in /PREFIX/* and matches the Key Prefix field exactly.
  • NoSuchBucket: the bucket name is wrong, or the Region does not match the bucket's actual region.
  • InvalidAccessKeyId or SignatureDoesNotMatch: the access key ID or secret access key is incorrect, or the key has been deactivated.
  • KMS.AccessDeniedException: the IAM user is missing kms:GenerateDataKey on the key protecting the bucket.
  • Objects arrive but the SIEM shows nothing: the S3 event notifications are missing or the log schema does not match. Verify in the SIEM's own ingestion logs.

Delivery outcomes, including the exact AWS error, are recorded on the Log Delivery page. See Log Settings for shared guidance.

  • Panther: the HTTP log source alternative for Panther users
  • Log Settings: retention, content options, and Anonymous Mode
  • Log Delivery: per-delivery success and failure history
  • Logs: view and search the logs themselves
On this page