Skip to content
pipeo
Menu

Amazon Web Services

Amazon S3 file sink

Write events as gzipped NDJSON files into your own S3 bucket, for a warehouse or lake to load.

Each batch becomes one gzipped NDJSON object under your key prefix, partitioned by the hour of its earliest event. A retried batch writes the same key, so it overwrites instead of duplicating, and every line carries its message id so loaders can dedupe.

The bucket and credentials are yours. Only AWS is reachable unless the operator allows specific self-hosted S3 endpoints. Files are an at-least-once feed with no ordering promise.