File

The File sink allows worker to write observability events directly to files on disk. It supports logs, metrics, and traces, and is commonly used for local inspection, debugging, archival purposes, or integration with downstream systems that consume data from files.

Events are written sequentially to files based on the configured path template. worker can manage file rotation implicitly through time-based path templates, and can optionally apply compression and framing depending on the selected encoding.

This sink is stateless and streams events as they arrive, making it suitable for simple, reliable file-based output scenarios.

This sink supports logs, metrics, and traces as input types.

Path (required) Defines the file path where events are written. The path supports time-based templating, allowing automatic file rotation based on timestamps. Compression format extensions must be explicitly included in the path if compression is enabled. This parameter supports worker's template syntax, enabling dynamic file paths per event or per time window.

Encoding (required) Controls how events are serialized before being written to disk. The selected codec determines how log, metric, or trace data is represented in the output file. Encoding configuration is the primary mechanism for choosing between structured formats such as JSON or CSV, unstructured text output, or protocol-oriented formats such as Protobuf or OTLP. Encoding options also control timestamp formatting and field inclusion or exclusion.

Compression (optional) Specifies whether file output is compressed before being written. Supported compression formats include gzip and zstd, as well as no compression. Compression can significantly reduce disk usage at the cost of additional CPU overhead.

Buffer (optional) Controls how events are buffered before being written to disk. Memory buffering provides higher throughput but loses data on crash or forced restart. Disk buffering is more durable and can survive restarts after data is flushed, at the cost of lower performance. Buffer settings also determine how worker behaves when buffers are full, allowing either backpressure or intentional dropping of new events.

Idle Timeout (optional) Defines how long a file can remain open without receiving new events. When the idle timeout is reached, the file is flushed and closed. This helps reduce the number of open file descriptors and ensures data is periodically flushed to disk.

Framing (optional) Controls how individual events are delimited within the file. Framing options determine whether events are written as newline-delimited records, length-prefixed frames, or raw byte streams. This is particularly important when downstream consumers expect a specific framing format.

Truncation (optional) Provides mechanisms for truncating files after certain conditions are met, such as a fixed amount of time after closing or after a period of inactivity. This can be useful in environments where disk usage must be tightly controlled.

Timezone (optional) Specifies the timezone used when rendering time-based components in the file path template. If not set, worker uses the globally configured timezone or the system local timezone.

The File sink is frequently used for local debugging and development, where engineers want to inspect raw worker output without involving external systems. It is also used for batch-oriented workflows, where logs or metrics are written to files and later ingested by other tools. Time-based file rotation via templated paths is a common pattern for log archival and offline processing pipelines.