---
title: HTTP
slug: telemetry-pipeline/http
docTags: 
createdAt: 2025-04-21T01:00:35.699Z
---

# HTTP Sink

## Overview

The **HTTP sink** sends observability events to an arbitrary **HTTP endpoint**. It’s the “generic egress” option when you want to push **logs, metrics, and traces** into:

- your own ingestion API (custom collector / gateway), 
- an internal event intake service, 
- a vendor endpoint that accepts HTTP payloads, 
- or a lightweight bridge that forwards to Kafka/Loki/ClickHouse/etc. 

It supports **batching**, **buffering**, configurable **encoding + framing**, and multiple auth methods (basic, bearer, custom header, AWS signing).

## Supported Input Types

- Logs 
- Metrics 
- Traces 

## Core Configuration Parameters

### URI (required)

The full destination **URI** to send requests to.

Because it’s **templateable**, you can route per-event to different paths/hosts (careful: this can explode the number of distinct upstream targets, which affects connection reuse and performance).

## Authentication

### Auth (optional)

Defines how worker authenticates requests.

Supported strategies:

- **basic**: username + password (base64 in Authorization) 
- **bearer**: token passed as-is in Authorization: Bearer \<token>&#x20;
- **custom**: arbitrary authorization header value inserted as Authorization: \<value>&#x20;
- **aws**: AWS signing (SigV4-style) using access key/secret, optional assume role, etc.&#x20;

Practical notes:

- Use HTTPS with any header-based auth (basic/bearer/custom), otherwise credentials travel in cleartext.&#x20;
- AWS auth is for cases where the destination is an AWS service that expects signed requests.&#x20;

## Payload Encoding

### Encoding (required)

Controls how events are serialized into bytes and also determines which input types are supported for a chosen codec.

Common choices in practice:

- **json**: easiest for custom collectors and modern ingest APIs&#x20;
- **text / raw\_message**: when your endpoint expects plain text lines&#x20;
- **otlp**: when the receiver expects OpenTelemetry Protocol (protobuf)&#x20;
- **protobuf / avro**: when you have a strict schema and want compact payloads&#x20;

You can also constrain output fields:

- encoding.only\_fields: whitelist&#x20;
- encoding.except\_fields: blacklist&#x20;
- encoding.timestamp\_format: how timestamps are represented&#x20;

## Framing

### Framing (optional)

Defines **how multiple encoded events are delimited** inside the request body.

Key methods:

- **newline\_delimited**: typical for “one JSON per line” ingestion endpoints&#x20;
- **character\_delimited**: delimiter is configurable (e.g., ,), used when upstream expects a specific separator&#x20;
- **length\_delimited / varint\_length\_delimited**: useful for binary/protobuf-style transports&#x20;
- **bytes**: no delimiter (you usually pair this with a batch that is already a single blob)&#x20;

If your receiver expects NDJSON, the classic pairing is:

- encoding.codec = "json"&#x20;
- framing.method = "newline\_delimited"&#x20;

## Delivery Shaping

### Batch (optional)

Controls **when worker flushes** a set of events into one HTTP request:

- batch.max\_bytes: cap by size (uncompressed size before serialization/compression)&#x20;
- batch.max\_events: cap by event count&#x20;
- batch.timeout\_secs: flush by time&#x20;

Batch tuning is the main lever for:

- throughput (bigger batches → fewer requests),&#x20;
- latency (shorter timeout → quicker delivery),&#x20;
- and upstream load (request rate).&#x20;

### Buffer (optional)

Controls local buffering for backpressure / durability:

- buffer.type: memory or disk&#x20;
- buffer.max\_size: total buffer memory/disk budget&#x20;
- buffer.max\_events: only relevant for memory buffers&#x20;
- buffer.when\_full: block vs drop\_newest&#x20;

This is the main control for “what happens when the endpoint slows down”.

## Compression

### Compression (optional)

Compresses the HTTP request body:

- gzip, snappy, zlib, zstd, or none&#x20;

Use compression when bandwidth is expensive or payload volume is high. It trades CPU for network savings and can help if the receiver supports it efficiently.

## HTTP Request Customization

### Method (optional)

HTTP method used for sending requests. Default is POST.

### Request Headers (optional)

request.headers lets you attach arbitrary headers to every request.

Both header names and values can be **templateable**, so you can do patterns like:

- per-tenant API keys,&#x20;
- dynamic routing hints,&#x20;
- correlation identifiers.&#x20;

Note: the older headers option is deprecated in favor of request.headers.

## Proxy and TLS

### Proxy (optional)

Standard proxy config with:

- proxy.http, proxy.https&#x20;
- proxy.no\_proxy list/patterns&#x20;

### TLS (optional)

TLS settings for secure delivery:

- custom CA (tls.ca\_file)&#x20;
- client identity (mTLS: tls.crt\_file + tls.key\_file)&#x20;
- hostname verification, cert verification controls&#x20;
- SNI server name override (tls.server\_name)&#x20;

## Advanced Shaping: JSON Envelope

### Payload Prefix / Suffix (optional)

payload\_prefix and payload\_suffix allow wrapping character-delimited JSON payloads into a larger JSON object.

This is used when the receiver expects a JSON envelope like:

:::BlockQuote
\{ "records": \[ \<event1>, \<event2>, ... ] }
:::

worker can emit the delimiters + wrapper as long as the prefix/suffix form valid JSON together.