Honeycomb
Honeycomb Sink
Overview
The Honeycomb sink delivers log events to Honeycomb using Honeycomb’s HTTP ingestion API. It is designed for high-volume log ingestion with support for batching, buffering, compression, and request-level flow control.
This sink is typically used when Honeycomb is the primary platform for:
- exploratory analysis,
- high-cardinality debugging,
- and service-level observability workflows based on structured logs.
Supported Input Types
- Logs
Authentication
API Key (required)
Authentication is performed using a Honeycomb API key.
The API key:
- is sent with each request,
- authorizes writes to the configured dataset,
- should be scoped and rotated according to Honeycomb best practices.
Dataset Configuration
Dataset (required)
Defines the target dataset where log events are written.
Key considerations:
- The dataset is static per sink instance.
- For multi-dataset routing (per service, tenant, or environment), logs should be split upstream and routed to separate sink definitions.
- Dataset naming strategy directly affects query ergonomics and cost visibility in Honeycomb.
Delivery Shaping
Batch Behavior
Controls how log events are grouped before being sent to Honeycomb.
Batching parameters determine:
- request frequency,
- payload size,
- ingestion latency.
Typical usage:
- Larger batches reduce HTTP overhead.
- Smaller batches reduce end-to-end delivery latency.
Buffering Behavior
Defines how events are buffered locally when downstream delivery is slower than ingestion.
Supported buffer types:
- Memory buffer Higher throughput, volatile on restart or crash.
- Disk buffer More durable, suitable for environments where data loss during restarts is unacceptable.
Buffer behavior when full:
- Block: applies backpressure to upstream components.
- Drop newest: prioritizes pipeline continuity over data completeness.
Payload Optimization
Compression
Controls compression applied to outbound payloads.
Supported algorithms:
- gzip
- snappy
- zlib
- zstd
- none
Default behavior favors zstd, which provides a strong compression ratio while maintaining acceptable CPU usage in most environments.
Field Selection and Timestamp Handling
The sink allows field-level control before serialization:
- Include-only specific fields
- Exclude noisy or high-cardinality fields
- Normalize timestamp representation (RFC3339, Unix, millisecond/nanosecond precision)
This is a critical mechanism for:
- reducing ingestion volume,
- controlling cost,
- and improving query performance inside Honeycomb.
Network and Transport Controls
Endpoint Configuration
Defines the Honeycomb ingestion endpoint.
The default endpoint targets Honeycomb’s public SaaS API. Custom endpoints may be used when:
- routing through an internal gateway,
- enforcing egress controls,
- or integrating with Honeycomb-compatible ingestion services.
Proxy Support
Supports HTTP and HTTPS proxies, including exclusion lists for direct connections.
Used primarily in:
- enterprise networks with restricted outbound access,
- segmented environments with controlled egress paths.
Request Behavior
Advanced request controls are available to manage delivery under load:
- adaptive or fixed concurrency
- rate limiting
- retry and backoff behavior
- request timeouts
These settings are typically tuned when:
- Honeycomb imposes ingestion limits,
- network reliability is variable,
- or multiple sinks compete for shared egress capacity.