Keep
Keep Sink
Overview
The Keep sink enables worker to deliver log events to Keep, allowing logs to be forwarded to Keep’s alerting and event ingestion endpoints. This sink is designed for environments that require external alert correlation or log-driven incident workflows.
The sink operates in batch mode, aggregating log events before transmission to improve delivery efficiency.
Supported Input Types
- Logs
Prerequisites
Before configuring the Keep sink, ensure the following prerequisites are met:
- A valid Keep API key with permission to ingest events
- Network connectivity from worker to the Keep endpoint
- Upstream worker sources or transforms producing log events
Core Configuration Parameters
API Key
The API key is used to authenticate worker against the Keep ingestion service. This value must be kept secure and should be managed using environment variables or a secret management system.
Endpoint
The endpoint defines the HTTP destination where log events are delivered. If not explicitly set, the default Keep event ingestion endpoint is used.
Typical use cases for custom endpoints include:
- Development or staging Keep deployments
- Multi-tenant or provider-specific routing
- Custom provider identifiers
Event Batching
The Keep sink supports event batching to optimize throughput and reduce request overhead.
Batching behavior can be controlled by:
- Maximum batch size in bytes
- Maximum number of events per batch
- Maximum time a batch can remain open before being flushed
Batching is especially recommended for high-volume log streams.
Buffering Behavior
The sink supports both in-memory and disk-based buffering strategies.
Memory Buffering
- Provides higher performance
- Events are lost if the process restarts unexpectedly
Disk Buffering
- Provides higher durability
- Buffered events survive restarts and crashes
- Recommended for production environments where data loss is unacceptable
Buffer limits can be defined to control memory or disk usage and to apply backpressure or dropping behavior when limits are reached.
Encoding and Field Selection
The Keep sink allows preprocessing of events before transmission:
- Include only specific fields
- Exclude sensitive or unnecessary fields
- Control timestamp formatting
This is useful for:
- Reducing payload size
- Removing confidential fields
- Aligning log structure with Keep’s event model
Proxy Support
Outbound traffic to Keep can be routed through HTTP or HTTPS proxies.
Proxy configuration supports:
- Separate HTTP and HTTPS proxy endpoints
- Host-based proxy bypass rules
- Enterprise network and egress control requirements
Request Behavior and Reliability
The sink supports advanced request handling options to improve delivery stability under load or partial failures, including:
- Adaptive request concurrency
- Rate limiting
- Retry behavior with backoff and jitter
- Request timeout control
These controls help ensure stable operation when sending events to Keep under variable network or service conditions.
Typical Use Cases
The Keep sink is commonly used in scenarios such as:
- Forwarding application or infrastructure logs to Keep for alert correlation
- Integrating worker-based log pipelines with Keep-driven incident workflows
- Centralizing alert-triggering logs from distributed systems
- Supporting DevSecOps and SRE monitoring pipelines