Redis
Redis Source
The Redis source collects observability data from Redis by consuming events stored in Redis data structures. It supports logs, metrics, and traces as output signals, enabling Redis to act as a lightweight ingestion or streaming layer within an observability pipeline.
This source operates as an aggregator, runs in stateless mode, and delivers data using best-effort semantics. Acknowledgements are not supported, making this source suitable for high-throughput, low-latency ingestion scenarios where occasional data loss is acceptable.
Connection Settings
url (required, string)
Defines the Redis endpoint to connect to.
- Must follow the format protocol://server:port/db
- Supported protocols:
- redis (unencrypted)
- rediss (TLS-enabled)
Operational note: Use rediss for all non-local or production deployments to protect data in transit.
key (required, string)
Specifies the Redis key from which events are consumed.
- Acts as the logical source of observability events.
- The interpretation of the key depends on the configured Redis data type.
redis_key (optional, string)
Adds the Redis key name as a field to each emitted event.
- Useful for identifying the origin key when consuming from multiple sources.
- If not set, the key name is not included in the event payload.
Redis Data Type Configuration
data_type (optional, string enum)
Specifies the Redis data structure used for ingestion.
Supported values:
- list – Consume events from a Redis list.
- channel – Consume events via Redis Pub/Sub.
Default: list
Operational note: Use list for durability-oriented ingestion patterns. Use channel for real-time, ephemeral streaming where message persistence is not required.
list (optional, object)
Configuration specific to the Redis list data type.
list.method (required, string enum)
Defines how events are removed from the list.
Supported values:
- lpop – Pop events from the head of the list.
- rpop – Pop events from the tail of the list.
Operational note: Ensure producers and consumers agree on list semantics to avoid unexpected ordering behavior.
Decoding Settings
decoding (optional, object)
Controls how raw bytes read from Redis are decoded into structured events.
Some decoders can automatically determine the resulting signal type (logs, metrics, traces).
decoding.codec (optional, string enum)
Specifies the codec used to decode incoming data.
Supported values include:
- bytes
- json
- avro
- gelf
- influxdb
- protobuf
- syslog
- otlp
- native
- native_json
- vrl
Default: bytes
Operational note: Selecting a structured codec reduces downstream processing complexity and improves schema consistency.
Avro Decoding
Used when codec = avro.
- Requires an explicit Avro schema.
- Optionally strips Confluent-style schema ID prefixes.
Operational note: Only supported Avro types should be used to avoid decoding failures.
JSON / Native JSON / InfluxDB / Syslog / GELF
These codecs support a lossy mode.
- When enabled, invalid UTF-8 sequences are replaced with a replacement character.
- Prevents decoding failures caused by malformed input.
Operational note: Lossy decoding is recommended for untrusted or user-generated data sources.
Protobuf Decoding
Used when codec = protobuf.
- Supports descriptor-based decoding.
- Can emit JSON-style field names if required.
Operational note: Descriptor files must be kept in sync with producer schemas.
OTLP Decoding
Used when codec = otlp.
- Automatically detects logs, metrics, or traces.
- Signal parsing order can be restricted for performance optimization.
VRL Decoding
Used when codec = vrl.
- Executes a VRL program to transform raw bytes.
- The final value of the . target becomes the decoded event.
Operational note: Compilation errors or explicit abort calls result in decoding failures.
Framing Settings
framing (optional, object)
Defines how raw byte streams are split into individual events.
Framing is critical when events are concatenated, chunked, or length-prefixed.
framing.method (optional, string enum)
Specifies the framing strategy.
Supported values:
- bytes
- character_delimited
- newline_delimited
- chunked_gelf
- length_delimited
- varint_length_delimited
- octet_counting
Default: bytes
Character / Newline Delimited Framing
Splits events using a delimiter character.
- Optional maximum buffer length can be enforced to prevent unbounded memory usage.
Chunked GELF Framing
Reassembles multi-part GELF messages.
- Supports decompression.
- Enforces limits on message size and pending messages.
Operational note: Recommended limits should be set in high-throughput environments to prevent memory exhaustion.
Length-Delimited & Varint Framing
Splits events using explicit length prefixes.
- Commonly used with binary or protobuf-based payloads.
- Maximum frame length can be enforced.
Octet Counting
Implements RFC-compliant octet counting framing.
Operational Considerations
- This source provides best-effort delivery; message loss is possible during failures.
- Redis persistence and eviction policies directly impact data reliability.
- High-throughput environments should carefully tune framing limits and decoding behavior.
- Redis Pub/Sub (channel) does not provide replay or durability guarantees.