AMQP
AMQP Source
The AMQP source collects events from AMQP 0.9.1–compatible message brokers, such as RabbitMQ. It consumes messages from configured queues and exchanges, decodes message payloads, and emits them into the telemetry pipeline.
This source is typically used in event-driven architectures where logs, metrics, or traces are transported asynchronously via message queues.
Collection Model
The AMQP source acts as a consumer that connects to an AMQP broker and continuously pulls messages from a queue.
Key properties of the collection model include:
- Pull-based consumption
- Broker-managed offsets and delivery guarantees
- Stateless processing at the source level
- Compatibility with multiple event signal types
Reliability guarantees depend on broker configuration and downstream acknowledgement handling.
Broker Connectivity
connection_string (required, string)
Defines the AMQP connection URI, including authentication credentials, broker address, port, virtual host, and connection timeout.
Both non-TLS (amqp) and TLS-enabled (amqps) connections are supported. Additional TLS parameters are configured separately.
consumer (optional, string)
Specifies the consumer identifier used when registering with the broker.
This value is useful for:
- Consumer identification
- Debugging and observability
- Broker-side monitoring
Queue and Routing
queue (optional, string)
Defines the name of the queue from which messages are consumed.
If not explicitly configured, the broker’s default behavior applies.
exchange_key (optional, string)
Specifies the exchange key associated with message routing.
This value is attached to emitted events as metadata.
routing_key_field (optional, string)
Defines the event field name under which the AMQP routing key is stored.
This allows routing information to be preserved for downstream processing and filtering.
offset_key (optional, string)
Specifies the event field used to store broker offset or delivery position metadata.
This field can be used for troubleshooting and correlation but does not control offset management.
Flow Control
prefetch_count (optional, uint)
Controls the maximum number of unacknowledged messages delivered by the broker to the consumer.
Lower values:
- Reduce memory usage
- Protect slow consumers
Higher values:
- Increase throughput
- Require more memory
This setting maps directly to AMQP QoS prefetch behavior.
Decoding Model
decoding (optional, object)
Controls how raw message payloads are decoded into structured events.
The decoding configuration determines:
- Payload format
- Output signal type (logs, metrics, traces)
- Error handling behavior
If not explicitly configured, messages are treated as raw bytes.
decoding.codec (optional, string)
Specifies the decoding format applied to incoming message payloads.
Supported decoding formats include:
- Structured formats (JSON, Avro, Protobuf)
- Observability protocols (OTLP)
- Logging formats (Syslog, GELF)
- Raw byte and custom VRL-based decoding
Some codecs can automatically infer and emit different signal types.
Signal-Aware Decoding
Certain codecs (such as OTLP and native formats) can emit:
- Logs
- Metrics
- Traces
When enabled, the source automatically routes decoded events to the appropriate signal pipeline.
Framing
framing (optional, object)
Controls how individual events are extracted from raw byte streams.
Framing is required when messages contain multiple events or when boundaries are not implicit.
Supported framing methods include:
- Byte-based pass-through
- Character-delimited frames
- Newline-delimited frames
- Length-prefixed frames
- GELF chunked frames
- Protobuf-compatible varint framing
Proper framing configuration is critical to prevent message corruption or memory overuse.
TLS Configuration
tls (optional, object)
Enables TLS encryption and authentication when connecting to the AMQP broker.
TLS can be used to:
- Encrypt broker traffic
- Validate broker identity
- Authenticate clients using certificates
Certificate Verification
When enabled, TLS verification ensures:
- Broker certificates are trusted
- Certificates are not expired
- Hostnames match certificate identities
Disabling verification weakens transport security and should only be used in controlled environments.
Acknowledgements Behavior
The AMQP source does not control acknowledgements at the source level.
Source-level acknowledgement configuration is deprecated. End-to-end acknowledgement behavior is managed globally or at sink level.
Reliability Considerations
- Message delivery follows at-least-once semantics
- Duplicate messages may occur
- Ordering is determined by broker configuration
- Backpressure is managed via prefetch and broker flow control
Downstream systems should be designed to handle duplicates safely.
Common Use Cases
- Asynchronous log ingestion pipelines
- Metrics and trace transport via message queues
- Decoupled telemetry architectures
- Integration with RabbitMQ-based systems
- Buffering and smoothing bursty telemetry traffic