Papertrail
Papertrail Sink
The Papertrail sink delivers log events to Papertrail (SolarWinds) over TCP. This guide explains what each parameter does, with operational notes for reliability, performance, and security.
Core Destination
endpoint (required)
The TCP destination for Papertrail ingestion (host and port).
What it controls:
- Where logs are sent
- Which Papertrail system/endpoint receives the stream (often tied to a specific account/log destination)
Operational considerations:
- If DNS/host is wrong, or port is blocked, worker will fail delivery and backlog will accumulate in the buffer.
- Ensure firewall rules allow outbound connectivity to the configured endpoint.
Buffering and Backpressure
buffer (optional)
Controls how worker buffers logs when Papertrail is slow/unreachable.
buffer.type (optional, default: memory)
- memory: higher throughput, but buffered data is lost on crash/restart
- disk: more durable across restarts, but slower and requires disk sizing
buffer.max_size (required)
Hard cap on buffer memory/disk usage.
Operational impact:
- Too small → backlog fills quickly during outages, leading to blocking or drops.
- Too large → longer recovery time after outages, and more resource usage (especially on disk).
buffer.max_events (optional; relevant for memory, default: 500)
Maximum number of buffered events (memory mode only).
Operational impact:
- Prevents unbounded event accumulation in memory.
- Useful if event sizes vary significantly.
buffer.when_full (optional, default: block)
Behavior when buffer is full:
- block: applies backpressure upstream (safer when loss is unacceptable)
- drop_newest: drops the newest events to preserve throughput (use when continuity > completeness)
Encoding
encoding (required)
Controls how log events are serialized into bytes before sending over TCP.
Key point:
- The chosen codec determines the wire format and what Papertrail will “see” as the message content.
encoding.codec (required)
Selects the output format. Common operational choices include:
- text / raw_message: simplest and most compatible with line-oriented log viewers
- json: preserves structure (labels/fields), but can increase message size
- logfmt: human-readable key/value format
- cef / gelf / avro / protobuf / otlp: specialized formats for specific ecosystems (usually not the default choice for Papertrail unless you have a downstream parsing strategy)
Operational considerations:
- Larger encodings increase bandwidth and can stress both network and ingestion.
- If you rely on parsing in Papertrail, choose a format that aligns with your parsing/search strategy.
Codec-specific options (high-level meaning)
- encoding.json.pretty: improves readability but increases size (avoid in high-volume production)
- encoding.only_fields / encoding.except_fields: reduce payload and avoid leaking sensitive fields by filtering which fields are serialized
- encoding.timestamp_format: standardizes timestamp representation across logs if the codec supports timestamps
For CSV/CEF/Protobuf/Avro-specific settings:
- These define schemas, required identity fields, field extraction mappings, or serialization descriptors.
- Use them only when you have a strict consumer expectation for those formats.
Connection Liveness and Socket Behavior
keepalive (optional)
Controls TCP keepalive behavior.
keepalive.time_secs (optional)
- Idle time before keepalive probes begin.
- Helps detect dead connections faster when intermediate devices silently drop idle TCP sessions.
Operational impact:
- Useful in environments with NATs, load balancers, or firewalls that aggressively expire idle connections.
send_buffer_bytes (optional)
Sets the socket send buffer size (SO_SNDBUF).
Operational impact:
- Larger send buffers can improve throughput and smooth short bursts, but increase memory usage.
- Too large can hide downstream slowness until buffers fill, delaying backpressure visibility.
Process Labeling (Papertrail Field)
process (optional, default: vector)
The process value shown in Papertrail for the emitted logs.
What it controls:
- How events are grouped/identified in Papertrail UI and searches
- A convenient “source identity” dimension when multiple shippers send to the same destination
Operational considerations:
- Because it supports templating, you can map it to an event field to reflect service/app identity.
- Be careful: making process highly variable can effectively increase cardinality in your logging metadata and reduce usability (too many distinct process values).
TLS (Transport Security)
tls (optional)
Controls secure transport for the TCP connection.
tls.enabled (optional)
- When enabled, the sink uses TLS for the outgoing connection.
- Recommended whenever logs traverse untrusted networks.
tls.ca_file (optional)
- Additional trusted CA certificates used to validate the server certificate.
tls.crt_file / tls.key_file / tls.key_pass (optional)
- Client identity (mTLS) configuration where required.
- Used when the server expects client certificates.
tls.verify_certificate (optional)
- Validates certificate trust chain.
- Disabling this significantly reduces security and should be avoided outside tightly controlled test environments.
tls.verify_hostname (optional)
- Validates that the server certificate matches the hostname used for the connection.
- Disabling this also reduces security and should be avoided.
tls.alpn_protocols (optional)
- Advanced TLS negotiation hint; generally not required for typical Papertrail TCP ingestion.
Inputs
inputs (required)
Defines which upstream sources/transforms feed this sink.
Operational considerations:
- Wildcards are convenient but can unintentionally route unexpected log volume to Papertrail.
- Ensure upstream filtering/redaction is applied before exporting to third-party services.