--- title: Papertrail slug: telemetry-pipeline/papertrail docTags: createdAt: 2025-04-21T01:02:13.022Z --- # Papertrail Sink The **Papertrail** sink delivers **log events** to **Papertrail (SolarWinds)** over TCP. This guide explains **what each parameter does**, with operational notes for reliability, performance, and security. ## ## Core Destination ### endpoint (required) The TCP destination for Papertrail ingestion (host and port). What it controls: - Where logs are sent - Which Papertrail system/endpoint receives the stream (often tied to a specific account/log destination) Operational considerations: - If DNS/host is wrong, or port is blocked, worker will fail delivery and backlog will accumulate in the buffer. - Ensure firewall rules allow outbound connectivity to the configured endpoint. ## Buffering and Backpressure ### buffer (optional) Controls how worker buffers logs when Papertrail is slow/unreachable. buffer.type (optional, default: memory) - **memory:** higher throughput, but buffered data is lost on crash/restart - **disk:** more durable across restarts, but slower and requires disk sizing buffer.max\_size (required) Hard cap on buffer memory/disk usage. Operational impact: - Too small → backlog fills quickly during outages, leading to blocking or drops. - Too large → longer recovery time after outages, and more resource usage (especially on disk). buffer.max\_events (optional; relevant for memory, default: 500) Maximum number of buffered events (memory mode only). Operational impact: - Prevents unbounded event accumulation in memory. - Useful if event sizes vary significantly. buffer.when\_full (optional, default: block) Behavior when buffer is full: - **block:** applies backpressure upstream (safer when loss is unacceptable) - **drop\_newest:** drops the newest events to preserve throughput (use when continuity > completeness) ## Encoding ### encoding (required) Controls how log events are serialized into bytes before sending over TCP. Key point: - The chosen codec determines the wire format and what Papertrail will “see” as the message content. ### encoding.codec (required) Selects the output format. Common operational choices include: - **text / raw\_message:** simplest and most compatible with line-oriented log viewers - **json:** preserves structure (labels/fields), but can increase message size - **logfmt:** human-readable key/value format - **cef / gelf / avro / protobuf / otlp:** specialized formats for specific ecosystems (usually not the default choice for Papertrail unless you have a downstream parsing strategy) Operational considerations: - Larger encodings increase bandwidth and can stress both network and ingestion. - If you rely on parsing in Papertrail, choose a format that aligns with your parsing/search strategy. ### Codec-specific options (high-level meaning) - **encoding.json.pretty:** improves readability but increases size (avoid in high-volume production) - **encoding.only\_fields / encoding.except\_fields:** reduce payload and avoid leaking sensitive fields by filtering which fields are serialized - **encoding.timestamp\_format:** standardizes timestamp representation across logs if the codec supports timestamps For CSV/CEF/Protobuf/Avro-specific settings: - These define schemas, required identity fields, field extraction mappings, or serialization descriptors. - Use them only when you have a strict consumer expectation for those formats. ## Connection Liveness and Socket Behavior ### keepalive (optional) Controls TCP keepalive behavior. keepalive.time\_secs (optional) - Idle time before keepalive probes begin. - Helps detect dead connections faster when intermediate devices silently drop idle TCP sessions. Operational impact: - Useful in environments with NATs, load balancers, or firewalls that aggressively expire idle connections. ### send\_buffer\_bytes (optional) Sets the socket send buffer size (SO\_SNDBUF). Operational impact: - Larger send buffers can improve throughput and smooth short bursts, but increase memory usage. - Too large can hide downstream slowness until buffers fill, delaying backpressure visibility. ## Process Labeling (Papertrail Field) ### process (optional, default: vector) The process value shown in Papertrail for the emitted logs. What it controls: - How events are grouped/identified in Papertrail UI and searches - A convenient “source identity” dimension when multiple shippers send to the same destination Operational considerations: - Because it supports templating, you can map it to an event field to reflect service/app identity. - Be careful: making process highly variable can effectively increase cardinality in your logging metadata and reduce usability (too many distinct process values). ## TLS (Transport Security) ### tls (optional) Controls secure transport for the TCP connection. tls.enabled (optional) - When enabled, the sink uses TLS for the outgoing connection. - Recommended whenever logs traverse untrusted networks. tls.ca\_file (optional) - Additional trusted CA certificates used to validate the server certificate. tls.crt\_file / tls.key\_file / tls.key\_pass (optional) - Client identity (mTLS) configuration where required. - Used when the server expects client certificates. tls.verify\_certificate (optional) - Validates certificate trust chain. - Disabling this significantly reduces security and should be avoided outside tightly controlled test environments. tls.verify\_hostname (optional) - Validates that the server certificate matches the hostname used for the connection. - Disabling this also reduces security and should be avoided. tls.alpn\_protocols (optional) - Advanced TLS negotiation hint; generally not required for typical Papertrail TCP ingestion. ## Inputs ### inputs (required) Defines which upstream sources/transforms feed this sink. Operational considerations: - Wildcards are convenient but can unintentionally route unexpected log volume to Papertrail. - Ensure upstream filtering/redaction is applied before exporting to third-party services.