NATS
NATS Sink
The NATS sink publishes log events to NATS subjects. This guide explains what each parameter does, and what it means operationally for reliability, performance, and security.
Core Destination and Routing
url (required)
NATS server URL to connect to.
What it controls:
- Which NATS cluster/broker worker connects to
- Which port and protocol are used (defaults to port 4222 if omitted)
Operational considerations:
- Wrong URL or blocked port leads to publish failures and buffer growth.
- For HA, your NATS deployment may expose multiple URLs; ensure your configuration aligns with your cluster access strategy.
subject (required)
The NATS subject to publish messages to. This supports templating, enabling per-event routing.
What it controls:
- Message routing and subscription matching in NATS (subjects act like hierarchical topics)
- Downstream fan-out behavior based on subscriber patterns (wildcards, prefix matches)
Operational considerations:
- If you make the subject highly dynamic (e.g., per-host, per-container, per-namespace), you can create very high “subject cardinality,” which makes subscriber management and observability harder.
- A stable subject taxonomy (few predictable patterns) is typically easier to operate.
Connection Identity
connection_name (optional, default: vector)
A human-readable name assigned to the NATS connection.
What it controls:
- How this client appears in NATS monitoring/observability tools
- Easier troubleshooting (identifying which worker instance is connected)
Operational note:
- Useful when many worker agents connect to the same NATS cluster.
Authentication
auth (optional)
Defines how worker authenticates to NATS.
auth.strategy (required when auth is configured)
Selects the authentication method:
- user_password: traditional username/password
- token: shared token authentication
- nkey: NATS NKey public/seed-based auth (cryptographic identity)
- credentials_file: JWT-based credentials file authentication
Operational considerations:
- Choose the strategy that matches your NATS security posture:
- token/user-password is simpler but typically weaker operationally (rotation, leakage risk).
- nkey/credentials_file are more structured for secure environments (stronger identity model, better rotation practices).
user_password strategy
- auth.user_password.user: username
- auth.user_password.password: password
token strategy
- auth.token.value: token
nkey strategy
- auth.nkey.nkey: public identity (conceptually like a public key / user identity)
- auth.nkey.seed: private seed (conceptually like a private key)
Operational caution:
- Treat seed as a high-value secret. Leakage enables impersonation.
credentials_file strategy
- auth.credentials_file.path: path to the credentials file (JWT-based auth material)
Operational note:
- This is commonly used in secured NATS setups with operator/account/user JWTs.
JetStream Mode (Optional Reliability & Dedup Layer)
jetstream (optional)
If enabled, messages are published using JetStream.
jetstream.enabled (optional, default: false)
When enabled:
- The subject must belong to an existing JetStream stream.
Operational impact:
- Enables durability and stronger delivery patterns compared to plain core NATS publish.
- Requires JetStream to be provisioned correctly (streams, storage, retention, limits).
jetstream.headers (optional)
Adds NATS headers to each message (JetStream-aware use cases).
jetstream.headers.message_id (optional)
A unique message identifier (supports templating) used for deduplication.
Operational impact:
- If your pipeline can re-send the same event (retries, restarts), a stable message_id helps JetStream deduplicate.
- If the message_id is not stable or not unique enough, dedup may be ineffective or may incorrectly deduplicate distinct events.
Buffering and Backpressure
buffer (optional)
Controls how worker buffers events when NATS is slow/unreachable.
buffer.type (optional, default: memory)
- memory: highest performance, but buffered data is lost on crash/restart
- disk: more durable across restarts; slower and needs disk sizing/IOPS planning
buffer.max_size (required)
Hard cap on buffer memory/disk usage.
Operational impact:
- Too small → backpressure/drops during short outages
- Too large → long drain times and burst load when NATS recovers
buffer.max_events (optional; memory only, default: 500)
Event-count cap for memory buffers.
buffer.when_full (optional, default: block)
Behavior when the buffer is full:
- block: backpressure upstream (preferred when you want to avoid data loss)
- drop_newest: drop incoming events (preferred when keeping the system responsive is more important than completeness)
Encoding (Message Payload Format)
encoding (required)
Controls how a log event is serialized into bytes before publishing to NATS.
encoding.codec (required)
Selects the output format (for example: json, text, logfmt, raw_message, etc.).
Operational considerations:
- json: preserves structure, best for downstream parsing—larger payloads
- text / raw_message: simplest, smallest, but less structured
- logfmt: human-friendly key/value format
- Specialized formats (CEF, GELF, Avro, Protobuf, OTLP) should be chosen only if consumers expect them.
Field filtering and timestamps
- encoding.only_fields / encoding.except_fields: include/exclude fields to reduce size and prevent leakage of sensitive data
- encoding.timestamp_format: standardizes timestamp representation (important when downstream systems rely on strict formats)
Request / Publish Behavior Controls
request (optional)
Controls concurrency, retries, timeouts, and rate-limiting for outbound operations.
Key settings:
- request.concurrency (default: none): fixed concurrency of 1 (predictable behavior)
- rate_limit_num / rate_limit_duration_secs: caps outbound rate
- retry_attempts / retry backoff / timeout_secs: controls how aggressively worker retries when NATS is not accepting publishes
Operational impact:
- More retries improve survivability but can amplify duplicates (especially without JetStream dedup) and increase backlog during outages.
- Timeouts that are too low can cause unnecessary retries; too high can stall pipeline recovery.
TLS (Transport Security)
tls (optional)
Controls TLS security for the NATS connection.
Key options:
- tls.enabled: enables TLS
- tls.ca_file: trust roots
- tls.crt_file / tls.key_file / tls.key_pass: client certificate authentication (mTLS), if required
- tls.server_name: SNI override
- tls.verify_certificate / tls.verify_hostname: certificate and hostname verification
Operational guidance:
- Do not disable verification in production unless you fully understand the security trade-off.