Vector
Vector
Relay observability data to another Vector instance
Configuration
address
requiredstringliteral
The downstream Vector address to which to connect.
Both IP address and hostname are accepted formats.
The address must include a port.
Examples
batch
optionalobjectEvent batching behavior.batch.max_bytesoptionaluint
The maximum size of a batch that is processed by a sink.
This is based on the uncompressed size of the batched events, before they are serialized or compressed.
batch.max_eventsoptionaluintThe maximum size of a batch before it is flushed.default: 1000(events)batch.timeout_secsoptionalfloatThe maximum age of a batch before it is flushed.default: 1(seconds)
buffer
optionalobject
Configures the buffering behavior for this sink.
More information about the individual buffer types, and buffer behavior, can be found in the Buffering Model section.
buffer.max_eventsoptionaluintThe maximum number of events allowed in the buffer.Relevant when: type = "memory"default: 500buffer.max_sizerequireduint
The maximum allowed amount of allocated memory the buffer can hold.
If type = "disk" then must be at least ~256 megabytes (268435488 bytes).
buffer.typeoptionalstringliteralenumThe type of buffer to use.Enum options
Option | Description |
|---|---|
disk | Events are buffered on disk. This is less performant, but more durable. Data that has been synchronized to disk will not be lost if Vector is restarted forcefully or crashes. Data is synchronized to disk every 500ms. |
memory | Events are buffered in memory. This is more performant, but less durable. Data will be lost if Vector is restarted forcefully or crashes. |
default: memorybuffer.when_fulloptionalstringliteralenumEvent handling behavior when a buffer is full.Enum options
Option | Description |
|---|---|
block | Wait for free space in the buffer. This applies backpressure up the topology, signalling that sources should slow down the acceptance/consumption of events. This means that while no data is lost, data will pile up at the edge. |
drop_newest | Drops the event instead of waiting for free space in buffer. The event will be intentionally dropped. This mode is typically used when performance is the highest priority, and it is preferable to temporarily lose events rather than cause a slowdown in the acceptance/consumption of events. |
default: block
compression
optionalbool
Whether or not to compress requests.
If set to true, requests are compressed with gzip.
default: false
request
optionalobject
Middleware settings for outbound requests.
Various settings can be configured, such as concurrency and rate limits, timeouts, and retry behavior.
Note that the retry backoff policy follows the Fibonacci sequence.
request.adaptive_concurrencyoptionalobject
Configuration of adaptive concurrency parameters.
These parameters typically do not require changes from the default, and incorrect values can lead to meta-stable or unstable performance and sink behavior. Proceed with caution.
request.adaptive_concurrency.decrease_ratiooptionalfloat
The fraction of the current value to set the new concurrency limit when decreasing the limit.
Valid values are greater than 0 and less than 1. Smaller values cause the algorithm to scale back rapidly when latency increases.
Note: The new limit is rounded down after applying this ratio.
default: 0.9request.adaptive_concurrency.ewma_alphaoptionalfloat
The weighting of new measurements compared to older measurements.
Valid values are greater than 0 and less than 1.
ARC uses an exponentially weighted moving average (EWMA) of past RTT measurements as a reference to compare with the current RTT. Smaller values cause this reference to adjust more slowly, which may be useful if a service has unusually high response variability.
default: 0.4request.adaptive_concurrency.initial_concurrencyoptionaluint
The initial concurrency limit to use. If not specified, the initial limit is 1 (no concurrency).
Datadog recommends setting this value to your service’s average limit if you’re seeing that it takes a long time to ramp up adaptive concurrency after a restart. You can find this value by looking at the adaptive_concurrency_limit metric.
default: 1request.adaptive_concurrency.max_concurrency_limitoptionaluint
The maximum concurrency limit.
The adaptive request concurrency limit does not go above this bound. This is put in place as a safeguard.
default: 200request.adaptive_concurrency.rtt_deviation_scaleoptionalfloat
Scale of RTT deviations which are not considered anomalous.
Valid values are greater than or equal to 0, and reasonable values range from 1.0 to 3.0.
When calculating the past RTT average, a secondary “deviation” value is also computed that indicates how variable those values are. That deviation is used when comparing the past RTT average to the current measurements, so we can ignore increases in RTT that are within an expected range. This factor is used to scale up the deviation to an appropriate range. Larger values cause the algorithm to ignore larger increases in the RTT.
default: 2.5request.concurrencyoptionalstringliteralenumuint
Configuration for outbound request concurrency.
This can be set either to one of the below enum values or to a positive integer, which denotes a fixed concurrency limit.
Enum options
Option | Description |
|---|---|
adaptive | Concurrency is managed by Vector’s Adaptive Request Concurrency feature. |
none | A fixed concurrency of 1. Only one request can be outstanding at any given time. |
default: adaptiverequest.rate_limit_duration_secsoptionaluintThe time window used for the rate_limit_num option.default: 1(seconds)request.rate_limit_numoptionaluintThe maximum number of requests allowed within the rate_limit_duration_secs time window.default: 9.223372036854776e+18(requests)request.retry_attemptsoptionaluintThe maximum number of retries to make for failed requests.default: 9.223372036854776e+18(retries)request.retry_initial_backoff_secsoptionaluint
The amount of time to wait before attempting the first retry for a failed request.
After the first retry has failed, the Fibonacci sequence is used to select future backoffs.
default: 1(seconds)request.retry_jitter_modeoptionalstringliteralenumThe jitter mode to use for retry backoff behavior.Enum options
Option | Description |
|---|---|
Full | Full jitter. The random delay is anywhere from 0 up to the maximum current delay calculated by the backoff strategy. Incorporating full jitter into your backoff strategy can greatly reduce the likelihood of creating accidental denial of service (DoS) conditions against your own systems when many clients are recovering from a failure state. |
None | No jitter. |
default: Fullrequest.retry_max_duration_secsoptionaluintThe maximum amount of time to wait between retries.default: 30(seconds)request.timeout_secsoptionaluint
The time a request can take before being aborted.
Datadog highly recommends that you do not lower this value below the service’s internal timeout, as this could create orphaned requests, pile on retries, and result in duplicate data downstream.
default: 60(seconds)
tls
optionalobjectConfigures the TLS options for incoming/outgoing connections.tls.alpn_protocolsoptional[string]
Sets the list of supported ALPN protocols.
Declare the supported ALPN protocols, which are used during negotiation with a peer. They are prioritized in the order that they are defined.
tls.ca_fileoptionalstringliteral
Absolute path to an additional CA certificate file.
The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format.
Examplestls.crt_fileoptionalstringliteral
Absolute path to a certificate file used to identify this server.
The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format.
If this is set and is not a PKCS#12 archive, key_file must also be set.
Examplestls.enabledoptionalbool
Whether to require TLS for incoming or outgoing connections.
When enabled and used for incoming connections, an identity certificate is also required. See tls.crt_file for more information.
tls.key_fileoptionalstringliteral
Absolute path to a private key file used to identify this server.
The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format.
Examplestls.key_passoptionalstringliteral
Passphrase used to unlock the encrypted key file.
This has no effect unless key_file is set.
Examplestls.server_nameoptionalstringliteral
Server name to use when using Server Name Indication (SNI).
Only relevant for outgoing connections.
Examplestls.verify_certificateoptionalbool
Enables certificate verification. For components that create a server, this requires that the client connections have a valid client certificate. For components that initiate requests, this validates that the upstream has a valid certificate.
If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on, until the verification process reaches a root certificate.
Do NOT set this to false unless you understand the risks of not verifying the validity of certificates.
tls.verify_hostnameoptionalbool
Enables hostname verification.
If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension.
Only relevant for outgoing connections.
Do NOT set this to false unless you understand the risks of not verifying the remote hostname.