Throttle

The throttling function is a crucial component in managing the flow of telemetry data in a pipeline. It regulates the rate at which events are processed, ensuring that the system does not become overwhelmed by an excessive volume of data. This function is especially useful in environments with high-throughput data sources where rate control is necessary to maintain stability and performance.

key_field

Optional: Defines the value used to group events into separate buckets that can be rate-limited independently.

Description:

When specified, events are grouped by the value of this field, and each group (or bucket) is rate-limited independently.

If not specified, or if the event does not have a corresponding value for the field, the event will not be rate-limited separately.

Example: key_field: "user_id"



Required: Specifies the maximum number of events allowed for a given bucket within the configured time window.

Description: Each unique key (from key_field) has its own threshold to limit the number of events.

Example: threshold: 100



window_secs

Required: Defines the time window, in seconds, during which the threshold is applied.

Description: The rate limit is enforced based on the number of events allowed in this time period.

Example: window_secs: 60.0 (applies a 100-event threshold every 60 seconds).