Kubernetes Logs

Collect logs directly from Kubernetes Nodes by reading Pod log files on the host filesystem. This source automatically enriches log events with Pod, Namespace, and Node metadata using the Kubernetes API.

It is designed to run as a DaemonSet, collecting logs only from the Node it is scheduled on.

This source is optimized for low-latency log shipping rather than guaranteed delivery.

Controls whether partial log messages are automatically merged.

Some container runtimes split long log lines into multiple fragments. When enabled, these fragments are reassembled into a single event before forwarding.

This is enabled by default and recommended for application logs.

Defines the directory where file read positions are persisted.

Operational notes:

Specifies how long metadata entries are retained after a Pod deletion event is observed.

A longer delay allows:

If metadata expires before log ingestion completes, logs are forwarded without enrichment and a warning is emitted.

Defines which log files are eligible for collection.

By default, all files under the Pod log directory are included.

Defines glob patterns for files that should be ignored.

Common exclusions include:

Skips files whose last modification time exceeds the specified age.

Useful for:

Applies an additional Kubernetes field selector when watching Pods.

This works alongside the built-in Node filter, which ensures only Pods scheduled on the current Node are observed.

Filters Pods based on Kubernetes labels.

This allows selective log collection without modifying workloads.

Filters Namespaces based on labels before Pod discovery occurs.

This is useful in large clusters to limit API load and memory usage.

Defines how many lines are read to generate a file checksum.

If a file contains fewer lines than this value, it will not be read at all.

This helps distinguish between files with similar paths during rotation.

Controls how frequently the filesystem is scanned for new files.

Lower values increase responsiveness but may introduce additional filesystem overhead.

Overrides the field name used to store the ingestion timestamp.

This enables latency analysis between:

Defines the default timezone for timestamps that do not include explicit timezone information.

Controls whether Namespace-level metadata is fetched and attached to events.

Disabling this:

Enabled by default.

The following configuration groups control how Kubernetes metadata is mapped into event fields. Each field can be disabled by setting its target to an empty value.

Controls enrichment with Namespace metadata, including labels.

Controls enrichment with Node metadata, including node labels.

Controls enrichment with Pod-level metadata, including:

This enrichment is central to correlating logs with Kubernetes workloads.

Defines the maximum size of a single log line.

Lines exceeding this limit are discarded to protect against malformed or unexpected input.

Defines the maximum size of a merged log line after partial events are combined.

This only applies when partial merge is enabled.

Limits how many bytes are read from a single file before switching to another file.

This helps distribute read capacity across many active log files.

Controls whether older files are fully drained before newer files are read.

When enabled, this prioritizes backlog reduction over near-real-time logs.

Defines where reading starts when a file is first discovered:

Specifies how long file handles are kept open after log rotation.

This prevents premature closure when log writers are slow to release files.

Specifies an explicit kubeconfig file.

If not set, in-cluster configuration is used automatically.

Controls whether Kubernetes API requests may be served from a local cache.

Enabling this can reduce API load but may slightly delay metadata updates.

Defines the name of the Node the source is running on.

By default, this is populated from an environment variable injected by Kubernetes at Pod creation time.

Controls emission of internal file-related metrics.

An optional file tag can be enabled, but this introduces unbounded cardinality and should be used with caution.