--- title: Internal Metrics slug: telemetry-pipeline/internal-metrics docTags: createdAt: 2025-04-21T00:45:47.553Z --- # Internal Metrics Source Access and process metrics produced by worker itself within the same worker pipeline. This source exposes worker's internal operational metrics, enabling you to collect, transform, and route them like any other metrics signal. It is commonly used for self-observability, health monitoring, and pipeline performance analysis. ## Key Characteristics **Status:** Stable **Roles:** Aggregator, Daemon, Sidecar **Delivery:** At-least-once **Acknowledgements:** Not supported **Egress Mode:** Batch **State:** Stateless **Outputs:** Metrics ## Ingestion Model - Worker periodically samples its own internal state - Metrics are generated in-memory - Collected metrics are emitted into the pipeline as standard metric events - Metrics can be filtered, enriched, aggregated, or exported using normal worker components This source enables self-monitoring without requiring an external scraper. ## Source Configuration Parameters ## sources.\.type **required** **string** Specifies the source type. - Must be set to internal\_metrics ## sources.\.namespace **optional** **string** Overrides the default namespace applied to emitted metrics. - Useful for avoiding naming collisions - Enables alignment with organizational metric naming standards ## sources.\.scrape\_interval\_secs **optional** **float** Interval, in seconds, at which worker collects and emits its internal metrics. - Lower values increase metric resolution - Higher values reduce overhead This setting directly impacts: - Metric freshness - CPU and memory overhead **Default** :::BlockQuote 1.0 ::: ## Tag Configuration Controls automatic tag enrichment for emitted metrics. ## sources.\.tags **optional** **object** Defines tag-related behavior for the internal metrics source. ### sources.\.tags.host\_key **optional** **string** Overrides the tag name used to attach the peer host address to each metric. - The value contains the peer address including port (e.g. 1.2.3.4:9000) - By default, the global log\_schema.host\_key is used Special behavior: - Set to an empty string ("") to suppress this tag entirely ### sources.\.tags.pid\_key **optional** **string** Defines the tag name used to attach the current process ID (PID) to each metric. - Useful for debugging multi-process or sidecar deployments - Not enabled by default If unset, the PID tag is not added. ## Metric Categories The internal metrics source emits metrics covering: - Pipeline throughput and latency - Component-level event counts - Error and drop counters - Buffer usage - Backpressure indicators - Resource utilization related to worker internals These metrics reflect worker real-time operational health. ## Reliability Characteristics - Metrics are generated locally - No external dependencies - Stateless emission model - Batch-oriented export - No acknowledgement semantics This design prioritizes low overhead and continuous visibility. ## Deployment Patterns The internal metrics source can be used in: - Central aggregators - Per-node daemons - Sidecar deployments - Edge collectors It adapts automatically to the deployment role. ## Common Use Cases - Monitoring worker pipeline health - Detecting backpressure and bottlenecks - Observing ingestion and delivery rates - Exporting worker metrics to Prometheus, OTLP, or other backends - Building dashboards for telemetry infrastructure itself