[[add-kubernetes-metadata]]
=== Add Kubernetes metadata

++++
<titleabbrev>add_kubernetes_metadata</titleabbrev>
++++

The `add_kubernetes_metadata` processor annotates each event with relevant
metadata based on which Kubernetes pod the event originated from.
At startup it detects an `in_cluster` environment and caches the
Kubernetes-related metadata. Events are only annotated if a valid configuration
is detected. If it's not able to detect a valid Kubernetes configuration,
the events are not annotated with Kubernetes-related metadata.

Each event is annotated with:

* Pod Name
* Pod UID
* Namespace
* Labels

In addition Node's metadata and Namespace's metadata are being added in Pod's metadata.

The `add_kubernetes_metadata` processor has two basic building blocks which are:

* Indexers
* Matchers

Indexers use pods metadata to create unique identifiers for each one of the
pods, these identifiers help to correlate the metadata of the observed pods with
actual events. For example, the `ip_port` indexer can take a Kubernetes pod and
create identifiers for it based on all its `pod_ip:container_port` combinations.

Matchers use information in events to construct lookup keys that match the
identifiers created by the indexers. For example, when the `fields` matcher takes
`["metricset.host"]` as a lookup field, it would construct a lookup key with the
value of the field `metricset.host`. When one of this lookup keys match with one
of the identifiers, the event is enriched with the metadata of the identified
pod.

ifdef::kubernetes_default_indexers[]
include::{kubernetes_default_indexers}[]
endif::kubernetes_default_indexers[]
ifndef::kubernetes_default_indexers[]
Each Beat can define its own default indexers and matchers which are enabled by
default. For example, Filebeat enables the `container` indexer, which identifies
pod metadata based on all container IDs, and a `logs_path` matcher, which takes
the `log.file.path` field, extracts the container ID, and uses it to retrieve
metadata.
endif::kubernetes_default_indexers[]

You can find more information about the available indexers and matchers, and some
examples in <<kubernetes-indexers-and-matchers>>.

The configuration below enables the processor when {beatname_lc} is run as a pod in
Kubernetes.

[source,yaml,subs="attributes+"]
-------------------------------------------------------------------------------
processors:
  - add_kubernetes_metadata:
ifndef::kubernetes_default_indexers[]
      # Defining indexers and matchers manually is required for {beatname_lc}, for instance:
      #indexers:
      #  - ip_port:
      #matchers:
      #  - fields:
      #      lookup_fields: ["metricset.host"]
endif::kubernetes_default_indexers[]
-------------------------------------------------------------------------------

The configuration below enables the processor on a Beat running as a process on
the Kubernetes node.

[source,yaml,subs="attributes+"]
-------------------------------------------------------------------------------
processors:
  - add_kubernetes_metadata:
      host: <hostname>
      # If kube_config is not set, KUBECONFIG environment variable will be checked
      # and if not present it will fall back to InCluster
      kube_config: ${HOME}/.kube/config
ifndef::kubernetes_default_indexers[]
      # Defining indexers and matchers manually is required for {beatname_lc}, for instance:
      #indexers:
      #  - ip_port:
      #matchers:
      #  - fields:
      #      lookup_fields: ["metricset.host"]
endif::kubernetes_default_indexers[]
-------------------------------------------------------------------------------

The configuration below has the default indexers and matchers disabled and
enables ones that the user is interested in.

[source,yaml]
-------------------------------------------------------------------------------
processors:
  - add_kubernetes_metadata:
      host: <hostname>
      # If kube_config is not set, KUBECONFIG environment variable will be checked
      # and if not present it will fall back to InCluster
      kube_config: ~/.kube/config
      default_indexers.enabled: false
      default_matchers.enabled: false
      indexers:
        - ip_port:
      matchers:
        - fields:
            lookup_fields: ["metricset.host"]
-------------------------------------------------------------------------------

The `add_kubernetes_metadata` processor has the following configuration settings:

`host`:: (Optional) Specify the node to scope {beatname_lc} to in case it
cannot be accurately detected, as when running {beatname_lc} in host network
mode.
`namespace`:: (Optional) Select the namespace from which to collect the
metadata. If it is not set, the processor collects metadata from all namespaces.
It is unset by default.
`add_resource_metadata`:: (Optional) Specify labels and annotations filters for the extra metadata coming from Node and Namespace.
 `add_resource_metadata` can be done for `node` or `namespace`. By default all labels will be included
while annotations are not added by default. This settings are useful when labels' and annotations'
storing requires special handling to avoid overloading the storage output. The enrichment of `node` or `namespace` metadata
can be individually disabled by setting `enabled: false`.
Example:

["source","yaml",subs="attributes"]
-------------------------------------------------------------------------------------
      add_resource_metadata:
        namespace:
          include_labels: ["namespacelabel1"]
        node:
          include_labels: ["nodelabel2"]
          include_annotations: ["nodeannotation1"]
-------------------------------------------------------------------------------------
`kube_config`:: (Optional) Use given config file as configuration for Kubernetes
client. It defaults to `KUBECONFIG` environment variable if present.
`default_indexers.enabled`:: (Optional) Enable/Disable default pod indexers, in
case you want to specify your own.
`default_matchers.enabled`:: (Optional) Enable/Disable default pod matchers, in
case you want to specify your own.
