Summary - Logging

Combine Fluentd, Elasticsearch and Kibana to write a logging pipeline.

Summary: logging

In this group:

Centralized logging: collector (DaemonSet) → store and search → interface; logs of deleted pods are kept too.
Applications write to stdout, preferably in JSON with a request ID.
Fluent Bit/Fluentd: [INPUT] tail, [FILTER] kubernetes, [OUTPUT] es.
Elasticsearch stores in daily indices (cleaned up with ILM); Loki is a cheaper label-based alternative.
Kibana searches with data views and KQL; the collector needs hostPath, RBAC and tolerations.

node log -> Fluent Bit -> Elasticsearch -> Kibana
         -> Promtail   -> Loki          -> Grafana

Exercise The collector's permission

Write a ClusterRole YAML: rbac.authorization.k8s.io/v1; name fluent-bit; a single rule: apiGroups: [""], resources: ["pods", "namespaces"], verbs: ["get", "list", "watch"].

Show example solution

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: fluent-bit
rules:
  - apiGroups: [""]
    resources: ["pods", "namespaces"]
    verbs: ["get", "list", "watch"]

Test yourself

If an app writes its logs to /app/log.txt inside the container, what does centralized logging see?
- Only the errors
- The file name
- Nothing; the collector only reads the files coming from stdout/stderr
- All the lines
Show answer

Nothing; the collector only reads the files coming from stdout/stderr — The fix is writing to stdout or adding a sidecar that reads the file.

Write the filter that adds Kubernetes metadata to logs.

[FILTER]
    Name   ____
    Match  kube.*

Show answer

[FILTER]
    Name   kubernetes
    Match  kube.*

Match.

Fluent Bit

Collect
Elasticsearch

Store
Kibana

Search
Loki

Label index

Next lesson: Production design

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies