Centralized logging
- Docker & Kubernetes course
- Advanced
- Lesson 55/70
- 4 min
Learn the centralized logging concept.
Centralized logging
kubectl logs is enough for a single pod, but it's useless across hundreds of pods, for deleted pods and for events from days ago. Centralized logging gathers all container logs in one place, stores them and makes them searchable. A typical pipeline has three parts:
- Collector (a DaemonSet on every node): Fluent Bit, Fluentd, Promtail, Vector
- Store and search: Elasticsearch/OpenSearch or Grafana Loki
- Interface: Kibana/OpenSearch Dashboards or Grafana
node: /var/log/containers/*.log ─► Fluent Bit (DaemonSet) ─► Elasticsearch ─► KibanaStructured logging
Applications should write their logs to stdout, preferably in JSON format. That way the collector parses the fields (level, request ID, user) and they can be filtered in queries:
# plain text: hard to search
2026-11-03 10:12:01 ERROR sepet kaydedilemedi kullanici=42
# JSON: separate fields
{"zaman":"2026-11-03T10:12:01Z","seviye":"error","mesaj":"sepet kaydedilemedi","kullanici":42,"istek_id":"a1b2c3"}Giving each request an ID and writing it in every service's log makes it possible to follow a request across services.
Practice Look at the log source
Write two commands: open an interactive debug session targeting the worker-1 node with the busybox:1.36 image (kubectl debug node/worker-1 -it --image=busybox:1.36); then show the last 3 log lines of the pods labelled app=api with prefixes.
Starter code
# buraya yazSample solution
kubectl debug node/worker-1 -it --image=busybox:1.36
kubectl logs -l app=api --tail=3 --prefixExpected output
Creating debugging pod node-debugger-worker-1-x7k2p with container debugger on node worker-1.
/ # ls /host/var/log/containers | head -2
api-7f8c9d6b5-abcde_magaza_api-3f2a….log
[pod/api-7f8c9d6b5-abcde/api] {"seviye":"info","mesaj":"istek","yol":"/urunler","sure_ms":12}
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 Why does the collector run as a DaemonSet?
- For less memory
- Because Elasticsearch requires it
- Because one copy is enough
- To read each node's container log files on that node Correct answer
Explanation: The kubelet keeps logs on the node's disk.
2 Complete the path of the container log files.
node: /var/log/____/*.lognode: /var/log/containers/*.logExplanation: The kubelet puts a link here for each container.
3 Match.
| Term | Meaning |
|---|---|
Fluent Bit |
Collector |
Elasticsearch |
Store and search |
Kibana |
Interface |
JSON log |
Parsable fields |