Summary - Logging
- Docker & Kubernetes course
- Advanced
- Lesson 60/70
- 5 min
Combine Fluentd, Elasticsearch and Kibana to write a logging pipeline.
Summary: logging
In this group:
- Centralized logging: collector (DaemonSet) → store and search → interface; logs of deleted pods are kept too.
- Applications write to stdout, preferably in JSON with a request ID.
- Fluent Bit/Fluentd:
[INPUT] tail,[FILTER] kubernetes,[OUTPUT] es. - Elasticsearch stores in daily indices (cleaned up with ILM); Loki is a cheaper label-based alternative.
- Kibana searches with data views and KQL; the collector needs hostPath, RBAC and tolerations.
node log -> Fluent Bit -> Elasticsearch -> Kibana
-> Promtail -> Loki -> GrafanaPractice The collector's permission
Write a ClusterRole YAML: rbac.authorization.k8s.io/v1; name fluent-bit; a single rule: apiGroups: [""], resources: ["pods", "namespaces"], verbs: ["get", "list", "watch"].
Starter code
# buraya yazSample solution
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: fluent-bit
rules:
- apiGroups: [""]
resources: ["pods", "namespaces"]
verbs: ["get", "list", "watch"]Expected output
clusterrole.rbac.authorization.k8s.io/fluent-bit created
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 If an app writes its logs to /app/log.txt inside the container, what does centralized logging see?
- Only the errors
- The file name
- Nothing; the collector only reads the files coming from stdout/stderr Correct answer
- All the lines
Explanation: The fix is writing to stdout or adding a sidecar that reads the file.
2 Write the filter that adds Kubernetes metadata to logs.
Question
[FILTER]
Name ____
Match kube.*
Answer
[FILTER]
Name kubernetes
Match kube.*Explanation: The kubernetes filter adds pod/namespace/labels.
3 Match.
| Term | Meaning |
|---|---|
Fluent Bit |
Collect |
Elasticsearch |
Store |
Kibana |
Search |
Loki |
Label index |