Fluentd
- Docker & Kubernetes course
- Advanced
- Lesson 56/70
- 4 min
Learn log collection with Fluentd.
Fluentd and Fluent Bit
Fluentd and its lightweight sibling Fluent Bit are collectors that read logs (input), process them (filter/parser) and send them to destinations (output). In Kubernetes, Fluent Bit per node is common: it uses little memory and, with the kubernetes filter, adds the pod name, namespace and labels to every line:
[INPUT]
Name tail
Path /var/log/containers/*.log
Tag kube.*
[FILTER]
Name kubernetes
Match kube.*
Merge_Log On
[OUTPUT]
Name es
Match *
Host elasticsearch.loglama.svc
Port 9200Installing with Helm
Fluent Bit's official Helm chart installs the DaemonSet, the RBAC permissions and the configuration together. The output destination is changed with values:
helm repo add fluent https://fluent.github.io/helm-charts
helm install fluent-bit fluent/fluent-bit -n loglama --create-namespace
kubectl get ds -n loglama
kubectl logs -n loglama ds/fluent-bit --tail=5Practice Install Fluent Bit
Write three commands: add the fluent repository with the address https://fluent.github.io/helm-charts; install the fluent/fluent-bit chart as fluent-bit into the loglama namespace (creating it if missing); list the DaemonSets in loglama.
Starter code
# buraya yazSample solution
helm repo add fluent https://fluent.github.io/helm-charts
helm install fluent-bit fluent/fluent-bit -n loglama --create-namespace
kubectl get ds -n loglamaExpected output
"fluent" has been added to your repositories NAME: fluent-bit STATUS: deployed NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE fluent-bit 3 3 3 3 3
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 What is the job of the kubernetes filter?
- Restarting pods
- Encrypting logs
- Adding pod, namespace and label information to log lines Correct answer
- Deleting logs
Explanation: It queries this info from the API server; that's why it needs pod-read RBAC permission.
2 Write the input plugin that reads the end of files.
[INPUT]
Name ____
Path /var/log/containers/*.log[INPUT]
Name tail
Path /var/log/containers/*.logExplanation: tail follows new lines appended to the file.
3 Match the sections.
| Term | Meaning |
|---|---|
[INPUT] |
Reading |
[FILTER] |
Processing |
[OUTPUT] |
Sending |
Match |
Which tags |