diff --git a/cmd/fluent-bit/README.md b/cmd/fluent-bit/README.md index a6687c36e22a..14a8cf3969be 100644 --- a/cmd/fluent-bit/README.md +++ b/cmd/fluent-bit/README.md @@ -1,8 +1,8 @@ -# fluent-bit output plugin +# Fluent Bit output plugin -[Fluent-bit](https://fluentbit.io/) is a Fast and Lightweight Data Forwarder, it can be configured with the [Loki output plugin](https://fluentbit.io/documentation/0.12/output/) to ship logs to Loki. You can define which log files you want to collect using the [`Tail`](https://fluentbit.io/documentation/0.12/input/tail.html) [input plugin](https://fluentbit.io/documentation/0.12/getting_started/input.html). Additionally fluent-bit supports multiple `Filter` and `Parser` plugins (`Kubernetes`, `JSON`, etc..) to structure and alter log lines. +[Fluent Bit](https://fluentbit.io/) is a Fast and Lightweight Data Forwarder, it can be configured with the [Loki output plugin](https://fluentbit.io/documentation/0.12/output/) to ship logs to Loki. You can define which log files you want to collect using the [`Tail`](https://fluentbit.io/documentation/0.12/input/tail.html) [input plugin](https://fluentbit.io/documentation/0.12/getting_started/input.html). Additionally Fluent Bit supports multiple `Filter` and `Parser` plugins (`Kubernetes`, `JSON`, etc..) to structure and alter log lines. -This plugin is implemented with [fluent-bit's go plugin](https://github.com/fluent/fluent-bit-go) interface. It pushes logs to Loki using a GRPC connection. +This plugin is implemented with [Fluent Bit's Go plugin](https://github.com/fluent/fluent-bit-go) interface. It pushes logs to Loki using a GRPC connection. > syslog and systemd input plugin have not been tested yet, feedback appreciated. @@ -29,7 +29,7 @@ You can use `Labels`, `RemoveKeys` , `LabelKeys` and `LabelMapPath` to how the o ### LabelMapPath -When using the `Parser` and `Filter` plugins fluent-bit can extract and add data to the current record/log data. While Loki labels are key value pair, record data can be nested structures. +When using the `Parser` and `Filter` plugins Fluent Bit can extract and add data to the current record/log data. While Loki labels are key value pair, record data can be nested structures. You can pass a json file that defines how to extract [labels](../../docs/overview/README.md#overview-of-loki) from each record. Each json key from the file will be matched with the log record to find label values. Values from the configuration are used as label names. Considering the record below : @@ -103,7 +103,7 @@ make fluent-bit-plugin ## Local -If you have fluent-bit installed in your `$PATH` you can run the plugin using: +If you have Fluent Bit installed in your `$PATH` you can run the plugin using: ```bash fluent-bit -e /path/to/built/out_loki.so -c fluent-bit.conf @@ -111,7 +111,7 @@ fluent-bit -e /path/to/built/out_loki.so -c fluent-bit.conf ## Docker -You can run a fluent-bit container with Loki output plugin pre-installed using our [docker hub](https://cloud.docker.com/u/grafana/repository/docker/grafana/fluent-bit-plugin-loki) image: +You can run a Fluent Bit container with Loki output plugin pre-installed using our [docker hub](https://cloud.docker.com/u/grafana/repository/docker/grafana/fluent-bit-plugin-loki) image: ```bash docker run -v /var/log:/var/log \ @@ -121,9 +121,9 @@ docker run -v /var/log:/var/log \ ## Kubernetes -You can run fluent-bit as a [Daemonset](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) to collect all your Kubernetes workload logs. +You can run Fluent Bit as a [Daemonset](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) to collect all your Kubernetes workload logs. -To do so you can use our [fluent-bit helm chart](../../production/helm/fluent-bit/README.md): +To do so you can use our [Fluent Bit helm chart](../../production/helm/fluent-bit/README.md): > Make sure [tiller](https://helm.sh/docs/install/) is installed correctly in your cluster @@ -136,7 +136,7 @@ helm upgrade --install fluent-bit loki/fluent-bit \ By default it will collect all containers logs and extract labels from Kubernetes API (`container_name`, `namespace`, etc..). -Alternatively you can install the Loki and fluent-bit all together using: +Alternatively you can install the Loki and Fluent Bit all together using: ```bash helm upgrade --install loki-stack loki/loki-stack \ diff --git a/docs/README.md b/docs/README.md index abc7db226494..9b799b0c96d8 100644 --- a/docs/README.md +++ b/docs/README.md @@ -35,7 +35,7 @@ simplifies the operation and significantly lowers the cost of Loki. 5. [Troubleshooting](clients/promtail/troubleshooting.md) 2. [Docker Driver](clients/docker-driver/README.md) 1. [Configuration](clients/docker-driver/configuration.md) - 4. [Fluent-bit](../cmd/fluent-bit/README.md) + 4. [Fluent Bit](../cmd/fluent-bit/README.md) 3. [Fluentd](clients/fluentd.md) 6. [LogQL](logql.md) 7. [Operations](operations/README.md) diff --git a/docs/clients/README.md b/docs/clients/README.md index 98764766baee..064453d4aaf8 100644 --- a/docs/clients/README.md +++ b/docs/clients/README.md @@ -4,7 +4,7 @@ Loki supports the following official clients for sending logs: 1. [Promtail](./promtail/README.md) 2. [Docker Driver](./docker-driver/README.md) -4. [Fluent-bit](../../cmd/fluent-bit/README.md) +4. [Fluent Bit](../../cmd/fluent-bit/README.md) 3. [Fluentd](./fluentd.md) ## Picking a Client @@ -12,6 +12,8 @@ Loki supports the following official clients for sending logs: While all clients can be used simultaneously to cover multiple use cases, which client is initially picked to send logs depends on your use case. +### Promtail + Promtail is the client of choice when you're running Kubernetes, as you can configure it to automatically scrape logs from pods running on the same node that Promtail runs on. Promtail and Prometheus running together in Kubernetes @@ -19,12 +21,23 @@ enables powerful debugging: if Prometheus and Promtail use the same labels, users can use tools like Grafana to switch between metrics and logs based on the label set. -Promtail is also the client of choice on bare-metal: since it can be configured -to tail logs from all files given a host path, it is the easiest way to send +Promtail is also the client of choice on bare-metal since it can be configured +to tail logs from all files given a host path. It is the easiest way to send logs to Loki from plain-text files (e.g., things that log to `/var/log/*.log`). -When using Docker and not Kubernetes, the Docker Logging driver should be used, -as it automatically adds labels appropriate to the running container. +Lastly, Promtail works well if you want to extract metrics from logs such as +counting the occurrences of a particular message. + +### Docker Logging Driver + +When using Docker and not Kubernetes, the Docker logging driver for Loki should +be used as it automatically adds labels appropriate to the running container. -The Fluentd and fluent-bit plugins are ideal when you already have Fluentd deployed +### Fluentd and Fluent Bit + +The Fluentd and Fluent Bit plugins are ideal when you already have Fluentd deployed and you already have configured `Parser` and `Filter` plugins. + +Fluentd also works well for extracting metrics from logs when using its +Prometheus plugin. + diff --git a/docs/installation/helm.md b/docs/installation/helm.md index 6c5eb52a25fe..5cf2e711ea83 100644 --- a/docs/installation/helm.md +++ b/docs/installation/helm.md @@ -42,7 +42,7 @@ $ helm upgrade --install loki loki/loki --set "key1=val1,key2=val2,..." $ helm upgrade --install loki loki/loki-stack --set grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false ``` -### Deploy Loki Stack (Loki, fluent-bit, Grafana, Prometheus) +### Deploy Loki Stack (Loki, Fluent Bit, Grafana, Prometheus) ```bash $ helm upgrade --install loki loki/loki-stack \ diff --git a/production/helm/README.md b/production/helm/README.md index 3cc7cc3173aa..92499f315278 100644 --- a/production/helm/README.md +++ b/production/helm/README.md @@ -48,14 +48,14 @@ $ helm upgrade --install loki loki/loki $ helm upgrade --install promtail loki/promtail --set "loki.serviceName=loki" ``` -## Deploy Loki and fluent-bit to your cluster +## Deploy Loki and Fluent Bit to your cluster ```bash $ helm upgrade --install loki loki/loki-stack \ --set fluent-bit.enabled=true,promtail.enabled=false ``` -## Deploy fluent-bit only +## Deploy Fluent Bit only ```bash $ helm upgrade --install fluent-bit loki/fluent-bit \ diff --git a/production/helm/fluent-bit/README.md b/production/helm/fluent-bit/README.md index c85935a9710d..a9bb505b3e71 100644 --- a/production/helm/fluent-bit/README.md +++ b/production/helm/fluent-bit/README.md @@ -1,6 +1,6 @@ -# Fluent-bit Loki chart +# Fluent Bit Loki chart -This chart install the fluent-bit application to ship logs to Loki. It defines daemonset on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager. +This chart install the Fluent Bit application to ship logs to Loki. It defines daemonset on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager. ## Installing the Chart @@ -13,7 +13,7 @@ helm upgrade --install my-release loki/fluent-bit \ --set loki.serviceName=loki.svc.cluster.local ``` -The command deploys fluent-bit on the Kubernetes cluster with the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation. +The command deploys Fluent Bit on the Kubernetes cluster with the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation. To configure the chart to send to [Grafana Cloud](https://grafana.com/products/cloud) use: @@ -34,7 +34,7 @@ helm upgrade --install my-release loki/fluent-bit \ The full list of available tags on [docker hub](https://cloud.docker.com/u/grafana/repository/docker/grafana/fluent-bit-plugin-loki). -Alternatively you can install the full [Loki stack](../loki-stack) (Loki + fluent-bit) using: +Alternatively you can install the full [Loki stack](../loki-stack) (Loki + Fluent Bit) using: ```bash helm upgrade --install my-release loki/loki-stack \ @@ -45,9 +45,9 @@ This will automatically configured the `loki.serviceName` configuration field to ## RBAC -By default, `rbac.create` is set to true. This enable RBAC support in fluent-bit and must be true if RBAC is enabled in your cluster. +By default, `rbac.create` is set to true. This enable RBAC support in Fluent Bit and must be true if RBAC is enabled in your cluster. -The chart will take care of creating the required service accounts and roles for fluent-bit. +The chart will take care of creating the required service accounts and roles for Fluent Bit. If you have RBAC disabled, or to put it another way, ABAC enabled, you should set this value to `false`. @@ -63,9 +63,9 @@ The command removes all the Kubernetes components associated with the chart and ## Configuration -The following tables lists the configurable parameters of the fluent-bit chart and their default values. +The following tables lists the configurable parameters of the Fluent Bit chart and their default values. -For more details, read the [fluent-bit documentation](../../../cmd/fluent-bit/README.md) +For more details, read the [Fluent Bit documentation](../../../cmd/fluent-bit/README.md) | Parameter | Description | Default | |--------------------------|----------------------------------------------------------------------------------------------------|----------------------------------| @@ -74,27 +74,27 @@ For more details, read the [fluent-bit documentation](../../../cmd/fluent-bit/RE | `loki.serviceScheme` | The scheme of the Loki service. | `http` | | `loki.user` | The http basic auth username to access the Loki service. | | | `loki.password` | The http basic auth password to access the Loki service. | | -| `config.port` | the fluent-bit port to listen. (This is mainly used to serve metrics) | `2020` | -| `config.loglevel` | the fluent-bit log level (debug,info,warn,error). | `warn` | +| `config.port` | the Fluent Bit port to listen. (This is mainly used to serve metrics) | `2020` | +| `config.loglevel` | the Fluent Bit log level (debug,info,warn,error). | `warn` | | `config.lineFormat` | The line format to use to send a record (json/key_value) | `json` | | `config.removeKeys` | The list of key to remove from each record | `[removeKeys,stream]` | | `config.labels` | A set of labels to send for every log | `'{job="fluent-bit"}'` | -| `config.labelMap` | Mapping of labels from a record. See [fluent-bit documentation](../../../cmd/fluent-bit/README.md) | | +| `config.labelMap` | Mapping of labels from a record. See [Fluent Bit documentation](../../../cmd/fluent-bit/README.md) | | | `affinity` | [affinity][affinity] settings for pod assignment | `{}` | | `annotations` | Annotations to add to Kubernetes resources. | `{}` | | `deploymentStrategy` | The deployment strategy to use with the daemonset | `RollingUpdate` | -| `image.repository` | The fluent-bit docker image repository | `grafana/fluent-bit-plugin-loki` | -| `image.tag` | The fluent-bit docker image tag | `0.1` | -| `image.pullPolicy` | The fluent-bit docker image pull policy | `IfNotPresent` | -| `nodeSelector` | fluent-bit [node labels][nodeSelector] for pod assignment | `{}` | -| `podLabels` | additional fluent-bit pod labels | `{}` | -| `podAnnotations` | additional fluent-bit pod annotations | `Prometheus discovery` | +| `image.repository` | The Fluent Bit docker image repository | `grafana/fluent-bit-plugin-loki` | +| `image.tag` | The Fluent Bit docker image tag | `0.1` | +| `image.pullPolicy` | The Fluent Bit docker image pull policy | `IfNotPresent` | +| `nodeSelector` | Fluent Bit [node labels][nodeSelector] for pod assignment | `{}` | +| `podLabels` | additional Fluent Bit pod labels | `{}` | +| `podAnnotations` | additional Fluent Bit pod annotations | `Prometheus discovery` | | `rbac.create` | Activate support for RBAC | `true` | | `resources` | Resource requests/limit | | | `tolerations` | [Toleration][toleration] labels for pod assignment | `no schedule on master nodes` | | `volumes` | [Volume]([volumes]) to mount | `host containers log` | | `volumeMounts` | Volume mount mapping | | -| `serviceMonitor.enabled` | Create a [Prometheus Operator](operator) serviceMonitor resource for fluent-bit | `false` | +| `serviceMonitor.enabled` | Create a [Prometheus Operator](operator) serviceMonitor resource for Fluent Bit | `false` | [toleration]: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/