Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document known Docker driver issues. #4190

Merged
merged 4 commits into from
Aug 23, 2021
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions docs/sources/clients/docker-driver/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,3 +57,38 @@ To cleanly uninstall the plugin, disable and remove it:
docker plugin disable loki --force
docker plugin rm loki
```

# Know Issues

The driver keeps all logs in memory and will drop log entries if Loki is not reachable and if the quantity of `max_retries` has been exceeded. To avoid the dropping of log entries, setting `max_retries` to zero allows unlimited retries; the drive will continue trying forever until Loki is again reachable. Trying forever may have undesired consequences, because the Docker daemon will wait for the Loki driver to process all logs of a container, until the container is removed. Thus, the Docker daemon might wait forever if the container is stuck.

This issue is avoided by using [Promtail](../promtail) with this configuration:

```yaml
server:
disable: true

positions:
filename: loki-positions.yml

clients:
- url: http://ip_or_hostname_where_Loki_run:3100/loki/api/v1/push
# If using basic auth, configures the username and password sent.
basic_auth:
# The username to use for basic auth
username: <string>
# The password to use for basic auth
password: <string>

scrape_configs:
- job_name: system
pipeline_stages:
- docker: {}
static_configs:
- labels:
job: docker
__path__: /var/lib/docker/containers/*/*-json.log
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: will it always be JSON logs? (just *-json.log bugs me, can it be just *.log instead?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. The Docker pipeline steps requires the JSON log driver which is the default.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool. Thanks for clarification! :)


```

This will enable Promtail to tail *all* Docker container logs and publish them to Loki.