Skip to content
This repository has been archived by the owner on Apr 12, 2022. It is now read-only.

[doc] enhance ansible-beats doc #41

Merged
merged 1 commit into from
Sep 13, 2019
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
219 changes: 125 additions & 94 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,101 +1,64 @@
Beats Role
=========
# ansible-beats
[![Build Status](https://img.shields.io/jenkins/s/https/devops-ci.elastic.co/job/elastic+ansible-beats+master.svg)](https://devops-ci.elastic.co/job/elastic+ansible-beats+master/)
[![Ansible Galaxy](https://img.shields.io/badge/ansible--galaxy-elastic.beats-blue.svg)](https://galaxy.ansible.com/elastic/beats/)

This role provides a generic means of installing Elastic supported beats using Ansible. Currently this includes:
This role provides a generic means of installing Elastic supported Beats

- Filebeat
- MetricBeat (TopBeat in 1.x)
- Packetbeat
**Tested Beats**

Requirements
------------
* Filebeat
* MetricBeat (TopBeat in 1.x)
* Packetbeat

This role requires:
**Tested Versions**

- Ansible version > 2.0
- Beats version >= 1.x
- Target machines require access to an external repo, or means
* 7.x
* 6.x

The following OS are currently actively tested using kitchen:
**Tested Platforms**

- Centos 6 & 7
- Debian 8 & 9
- Ubuntu 14.04, 16.04 & 18.04
* Ubuntu 14.04
* Ubuntu 16.04
* Ubuntu 18.04
* Debian 8
* Debian 9
* CentOS 6
* CentOS 7

The above list is subject to change.
## Usage

Role Variables
--------------
Create your Ansible playbook with your own tasks, and include the role beats. You will have to have this repository accessible within the context of playbook.

Supported variables are as follows:
```sh
ansible-galaxy install elastic.beats,7.0.0
```

- **beat** (*MANDATORY*): Beat product. Supported values are: "filebeat", "metricbeat" & "packetbeat".
- **beat_conf** (*MANDATORY*): Beat Configuration. Should be defined as a map (see [Example Playbook](#example-playbook))
- **beats_version** (*Defaults to `7.0.0`*): Beats version.
- **version_lock** (*Defaults to `false`*): Locks the installed version if set to true, thus preventing other processes from updating. This will not impact the roles ability to update the beat on subsequent runs (it unlocks and re-locks if required).
- **use_repository** (*Defaults to `true`*): Use elastic repo for yum or apt if true. If false, a custom custom_package_url must be provided.
- **start_service** (*Defaults to `true`*): service will be started if true, false otherwise.
- **restart_on_change** (*Defaults to `true`*): Changes to configuration or installed versions, will result in a restart if true.
- **daemon_args** (*Applicable to version 1.x of beats*): Allows run time params to be passed to beats.
- **logging_conf** (*Defaults to `{"files":{"rotateeverybytes":10485760}}`*): Logging configuration. Should be defined as a map (see [Example Playbook](#example-playbook)). Map is serialized into logging section of beat config.
- **shipper_conf** (*Applicable to version 1.x of beats*): Shipper configuration. Should be defined as a map (see [Example Playbook](#example-playbook)). Map is serialized into shipper section of beat config.
- **output_conf** (*Defaults to `{"elasticsearch":{"hosts":["localhost:9200"]}}`): Output configuration. Should be defined as a map (see [Example Playbook](#example-playbook)). Map is serialized into output section of beat config.
- **beats_pid_dir** (*Defaults to `/var/run`*): Location of beats pid file.
- **beats_conf_dir** (*Defaults to `/etc/{beat}`*): Location of conf directory for beats configuration file.
Then create your playbook yaml adding the role beats.
The application of the beats role results in the installation of a node on a host.

The simplest configuration therefore consists of:

```yaml
hosts: localhost
roles:
- role: elastic.beats
vars:
beats_version: 7.0.0
beat: filebeat
beat_conf:
inputs:
- type: log
enabled: true
paths:
- /var/log/*.log
```

Dependencies
------------

- Ansible version > 2.0
- Beats version >= 6.x

Example Playbook
----------------

Example playbook is provided below. This installs Packetbeat and illustrates the need for configuration sections to be specified as maps.

- name: Example playbook for installing packetbeat
hosts: localhost
roles:
- { role: beats, beat: "packetbeat",
beat_conf: {
"interfaces": {"device":"any"},
"protocols": {
"dns": {
"ports": [53],
"include_authorities":true
},
"http": {
"ports": [80, 8080, 8000, 5000, 8002]
},
"memcache": {
"ports": [11211]
},
"mysql": {
"ports": [3306]
},
"pgsql": {
"ports": [5432]
},
"redis": {
"ports": [6379]
},
"thrift": {
"ports": [9090]
},
"mongodb": {
"ports": [27017]
}
}
},
output_conf : {
"elasticsearch": {
"hosts": ["localhost:9200"]
}
}
}
vars:
use_repository: "true"
The above installs Filebeat 7.0.0 on the hosts 'localhost'.

**Notes**:
- Beats default version is described in [`beats_version`](defaults/main.yml#L4). You can override this variable in your playbook to install another version.
While we are testing this role only with one 7.x and one 6.x version (respectively [7.0.0](defaults/main.yml#L4) and [6.7.1](test/integration/standard-6x.yml#L7) at the time of writing), this role should work with others version also in most cases.
- Beat product is described in `beat` variable. While currently tested Beats are Filebeat, Metricbeat & Packetbeat, this role should work also with other member of [The Beats Family](https://www.elastic.co/products/beats) in most cases.

## Testing

Expand Down Expand Up @@ -140,20 +103,88 @@ When you are finished testing you can clean up everything with
$ make destroy-all
```

License
-------
### Basic Beats configuration

All Betas configuration parameters are supported. This is achieved using a configuration map parameter `beat_conf` which is serialized into the `${beat}.yml` file.
The use of a map ensures the Ansible playbook does not need to be updated to reflect new/deprecated/plugin configuration parameters.

In addition to the `beat_conf` map, several other parameters are supported for additional functions e.g. script installation. These can be found in the role's `defaults/main.yml` file.

The following illustrates applying configuration parameters to Packetbeat instance.

```yaml
- name: Example playbook for installing packetbeat
hosts: localhost
roles:
- { role: beats, beat: "packetbeat",
beat_conf: {
"interfaces": {"device":"any"},
"protocols": {
"dns": {
"ports": [53],
"include_authorities":true
},
"http": {
"ports": [80, 8080, 8000, 5000, 8002]
},
"memcache": {
"ports": [11211]
},
"mysql": {
"ports": [3306]
},
"pgsql": {
"ports": [5432]
},
"redis": {
"ports": [6379]
},
"thrift": {
"ports": [9090]
},
"mongodb": {
"ports": [27017]
}
}
},
output_conf : {
"elasticsearch": {
"hosts": ["localhost:9200"]
}
}
}
vars:
use_repository: "true"
```

### Additional Configuration

Supported variables are as follows:

- **beat** (*MANDATORY*): Beat product. Supported values are: "filebeat", "metricbeat" & "packetbeat" (others beats from [The Beats Family](https://www.elastic.co/products/beats) should work in most cases but aren't currently tested).
- **beat_conf** (*MANDATORY*): Beat Configuration. Should be defined as a map.
- **beats_version** (*Defaults to `7.0.0`*): Beats version.
- **version_lock** (*Defaults to `false`*): Locks the installed version if set to true, thus preventing other processes from updating. This will not impact the roles ability to update the beat on subsequent runs (it unlocks and re-locks if required).
- **use_repository** (*Defaults to `true`*): Use elastic repo for yum or apt if true. If false, a custom custom_package_url must be provided.
- **start_service** (*Defaults to `true`*): service will be started if true, false otherwise.
- **restart_on_change** (*Defaults to `true`*): Changes to configuration or installed versions, will result in a restart if true.
- **daemon_args** (*Applicable to version 1.x of beats*): Allows run time params to be passed to beats.
- **logging_conf** (*Defaults to `{"files":{"rotateeverybytes":10485760}}`*): Logging configuration. Should be defined as a map. Map is serialized into logging section of beat config.
- **shipper_conf** (*Applicable to version 1.x of beats*): Shipper configuration. Should be defined as a map . Map is serialized into shipper section of beat config.
- **output_conf** (*Defaults to `{"elasticsearch":{"hosts":["localhost:9200"]}}`*): Output configuration. Map is serialized into output section of beat config.
- **beats_pid_dir** (*Defaults to `/var/run`*): Location of beats pid file.
- **beats_conf_dir** (*Defaults to `/etc/{beat}`*): Location of conf directory for beats configuration file.

## License

Apache 2.0

Limitations
------------
## Limitations

Multiple instances of the same beat cannot be installed on the same target server.

Author Information
------------------
## Questions on Usage

Dale McDiarmid
For further assistance, outside of issues, please use discuss.elastic.co.
We welcome questions on how to use the role. However, in order to keep the GitHub issues list focused on "issues" we ask the community to raise questions at https://discuss.elastic.co/c/beats. This is monitored by the maintainers.

Community Contributions always appreciated and welcome! Please ensure all contributions include tests as appropriate.