Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consolidate AWS credentials, Cloudwatch dimension wilcards #1155

Closed
wants to merge 6 commits into from

Conversation

johnrengelman
Copy link
Contributor

Wasn't sure if you'd be ok with trying to squeeze this into 0.13 or not (I can retarget it to master if you'd like).

This PR does 3 things:

  1. Consolidates how AWS credentials are configured across all inputs/outputs
  2. Adds support for configuration AWS credentials via shared credential file and profile and via assuming a role through STS
  3. Adds support to the cloudwatch input to specify a wildcard for the dimension value allowing for ingesting all metrics that have that dimension regardless of value.

Closes #1115, #1101

Region string `toml:"region"` // AWS Region
AccessKey string `toml:"access_key"` // Explicit AWS Access Key ID
SecretKey string `toml:"secret_key"` // Explicit AWS Secret Access Key
RoleArn string `toml:"role_arn"` // Role ARN to assume
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These toml tags aren't actually supported because they are on an embedded, anonymous struct.
I've created a PR to add that support here - influxdata/toml#6

@sparrc
Copy link
Contributor

sparrc commented May 5, 2016

Thanks @johnrengelman, probably won't be able to squeeze this into 0.13 as I'm OOO for the next few days, @ljosa & @joshglympse if you have time to review I'd appreciate it a lot!

Period internal.Duration `toml:"period"`
Delay internal.Duration `toml:"delay"`
Namespace string `toml:"namespace"`
Metrics []*Metric `toml:"metrics"`
CacheTTL string `toml:"cache_ttl"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use an internal.Duration for this to handle the duration string parsing?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, it looks like you made this a required config value - can you add to the sample config and README?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll switch it to Duration.

@@ -153,22 +173,19 @@ func (c *CloudWatch) Gather(acc telegraf.Accumulator) error {

func init() {
inputs.Add("cloudwatch", func() telegraf.Input {
return &CloudWatch{}
return &CloudWatch{
CacheTTL: "1hr",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joshhardy - this configures the default value to what it was before.
It's not required in the config file, if unspecified, it will be this.

@sparrc
Copy link
Contributor

sparrc commented May 6, 2016

@johnrengelman seems like you could accomplish this without embedded fields, which are confusing to me, because it requires guessing where the fields in the config file are going and then digging into different dependencies trying to find them.

You could design it by putting each of those fields on the different AWS plugins, and then instantiating an influxaws credential object using those. This is more explicit and I think easier for future debugging.

## 3) shared profile from 'profile'
## 4) environment variables
## 5) shared credentials file
## 6) EC2 Instance Profile
Copy link
Contributor

@sparrc sparrc May 6, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since we have example fields for the keys, should you put

#role_arn = ""

here?

@johnrengelman
Copy link
Contributor Author

@sparrc that's a good suggestion (still learning the best practices with Go). I'll refactor this PR to do that.

@johnrengelman
Copy link
Contributor Author

Closing this PR now that 0.13 is release and reopened against master: #1208

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants