Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TW review #41

Merged
merged 2 commits into from
Feb 16, 2024
Merged

TW review #41

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9a955f2
TW review
tothmano Feb 16, 2024
2112c14
Update list numbering
tothmano Feb 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 18 additions & 18 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,28 +1,28 @@
# Axiom CloudWatch Lambda [![CI](https://github.com/axiomhq/axiom-cloudwatch-lambda/actions/workflows/ci.yaml/badge.svg)](https://github.com/axiomhq/axiom-cloudwatch-lambda/actions/workflows/ci.yaml)

Easy to use AWS CloudFormation template to deploy a CloudWatch Log Group subscription filter and a Lambda to push the
logs from your CloudWatch to [Axiom](https://axiom.co).
Axiom CloudWatch Lambda is an easy-to-use AWS CloudFormation template to send logs from CloudWatch to [Axiom](https://axiom.co). It deploys a CloudWatch log group subscription filter and a Lambda.

Axiom CloudWatch Lambda uses the following CloudFormation stacks:

Axiom’s CloudWatch Lambda is deployed using three distinct CloudFormation stacks. These stacks are responsible for the following:

1. Axiom Ingester: This stack creates a Lambda function that ingests logs from CloudWatch and sends them to Axiom.
2. Backfiller: This stack runs once to create subscription filters on the ingest Lambda, for all existing CloudWatch log groups.
3. LogsSubscriber: This stack creates a Lambda function that listens for new log groups and creates subscription filters for them. This way you don't have to create subscription filters manually for new log groups.
- Axiom Ingester creates a Lambda function that ingests logs from CloudWatch and sends them to Axiom.
- Backfiller runs once to create subscription filters on the ingest Lambda for all existing CloudWatch log groups.
- Logs Subscriber creates a Lambda function that listens for new log groups and creates subscription filters for them. This way, you don't have to create subscription filters manually for new log groups.

## Guide

1. Create an account at [Axiom](https://app.axiom.co)
2. Create a dataset and an API token with ingest permission for that dataset
3. Launch the stack: [![Launch Stack](https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png)](https://console.aws.amazon.com/cloudformation/home?#/stacks/new?stackName=CloudWatch-Axiom&templateURL=https://axiom-cloudformation-stacks.s3.amazonaws.com/axiom-cloudwatch-lambda-cloudformation-stack.yaml)
4. Automatically subscribe to all existing log groups: [![Launch Stack](https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png)](https://console.aws.amazon.com/cloudformation/home?#/stacks/new?stackName=CloudWatch-Backfiller-Axiom&templateURL=https://axiom-cloudformation-stacks.s3.amazonaws.com/axiom-cloudwatch-backfiller-lambda-cloudformation-stack.yaml)
5. Automatically Subscribe to new log groups: [![Launch Stack](https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png)](https://console.aws.amazon.com/cloudformation/home?#/stacks/new?stackName=Axiom-CloudWatch-LogsSubscriber&templateURL=https://axiom-cloudformation-stacks.s3.amazonaws.com/axiom-cloudwatch-logs-subscriber-cloudformation-stack.yaml)
1. [Create an Axiom account](https://app.axiom.co).
2. Create a dataset in Axiom.
3. Create an API token in Axiom with permissions to ingest data to the dataset you created.
4. [Click this link to launch the Stack](https://console.aws.amazon.com/cloudformation/home?#/stacks/new?stackName=CloudWatch-Axiom&templateURL=https://axiom-cloudformation-stacks.s3.amazonaws.com/axiom-cloudwatch-lambda-cloudformation-stack.yaml).
5. [Click this link to automatically subscribe to all existing log groups](https://console.aws.amazon.com/cloudformation/home?#/stacks/new?stackName=CloudWatch-Backfiller-Axiom&templateURL=https://axiom-cloudformation-stacks.s3.amazonaws.com/axiom-cloudwatch-backfiller-lambda-cloudformation-stack.yaml).
6. [Click this link to automatically subscribe to new log groups](https://console.aws.amazon.com/cloudformation/home?#/stacks/new?stackName=Axiom-CloudWatch-LogsSubscriber&templateURL=https://axiom-cloudformation-stacks.s3.amazonaws.com/axiom-cloudwatch-logs-subscriber-cloudformation-stack.yaml).

## Logs Subscriber architecture

# Logs Subscriber architecture
The Logs Subscriber stack does the following:

- Creates an S3 bucket for Cloudtrail
- Creates a Trail to capture creation of new log groups
- Creates an Event Rule to pass those creation events to event bus
- EventBridge sends an event to a Lambda function when a new log group is created
- Lambda function creates a subscription filter for the new log group
- It creates an S3 bucket for Cloudtrail.
- It creates a trail to capture the creation of new log groups.
- It creates an event rule to pass those creation events to an EventBridge event bus.
- The EventBridge sends an event to a Lambda function when a new log group is created.
- The Lambda function creates a subscription filter for the new log group.
Loading