Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tracking: katana metrics #1369

Closed
5 tasks done
gabe-ashe opened this issue Jan 8, 2024 · 1 comment
Closed
5 tasks done

tracking: katana metrics #1369

gabe-ashe opened this issue Jan 8, 2024 · 1 comment
Assignees
Labels
backlog enhancement New feature or request katana This issue is related to Katana tracking issue

Comments

@gabe-ashe
Copy link

gabe-ashe commented Jan 8, 2024

Add metrics throughout katana

Tasks

  1. katana
@glihm glihm changed the title [KATANA] Monitoring / Profiling - Prometheus monitoring endpoints [creat0xr] [KATANA] Monitoring / Profiling - Prometheus monitoring endpoints Mar 7, 2024
@kariy kariy added katana This issue is related to Katana enhancement New feature or request backlog labels Mar 27, 2024 — with Linear
kariy added a commit that referenced this issue Mar 29, 2024
ref #1369

- re-export [reth-metrics-derive](https://github.com/paradigmxyz/reth/tree/v0.2.0-beta.4/crates/metrics/metrics-derive) crate from dojo-metrics
- rename metrics -> dojo-metrics crate
- re-export core metrics stuff from dojo-metrics

---

this PR relies on #1720 bcs reth's MSRV is 1.76.0, otherwise can't build reth-metrics-derive.
@kariy kariy changed the title [KATANA] Monitoring / Profiling - Prometheus monitoring endpoints tracking: katana metrics Mar 29, 2024
kariy added a commit that referenced this issue Mar 30, 2024
ref #1369 

add metrics for rpc server connections and methods calls
@kariy kariy closed this as completed Apr 3, 2024
@kariy kariy reopened this Apr 8, 2024
kariy added a commit that referenced this issue Apr 12, 2024
ref #1369 

Add metrics on katana executor; tracking the total L1 **gas** and Cairo **steps** used.

There were two approaches that i thought of; 
1. record the metrics on every tx execution, or
2. on every block

~Decided to go with (1) as it would allow to measure it in realtime (as the tx is being executed), instead of having to wait until the block is finished being processed.~

Thought im not exactly sure which one is the ideal one.

Doing (1) might be less performant bcs we have to acquire the lock to the metrics recorder more frequently (ie every tx), as opposed to only updating the metrics once every block.

another thing to note, currently doing (1) would require all executor implementations to define the metrics in their own implmentations, meaning have to duplicate code. If do (2) can just define it under `block_producer` scope and be executor agnostic.

EDIT: doing (2). metrics are collected upon completion of block production

---

some changes are made to gather the value after block production:
- simplify params on `backend::do_mine_block`, now only accept two args; `BlockEnv` and `ExecutionOutput`
- add a new type `ExecutionStats` under `katana-executor`, this is where executor would store the gas and steps value
kariy added a commit that referenced this issue Apr 12, 2024
ref #1791 #1369

<img width="1420" alt="Screenshot 2024-04-12 at 2 36 56 AM" src="https://github.com/dojoengine/dojo/assets/26515232/b97b49df-c5fc-429a-9cb0-bf66138a00b6">

showing total gas and steps using a simple line charts, tracking its growth over time
matzayonc pushed a commit that referenced this issue Apr 12, 2024
ref #1369 

Add metrics on katana executor; tracking the total L1 **gas** and Cairo **steps** used.

There were two approaches that i thought of; 
1. record the metrics on every tx execution, or
2. on every block

~Decided to go with (1) as it would allow to measure it in realtime (as the tx is being executed), instead of having to wait until the block is finished being processed.~

Thought im not exactly sure which one is the ideal one.

Doing (1) might be less performant bcs we have to acquire the lock to the metrics recorder more frequently (ie every tx), as opposed to only updating the metrics once every block.

another thing to note, currently doing (1) would require all executor implementations to define the metrics in their own implmentations, meaning have to duplicate code. If do (2) can just define it under `block_producer` scope and be executor agnostic.

EDIT: doing (2). metrics are collected upon completion of block production

---

some changes are made to gather the value after block production:
- simplify params on `backend::do_mine_block`, now only accept two args; `BlockEnv` and `ExecutionOutput`
- add a new type `ExecutionStats` under `katana-executor`, this is where executor would store the gas and steps value
matzayonc pushed a commit that referenced this issue Apr 12, 2024
ref #1791 #1369

<img width="1420" alt="Screenshot 2024-04-12 at 2 36 56 AM" src="https://github.com/dojoengine/dojo/assets/26515232/b97b49df-c5fc-429a-9cb0-bf66138a00b6">

showing total gas and steps using a simple line charts, tracking its growth over time
matzayonc pushed a commit that referenced this issue Apr 14, 2024
ref #1369 

Add metrics on katana executor; tracking the total L1 **gas** and Cairo **steps** used.

There were two approaches that i thought of; 
1. record the metrics on every tx execution, or
2. on every block

~Decided to go with (1) as it would allow to measure it in realtime (as the tx is being executed), instead of having to wait until the block is finished being processed.~

Thought im not exactly sure which one is the ideal one.

Doing (1) might be less performant bcs we have to acquire the lock to the metrics recorder more frequently (ie every tx), as opposed to only updating the metrics once every block.

another thing to note, currently doing (1) would require all executor implementations to define the metrics in their own implmentations, meaning have to duplicate code. If do (2) can just define it under `block_producer` scope and be executor agnostic.

EDIT: doing (2). metrics are collected upon completion of block production

---

some changes are made to gather the value after block production:
- simplify params on `backend::do_mine_block`, now only accept two args; `BlockEnv` and `ExecutionOutput`
- add a new type `ExecutionStats` under `katana-executor`, this is where executor would store the gas and steps value
matzayonc pushed a commit that referenced this issue Apr 14, 2024
ref #1791 #1369

<img width="1420" alt="Screenshot 2024-04-12 at 2 36 56 AM" src="https://github.com/dojoengine/dojo/assets/26515232/b97b49df-c5fc-429a-9cb0-bf66138a00b6">

showing total gas and steps using a simple line charts, tracking its growth over time
@glihm
Copy link
Collaborator

glihm commented Apr 22, 2024

@kariy would consider this as done due to all PRs being merged, please don't hesitate to re-open as needed.

@glihm glihm closed this as completed Apr 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backlog enhancement New feature or request katana This issue is related to Katana tracking issue
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

3 participants