Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(provider): LRUCache Layer #954

Open
wants to merge 54 commits into
base: main
Choose a base branch
from
Open

feat(provider): LRUCache Layer #954

wants to merge 54 commits into from

Conversation

yash-atreya
Copy link
Member

@yash-atreya yash-atreya commented Jun 21, 2024 •
edited
Loading

Motivation

Ref: #770

Solution

  • Adds a provider caching layer that dumps cache to fs.
  • Responses are saved as key-value pairs where key is the hash of the request and its params.

Overrides the following RPC methods

  • get_block_by_number
  • get_block_by_hash
  • get_block_receipts
  • get_proof
  • get_storage_at
  • get_code_at
  • get_transaction_count
  • get_logs
  • get_transaction_by_hash
  • get_raw_transaction_by_hash
  • get_transaction_receipt

PR Checklist

  • Added Tests
  • Added Documentation
  • Breaking changes

mattsse
mattsse requested changes Jun 22, 2024
View reviewed changes
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
mattsse
mattsse requested changes Jun 24, 2024
View reviewed changes
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
@yash-atreya yash-atreya mentioned this pull request Jun 27, 2024
Merged
10 tasks
@yash-atreya yash-atreya changed the title feat(providers): LRUCache Layer feat(transport): LRUCache Layer Aug 5, 2024
mattsse
mattsse requested changes Aug 5, 2024
View reviewed changes
Copy link
Member

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some suggestions

crates/transport/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/transport/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/transport/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/transport/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/transport/src/layers/cache.rs Outdated Show resolved Hide resolved
@yash-atreya yash-atreya marked this pull request as ready for review August 6, 2024 11:49
mattsse
mattsse reviewed Aug 6, 2024
View reviewed changes
Copy link
Member

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some questions

crates/transport/Cargo.toml Outdated Show resolved Hide resolved
crates/provider/src/provider/trait.rs Outdated Show resolved Hide resolved
crates/transport/src/layers/cache.rs Outdated Show resolved Hide resolved
@yash-atreya yash-atreya changed the title feat(transport): LRUCache Layer feat(provider): LRUCache Layer Sep 27, 2024
@yash-atreya yash-atreya self-assigned this Sep 30, 2024
mattsse
mattsse requested changes Oct 4, 2024
View reviewed changes
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs
) -> TransportResult<Option<Block>> {
let hash = RequestType::new("eth_getBlockByNumber", (number, hydrate));

cache_get_or_fetch!(self, hash, self.inner.get_block_by_number(number, hydrate))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is problematic because of reorgs, we'd need to track chain height somehow, but out of scope for this pr imo

crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
mattsse
mattsse requested changes Oct 4, 2024
View reviewed changes
Copy link
Member

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

almost there

few more suggestions

Cargo.toml Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
mattsse
mattsse requested changes Oct 5, 2024
View reviewed changes
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
crates/provider/src/layers/cache.rs Outdated Show resolved Hide resolved
@yash-atreya yash-atreya enabled auto-merge (squash) October 5, 2024 16:03
@yash-atreya
Copy link
Member Author

@mattsse addressed the suggestions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mattsse mattsse mattsse requested changes

@DaniPopes DaniPopes Awaiting requested review from DaniPopes DaniPopes is a code owner

@gakonst gakonst Awaiting requested review from gakonst gakonst is a code owner

@onbjerg onbjerg Awaiting requested review from onbjerg onbjerg is a code owner

@prestwich prestwich Awaiting requested review from prestwich prestwich is a code owner

@emhane emhane Awaiting requested review from emhane emhane is a code owner

Requested changes must be addressed to merge this pull request.

Assignees

@yash-atreya yash-atreya

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@yash-atreya @mattsse