Fixing race condition for analysis #906
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
dpaiton
approved these changes
Sep 26, 2023
Merged
slundqui
added a commit
that referenced
this pull request
Sep 30, 2023
Previously, we look at the minimum block number of transactions, wallet deltas, and pool info to see the last block updated. However, there's a case where a large number of blocks tick up without any transactions. In this case, the next transaction will result in analysis attempting to process a large number of blocks, which breaks the analysis pipeline. This PR resolves this issue, along with the race condition solved in #906 by ensuring we write pool_info as the last item, and using pool_info as the table to see how far along the data process is. This also deprecates all current wallet info tables from the db in favor of using wallet deltas, as current wallet info needed pool_info data for its calculations.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Analysis script had a race condition with the data collection script. Specifically, if the analysis script queried data from a block after pool_info was written, but before transactions and wallet deltas was written, the analysis script would potentially miss transactions.
Also adding in helper function useful for debugging to gather and filter hyperdrive events directly from the chain.