[sozo] Detect and manage manifests and artifacts discrepancies #1672
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
remybar
force-pushed
the
fix-detect_old_manifest_formats
branch
from
69af583
to
bb1a74f
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1672 +/- ##
==========================================
- Coverage 69.64% 69.46% -0.19%
==========================================
Files 268 269 +1
Lines 28642 28754 +112
==========================================
+ Hits 19948 19974 +26
- Misses 8694 8780 +86 ☔ View full report in Codecov by Sentry. |
remybar
changed the title
glihm
reviewed
Mar 15, 2024
remybar
force-pushed
the
fix-detect_old_manifest_formats
branch
from
bb1a74f
to
7ef516c
Compare
remybar
force-pushed
the
fix-detect_old_manifest_formats
branch
from
7ef516c
to
82d50cf
Compare
`sozo` uses several file types (previously generated by `sozo build` and `sozo migrate`) as input for migration. All of these files may be corrupt or may have discrepancies due to an older format version. This PR introduces a new `sozo clean` command to clean the project of these files. `sozo clean --manifest-abis` cleans only manifest and ABI files. `sozo clean --artifacts` cleans only artifacts. When an anomaly is detected in a file, the user is informed of the problem and advised to run `sozo clean` to clean up the project.
remybar
force-pushed
the
fix-detect_old_manifest_formats
branch
from
82d50cf
to
629199d
Compare
Sorry for that @remybar, if you didn't start, we will add the test in the rework of sozo crate once #1546 is merged. If you have started, let me know and I'll wait to merge. :) |
Yes, I'm on it but it should be done today. I keep you informed ;-) |
glihm
added a commit
that referenced
this pull request
Mar 25, 2024
* fix: handle primitive and custom types for enum introspect (#1643) The function `handle_introspect_enum` handles the derive Introspect attribute for an enum. At the moment, two cases are supported: tuples and type paths. A type path may be a primitive type or a custom type (struct or enum). A tuple may be composed of primitive and/or custom types. This PR updates the function `handle_introspect_enum` to handle all these cases properly. * fix: remove old reference to submodule (#1612) * fix: remove old reference to submodule * Update submodule * Will this now work? * Removed old files * Correctly updated gitmodules * chore: update forge std --------- Co-authored-by: glihm <dev@glihm.net> * fix(sozo): ensure warnings don't stop tests build (#1648) * fix: ensure warnings are not blocking the test build * fix: update default world address for spawn and move example * fix: [torii/graphql] Added timezone information to timestamps strings (#1657) * Check model exists on `sozo auth` (#1644) * feat: check model name and address before granting write access * Update crates/dojo-world/src/contracts/model.rs Co-authored-by: glihm <dev@glihm.net> * Update bin/sozo/src/ops/auth.rs Co-authored-by: glihm <dev@glihm.net> --------- Co-authored-by: glihm <dev@glihm.net> * fix: error on `world.model()` call gets mapped wrongly (#1661) * sozo: check existing class-hashes before registering model (#1576) * check existing class-hashes before registering model * update checking if models registered logic * apply review feedback * add initial test fixture for reregister * check if models need reregistering using remote manifest * test ouput of the reregister command * apply further review feedback * allow dead code for test utils * feat: add transaction_hash and block_number in the manifest file (#1651) * feat: add transaction_hash and block_number in the manifest file Store the `transaction_hash` and the `block_number` of the World deployment in the manifest file. * merge transaction_hash and block_number from previous deployed manifest if exists * update spawn-and-move example * fix fmt issues * refactor(sozo): make event parsing logic modular and reusable (#1556) * refacto: using cainome parser in order to parse events in sozo * refacto: clean logs + add tests * fix: clippy error * refacto: best error handling * update how felt are displayed + remove unnecessary expect in tests * fix tests * fix: fmt and artifacts parsing for missing events * fix: adjust event count as now world and base are included * return error from parse_and_print_events function * fix: ensure world address is always present to filter events --------- Co-authored-by: glihm <dev@glihm.net> * feat(sozo): add `--receipt` transaction option (#1647) * feat(katana): add simulate transactions (#1590) * add simulation to the RPC * simulate transaction on sequencer * add `new_query` * add fee type * lint * update to `simulate_transactions` * remove `simulation.rs` * individual parameters for `simulate_transactions` * lint rebase * update to fit spec * pin link * Prepare release: v0.6.0-alpha.6 (#1666) * feat(relay): persist and expose messages through grpc (#1526) * feat(libp2p): start working on persistence & grpc * feat: add relay proto and db migration * feat(libp2p): relaystorage for message persistence * feat(libp2p): relay subscription service * refactor: start implementing models/ty for libp2p * refactor(torii-core): generic set ty query * feat: use models for libp2p messages * feat: rpcs for subscring and retrieving messages * refactor(torii-core): arc rwlock for sql database & work on messages for models * refactor: tests & write message to db * refactor(grpc): grpc service & subscription * refactor: use entitiy type and models for messages * chore: fmt & clippy * refactor: unused async * chore: world descriptor * refactor: unused deps & fmt * chore: remove post types * chore: unused libp2p deps * chore: non wasm deps * chore * chore: fmt * fix: test * refactor: check if entity already exists for libp2p messages * chore: fmt & clippy * feat(libp2p): start working on identity and signatures for messages * refactor(libp2p): use array for signature type and check its ocmponents * refactor: decouple logic * refactor(libpo2p): extract identity & signature from validate message * feat(libp2p): eip 712 typed data * feat(libp2p): start workling on typed data structure snip-12 * refactor: change name * feat: handle type encoding * feat: typed data message encoding * refactor: rework in progress for using snip-12 * feat(libp2p): testing typed data with mocks & tests * fix: typed data passing test for enums * feat: test for presets and refactor type primitives * refactor: string parsing for felts * fix: dont include type hash on children objects * fix: json ordering issue with serde_json * fix: typed data for shortstring * fix: message encoding - passes all tests * feat: start supporting enums * feat: handle enum variant types * chore: rebase and fix db rwlock * refactor: encode all object types & start fixing enums * fix: enum test * fix: preset types. full compatibility with starknet js * refactor: more optimized approach for preset types * feat(libp2p): valid message signature & identity * refactor: wip identity and verifying message * refactor: get entity model identity & check * wip: start working on encoding typed data to Ty * refactor: implement ty types for encode typed data * refactor: parse object to ty function * refactor: set entity from relay w verification * feat: sign last signature and check signed last sig against last sig in db * chore: fmt * fix: torii grpc * refactor: sqlx onlyu server side * chore: clippy * chore: use char * refactor: sql reference * chore: fmt & clippy * refactor: typed data payload * fix: clippy * use RESOURCE_METADATA_MODEL constant * fix fmt * skip old ModelRegistered events * fix(torii): Added timezone informations to naive datetime strings (#1668) * fix(torii): Added timezone informations to naive datetimes * fix(torii_naive_datetimes): Quick parameters renaming * fix(torii_naive_datetimes): Used rust fmt script * fix(torii_naive_datetimes): Naive datetime refacto * fix(torii_naive_datetimes): Quick refacotring * fix(torii_naive_datetimes): Fixed cargo clippy error * fix(torii_naive_datetimes): Updated datetime retrieval * get version from dojo::model attribute * take version into account to generate selector and version functions * refactor: write manifest file even if migration failed (#1652) * refactor: write manifest file before trying to deploy them * fix lints * sort fs entries for deterministic order in manifest * write migration output for partial migrations * make suggested changes * feat(sozo): add seed to manifest (#1674) * feat(sozo): add seed to manifest * make suggested changes * fix(sozo): don't upload to ipfs if in offline mode (#1678) * Torii fix queries with keys regex (#1609) * [sozo] Detect and manage manifests and artifacts discrepancies (#1672) feat: detect and manage manifests and artifacts discrepancies `sozo` uses several file types (previously generated by `sozo build` and `sozo migrate`) as input for migration. All of these files may be corrupt or may have discrepancies due to an older format version. This PR introduces a new `sozo clean` command to clean the project of these files. `sozo clean --manifest-abis` cleans only manifest and ABI files. `sozo clean --artifacts` cleans only artifacts. When an anomaly is detected in a file, the user is informed of the problem and advised to run `sozo clean` to clean up the project. * Prepare release: v0.6.0-alpha.7 (#1680) Co-authored-by: glihm <glihm@users.noreply.github.com> * Add explorer flag to open World Explorer in browser (#1581) * Add explorer flag to open World Explorer * Log error message instead of unwrap when fail to open world explorer on browser * fix typo * fix typo 2 --------- Co-authored-by: glihm <dev@glihm.net> * after review * refactor(torii): retrieve events and store only relevant txns (#1631) * feat(torii): parallelize torii sync_range func * refactor: retrieve events & store only relevant txns * refactor: fix chunk size overflow & handle next events pages * chore: fmt * chore: clippy * chore: fmt * refactor: execute all txns at end of sync * chore: code * chroe: suggested changes * feat: configure events page chunk size * fmt * refactor: include chink size in config * fix * Split sozo ops into its own crate (#1546) * Split sozo ops into its own crate * fix: minor fixes and adjust commands documentation * fix: add missing file * fix: ensure migration failure is not silent --------- Co-authored-by: Jonatan Chaverri <jonatan.chaverri@kyndryl.com> Co-authored-by: glihm <dev@glihm.net> * fix base_test error * feat: add eventmessage for emitting models & start refactoring emit macro (#1656) * feat: add eventmessage for emitting models & start refactoring emit macro * refactor: emit multiple models event * feat: event message processor * feat: emit_message world func emit event mssage * refactor: catch all event processor * refactor: check model key as name * feat: model name keccak as id & rework model events * fix: storing entities with model hash * refactor: catch all event message and store * feat: emit model evrnt from spawn and move spawn * fix: pass keys as array * feat: fix emit macro and correctly index model events * feat: event messages migrations & set * feat: store events messages and new id system * chore: fmt * fix: keys array * feat: add grpc endpoint for event messages * feat: graphql schema for event messages * cadd comments for moel name hash * revert world.cario changes * fix: graphql entity and model connection * refactor: only test models name ordering * fix: entity/modeldata relation * refactor: remove event testing * fix: cairo code * refactor: add back event rxample * fix: merge * chore: migration * fix: subscription test * fix: tests * fix: subscription test * fix: sql test * chore: format model id correctly in test * fix: model subscription with id * fix: schema for model * chore: revert modelmmeber type * fix: event message query * fix: ensure sozo clean only affect base (#1685) The overlays are managed by the user. We don't want to clean them. The deployments are only output, and can't cause any conflict. For this reason, only base should be clean. * sozo: add dry-run mode (#1686) * fix: rebuild artifacts --------- Co-authored-by: RareSecond <jasper@codictive.be> Co-authored-by: glihm <dev@glihm.net> Co-authored-by: akhercha <akherchache@pm.me> Co-authored-by: Ammar Arif <evergreenkary@gmail.com> Co-authored-by: kwkr <kawka.maciej.93@gmail.com> Co-authored-by: Matthias <matthias.monnier@gmail.com> Co-authored-by: greged93 <82421016+greged93@users.noreply.github.com> Co-authored-by: Tarrence van As <tarrencev@users.noreply.github.com> Co-authored-by: Larko <59736843+Larkooo@users.noreply.github.com> Co-authored-by: lambda-0x <0xlambda@protonmail.com> Co-authored-by: Yun <broody@gmail.com> Co-authored-by: glihm <glihm@users.noreply.github.com> Co-authored-by: Junichi Sugiura <jun.sugiura.jp@gmail.com> Co-authored-by: Jonatan Chaverri <jonathan.chaverri12@gmail.com> Co-authored-by: Jonatan Chaverri <jonatan.chaverri@kyndryl.com>
This was referenced Mar 27, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related to DOJ-213, #1625.
sozouses several file types (previously generated bysozo buildandsozo migrate)as input for migration. All of these files may be corrupt or may have discrepancies due
to an older format version.
This PR introduces a new
sozo cleancommand to clean the project of these files.sozo clean --manifest-abiscleans only manifest and ABI files.sozo clean --artifactscleans only artifacts.When an anomaly is detected in a file, the user is informed of the problem
and advised to run
sozo cleanto clean up the project.