ibc: can't find consensus state for counterparty client #2612
Comments
|
We discussed this ticket last week and decided that it should come after higher priority work on completing the |
|
Fired up the tooling again today, and received a different error from which points to penumbra/crates/core/component/ibc/src/component/client_counter.rs Lines 235 to 240 in f79b25d The error from relayer remains unchanged: Let's see if we can reproduce that error with further testing today. |
|
Additionally logging output shows that the proof specs are indeed dissimilar: Specifically, the proof spec from the const in our Rust code does have |
So the go code is hardcoding a shape to the proofspec that isn't compatible with our most recent changes. To be compatible, we need the relayer code to increment the version of its ics23 dependency to at least |
|
Re-running the test based on the updated golang code, I no longer see the mismatched proofs, which is fantastic. However, the relayer ultimately fails with the message in the OP, with So, one problem down, at least one more to go! |
|
We have a hunch about another off-by-one error here, specifically on the height between chains. Likely the patch to resolve will be in the relayer code. If it's in the Rust code, it'd be in the query sections, not in the state machine. @avahowell is currently adding more logging messages to try to isolate the specific conditions that cause this problem. We believe it occurs but more detailed logging would be awesome for a lot of reasons. As for whether this will make it into 56, 🤞 |
We encountered a long (~20s) timeout on all `rly` commands, and traced it to a context timeout on a bad URL for the bablyon testchain. We're still focusing on preview <-> devnet functionality, so let's exclude the external testnet chains from relayer configuration until we're ready. Also includes some documentation updates so it's a bit easier to understand how test the relayer interactively. Eventually we'll want all this logic bottled up in integration tests, e.g. `interchaintest`, but still working toward that goal. Refs #2612.
|
Success! We've re-establishing working clients, channels, and connections. The changes required were:
After those changes landed, we were able to get a working connection between a local devnet and preview. There are some small docs tweaks in #2804. Next week, we'll work on manual testing of packet transfers between the two chains. Once we have that working, we'll evaluate running a relayer instance between preview & testnet (which mostly depends on whether further code changes are needed in the Penumbra repo, in which case the target will be 57). |
We encountered a long (~20s) timeout on all `rly` commands, and traced it to a context timeout on a bad URL for the bablyon testchain. We're still focusing on preview <-> devnet functionality, so let's exclude the external testnet chains from relayer configuration until we're ready. Also includes some documentation updates so it's a bit easier to understand how test the relayer interactively. Eventually we'll want all this logic bottled up in integration tests, e.g. `interchaintest`, but still working toward that goal. Refs #2612.
While trying to re-enable the relayer, I encountered some trouble getting relaying to work. As we saw with #2472, testing against a clean devnet and a fresh preview environment shows no problems. The problem seems to occur when previously configured relayer client state exists on at least one of the chains. Detailed STR below.
Steps to reproduce
You should see a successful run, ending in:
At that point, the following queries will all report clean results:
pcli -n http://localhost:8080 q ibc client 07-tendermint-0pcli -n http://localhost:8080 q ibc connection connection-0pcli -n http://localhost:8080 q ibc channel transfer channel-0You can try rerunning
build-pathand it should continue to succeed. Now, let's try reproducing the error. Cancel (i.e.ctrl+c) the devnet script in the first terminal session, and rerun those setup steps. This will create a new local devnet, but against the same preview instance. (N.B. if preview has changed out from under you while you were following these steps, you'll have to reset and go from the top again.)Then, when you rerun the
build-pathscript, you'll see an error:Inspecting the pd logs for the local devnet, I see:
Judging by the "counterparty consensus state not found for client 07-tendermint-1" message, we should check whether that client exists on the counterparty chain. Indeed, it does: running
pcli -n https://grpc.testnet-preview.penumbra.zone q ibc client 07-tendermint-1shows that client exists and appears to be configured well.The text was updated successfully, but these errors were encountered: