IDONTWANT message optimisation to cutoff for smaller messages #6456

hopinheimer · 2024-10-02T18:03:24Z

addresses #6437

AgeManning

Nice, thanks for the help.

AgeManning · 2024-10-03T01:02:16Z

beacon_node/lighthouse_network/gossipsub/src/behaviour.rs

@@ -1828,8 +1825,12 @@ where
                peer_score.duplicated_message(propagation_source, &msg_id, &message.topic);
            }
            self.mcache.observe_duplicate(&msg_id, propagation_source);
+
+            // Broadcast IDONTWANT messages.
+            self.send_idontwant(&raw_message, &msg_id, propagation_source);


I think we want this after the return, after the if statement.

At the moment, this will broadcast an IDONTWANT only if the message is a duplicate.

We want the logic to broadcast only if the message isn't a duplicate.

oh yeah makes sense!

AgeManning · 2024-10-03T01:06:33Z

beacon_node/lighthouse_network/gossipsub/src/config.rs

+    // traffic and CPU load. This acts as a lower bound cutoff for the message size to which 
+    // IDONTWANT won't be sent to peers. Only works if the peers support Gossipsub1.3 
+    // (see https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.2.md#idontwant-message)
+    // default is 1kb


Might be worth mentioning that the unit for the usize value is bytes.

I always mess this up. resolved!

jxs

Thanks! You need to cargo fmt to pass the check code CI job. The tests also need to be updated, and we should probably add another test for this case, i.e. to not send IDONTWANT when the size is inferior to the limit

jxs · 2024-10-04T14:17:46Z

beacon_node/lighthouse_network/gossipsub/src/behaviour.rs

@@ -2699,6 +2701,10 @@ where
            return;
        };

+        if message.raw_protobuf_len() < self.config.idontwant_message_size_threshold(){


can we move this check to line 1833? So that it's explicit that we are only calling send_idontwant if the message size is inferior to the threshold instead of having the function decide that.

also the the logic was a little off. I have changed it and written passing test for the same

jxs · 2024-10-04T14:40:15Z

beacon_node/lighthouse_network/gossipsub/src/config.rs

+    // Controls the message size threshold for which IDONTWANT messages are sent.
+    // Sending IDONTWANT messages for small messages can have a negative effect to the overall
+    // traffic and CPU load. This acts as a lower bound cutoff for the message size to which 
+    // IDONTWANT won't be sent to peers. Only works if the peers support Gossipsub1.3 
+    // (see https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.2.md#idontwant-message)
+    // default is 1kB


Suggested change

// Controls the message size threshold for which IDONTWANT messages are sent.

// Sending IDONTWANT messages for small messages can have a negative effect to the overall

// traffic and CPU load. This acts as a lower bound cutoff for the message size to which

// IDONTWANT won't be sent to peers. Only works if the peers support Gossipsub1.3

// (see https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.2.md#idontwant-message)

// default is 1kB

// The size threshold for which IDONTWANT messages are sent.

// Sending IDONTWANT messages for small messages can have a negative effect to the overall

// traffic and CPU load. This acts as a lower bound cutoff for the message size to which

// IDONTWANT won't be sent to peers. Only works if the peers support Gossipsub1.2

// (see https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.2.md#idontwant-message)

// default is 1kB

jxs · 2024-10-04T14:41:10Z

beacon_node/lighthouse_network/gossipsub/src/config.rs

+    // Controls the message size threshold for which IDONTWANT messages are sent.
+    // Sending IDONTWANT messages for small messages can have a negative effect to the overall
+    // traffic and CPU load. This acts as a lower bound cutoff for the message size to which 
+    // IDONTWANT won't be sent to peers. Only works if the peers support Gossipsub1.3 
+    // (see https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.2.md#idontwant-message)
+    // default is 1kB


Suggested change

// Controls the message size threshold for which IDONTWANT messages are sent.

// Sending IDONTWANT messages for small messages can have a negative effect to the overall

// traffic and CPU load. This acts as a lower bound cutoff for the message size to which

// IDONTWANT won't be sent to peers. Only works if the peers support Gossipsub1.3

// (see https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.2.md#idontwant-message)

// default is 1kB

// The size threshold for which IDONTWANT messages are sent.

// Sending IDONTWANT messages for small messages can have a negative effect to the overall

// traffic and CPU load. This acts as a lower bound cutoff for the message size to which

// IDONTWANT won't be sent to peers. Only works if the peers support Gossipsub1.2

// (see https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.2.md#idontwant-message)

// default is 1kB

pawanjay176 · 2024-10-04T18:47:06Z

beacon_node/lighthouse_network/gossipsub/src/config.rs

+    // IDONTWANT won't be sent to peers. Only works if the peers support Gossipsub1.3 
+    // (see https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.2.md#idontwant-message)
+    // default is 1kB
+    pub fn idontwant_message_size_threshold(&self) -> usize{


We would need to upstream this builder config to the lighthouse config as well https://github.com/sigp/lighthouse/blob/unstable/beacon_node/lighthouse_network/src/config.rs#L480-L502

I would also suggest adding a lighthouse cli flag to control the limit so we can test it for other feature devnets (e.g. peerdas)

understood will make the requested changes and add it to a cli flag

hopinheimer · 2024-10-08T05:46:03Z

waiting on #6436 to build the cli docs. I use a mac

idontwant message opitmising

6d28509

hopinheimer changed the title ~~idontwant message optimisation~~ IDONTWANT message optimisation to cutoff for smaller messages Oct 2, 2024

jimmygchen requested review from AgeManning and jxs and removed request for AgeManning October 3, 2024 01:03

jimmygchen added the Networking label Oct 3, 2024

jimmygchen requested a review from AgeManning October 3, 2024 01:03

jimmygchen added the ready-for-review The code is ready for review label Oct 3, 2024

AgeManning reviewed Oct 3, 2024

View reviewed changes

requested changes and linter appeasing

205078a

hopinheimer requested a review from AgeManning October 3, 2024 05:06

jxs reviewed Oct 4, 2024

View reviewed changes

pawanjay176 reviewed Oct 4, 2024

View reviewed changes

added the config cli flag

b59aee0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IDONTWANT message optimisation to cutoff for smaller messages #6456

IDONTWANT message optimisation to cutoff for smaller messages #6456

hopinheimer commented Oct 2, 2024

AgeManning left a comment

AgeManning Oct 3, 2024

hopinheimer Oct 3, 2024

AgeManning Oct 3, 2024

hopinheimer Oct 3, 2024

jxs left a comment

jxs Oct 4, 2024

hopinheimer Oct 5, 2024

jxs Oct 4, 2024

jxs Oct 4, 2024

pawanjay176 Oct 4, 2024

hopinheimer Oct 5, 2024

hopinheimer commented Oct 8, 2024

IDONTWANT message optimisation to cutoff for smaller messages #6456

Are you sure you want to change the base?

IDONTWANT message optimisation to cutoff for smaller messages #6456

Conversation

hopinheimer commented Oct 2, 2024

AgeManning left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jxs left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hopinheimer commented Oct 8, 2024