Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modularize beacon processor scheduler #6448

Open
wants to merge 10 commits into
base: unstable
Choose a base branch
from

Conversation

eserilev
Copy link
Collaborator

Issue Addressed

Partially #6291

Proposed Changes

This PR aims to "modularize" the beacon processor scheduler. The goal is to set up the scheduler in such a way that new scheduling algorithms can be implemented/swapped in.

For now we've decided on exposing an interface that implements the following trait

pub trait Scheduler<E: EthSpec, S: SlotClock> {
    fn new(
        beacon_processor: BeaconProcessor<E>,
        beacon_state: &BeaconState<E>,
        event_rx: mpsc::Receiver<WorkEvent<E>>,
        spec: &ChainSpec,
    ) -> Result<Box<Self>, String>;

    fn run(
        self,
        work_journal_tx: Option<mpsc::Sender<&'static str>>,
        slot_clock: S,
        maximum_gossip_clock_disparity: Duration,
    ) -> Result<(), String>;
}

We then define a SchedulerType enum

pub enum SchedulerType<E: EthSpec, S: SlotClock> {
    PriorityScheduler(priority_scheduler::Scheduler<E, S>),
}

Each variant is a beacon processor scheduling implementation with its own unique scheduling algorithm. Swapping between the variants can be done either at the config level, or with a small code change.

Additional Info

  • The concept of work queues is currently specific to the PriorityScheduler variant. Other scheduling algorithms might not use all these different queues to schedule the different WorkEvents and so these queues aren't exposed outside of the PriorityScheduler impl.
  • The reprocess event channel is no longer externally exposed. All work events are now sent through the single BeaconProcessorSend channel. I've introduced a new Work::Reprocess type which we then use to schedule jobs for reprocess. The reprocess logic itself is very specific to the PriorityScheduler so it seems to make more sense to give other implementations maximum flexibility.
  • There may be some ways to generalize this even further so that different scheduling implementations aren't duplicating similar code/logic... I think we'll find out more once we start actually implementing a new scheduling algorithm

@eserilev eserilev added code-quality work-in-progress PR is a work-in-progress labels Sep 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
code-quality work-in-progress PR is a work-in-progress
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant