Topdown Refactoring Plan

[cryptoAtwill]

Based on Topdown V2 spec, the skeleton code for new topdown refactoring is shown below:

struct TopdownReactor {
    event_bus;                 // the topdown event broadcaster for subscribers 
    topdown_rx;                // the topdown reactor thread request receiver, handling communication and requests from other threads
    parent_polling_interval;   // the tokio timer that wakes the polling
}

impl TopdownReactor {
    fn run(self) {

        loop {

            // first checks if there are requests for topdown syncer, such as
            // rpc requests for latest parent view, the parent chain head and etc
            // handle them quickly.
            // Might cap to N requests per loop, dont want to spend all loop handling requests
            while let Some(req) = self.topdown_rx.await {
                self.handle_topdown_req(req);
            }

            match self.poll_next_parent_view().await {
                Res::Ok(parent_view) => {
                    self.store_parent_view(parent_view);
                    if let Some(proposal) = self.deduce_topdown_proposal(parent_view.parent_height) {
                        self.event_bus.publish(NewProposal{ proposal })
                    }
                },
                Res::ParentReverted(parent_epoch) => {
                    self.event_bus.publish(ParentReverted{ parent_epoch })
                    self.pause();
                },
                Res::NoNewView => {
                    sleep(parent_polling_interval).await
                }
            }
        }
    }
}

struct VoteTallyReactor {
    vote_tally_rx;                // handles reqeust to vote tally reactor
    topdown_polling_subscriber;   // subscribing to topdown events
    gossip_ch;                    // the gossip pub/sub for topdown voting
}

impl VoteTallyReactor {
    fn run(self) {
        // the current operation mode of the vote tally
        let operation_mode = OperationMode::Active;

        loop {

            // this handles requests coming from other threads, such as 
            // checking if any quorum reached from interpreter
            while let Some(req) = self.vote_tally_rx.try_recv().await {
                self.handle_req(req)
            }

            // cap to processing N non-seen votes per loop
            while let Some(vote) = self.gossip_ch.try_recv().await {
                self.record_vote(vote);
            }

            while let Some(event) = self.topdown_polling_subscriber.try_recv().await {
                match event {
                    topdown::NewProposal(proposal) => self.broadcast_new_vote(proposal).await,
                    topdown::ParentReverted => {
                        self.pause(),
                    }
                }
            }

            let operation_mode = operation_mode.advance(self);
            operation_mode.process().await;
        }
    }
}

Tasks breakdown:

• Vote tally migrate to reactor model (5 days)
  • Remove stm and setup reactor + channels
  • Interpreter using certificates that contains parent block hash + side effect
  • Use cumulative hash
• Topdown syncer migrate to reactor model + simplification (3 days)
• Topdown parent view persistence (1 day)
• Vote tally vote persistence (1 day)
• Vote tally Soft/Hard recovery mode

[cryptoAtwill]

@raulk The open issues are:

• The detail cumulative hash approach. But this can be relatively straight forward, such as next_hash = hash( hash(proposal) || previous_hash).
• The detail operating mode in soft/hard recovery and while in these modes, how do we handling in coming vote gossips.