P2P Protocol Messages

Fantom p2p message exchange

This page describes the protocol interactions of a Fantom node.

Introduction

Fantom P2P is built on top of the p2p server and message passing of go-ethereum's devp2p protocol as the communication layer. Messages are RLP encoded. There are some significant differences compared to the original protocol.

Fantom basically instantiates the p2p server and the rpc server. It then creates a service, which is used to register its protocols (implementations of []p2p.Protocol) with the node stack.

The fantom code is notified by the node stack of new messages from peers, and sends messages via the stack. Thus, it reuses message listening and sending from lower layers.

Protocols

Fantom currently runs 3 protocols:

• FTM62
• FTM63
• fsnap/1 (not covered in this wiki page)

FTM62 and FTM63 are the custom protocols used by fantom.

Common Messages (FTM62 and FTM63)

Handshake

Handshake is the fundamental first step for nodes to know each other. The handshake is a cryptographic message exchange. Only after a successful handshake are nodes going to establish stable peer connections and accept further messages. The first part of handshake reuses the underlying go-ethereum's handshake.

Upon successful handshake, the p2p layer hands over the peer to the fantom protocol handler. The latter first runs some sanity checks to ban peers, which are evaluated as not compatible with the fantom protocols.

At this point, fantom performs its own handshake on top. It sends a Handshake message to the peer:

Send	Message	Handshake
	Parameters	Protocol version	uint32
		Network ID	uint64
		Genesis Hash	[32]byte

This message is immediately followed by a PeerProgress message. The PeerProgress message contains information about this peer’s (sender) state:

Send	Message	PeerProgress
	Parameters	Epoch	uint32
		LastBlockIndex	uint64
		LasBlockAtropos Hash	[32]byte

At this point the peer expects the counterpart to answer with a handshake message. If not, it will abort the handshake.

Receive	Message	Handshake
	Parameters	Protocol version	uint32
		Network ID	uint64
		Genesis Hash	[32]byte

Peer registration

If the handshake was successful, the next stage is to register the peer into its internal list of peers.

Fantom tracks the peer for different purposes inside its protocols.

For all protocols:

• Normal message handling
• DAG syncing

For FTM63:

• Epoch Pack syncing
• Block Votes syncing
• Block Records syncing

If fsnap/1 is supported: fsnap/* 1 syncing

After the peer has been registered to the protocols and all sub-protocols, the node can handle all subsequent messages from the peer.

Note: Handshake messages after this stage will not be handled.

PeerProgress

Receive	Message	PeerProgress
	Parameters	Epoch	uint32
		LastBlockIndex	uint64
		LasBlockAtropos Hash	[32]byte
		HighestLamport (currently unused	uint32

If a PeerProgress message is received, the node stores the progress as advertised in this message in its internal state for that peer. No further action is immediately taken.

This state is later evaluated when an Event occurs to decide if that peer may be interested in that Event (based on the Epoch it is on).

EvmTxsMsg

Receive	Message	EvmTxsMsg
	Parameters	List of transactions

This message signals the incoming of new transactions. This message content is equivalent to go-ethereums types.Transactions. 

Upon receipt of this message, the node marks the sender peer as having the same transactions (in order not to send it to it again in syncing) and adds the contained transactions into the internal transaction pool.

NewEvmTxHashesMsg

Receive	Message	NewEvmTxHashesMsg
	Parameters	List of transactions hashes

With this message, the peer advertizes to this node that it has new transaction hashes. The node marks the sender peer as having those same transactions (in order not to send it to it again in syncing).

After this it will request to download the transactions identified by those hashes from the peer with a GetEvmTxsMsg. If the list size exceeds a defined threshold, the node will send multiple such messages, and thus retrieve the transactions in batches.

Send	Message	GetEvmTxsMsg
	Parameters	List of transactions hashes

GetEvmTxsMsg

Receive	Message	GetEvmTxsMsg
	Parameters	List of transactions hashes requested

This is a request from the peer to have it sent the actual transactions represented by the hashes in the message. Upon which the node queues the requested transactions for sending in batches through EvmTxsMsg instances.

Send	Message	EvmTxsMsg
	Parameters	List of transactions

EventsMsg

Receive	Message	EventsMsg
	Parameters	List of event payloads

The peer notifies with this message that it has new events for this node. The list is an RLP encoded EventPayloads list. The node now filters the events for ones whose lamport is not too high. As the order of the DAG needs to be maintained, the node will filter out events which are too far in the future (based on lamport time), and prioritises like this earlier events. Subsequently it evaluates internally which events it wants to request (for example, missing parent events of the ones it got, or others which it got advertised via NewEventIDsMsg), and issues GetEventMsg to the peer (in batches if required).

Send	Message	GetEventMsg
	Parameters	List of events to get

GetEventMsg

Receive	Message	GetEventMsg
	Parameters	List of events requested

The peer is requesting events as listed inside the message. This node will gather all events it has from this list and enqueue them for sending via EventsMsg instances (in batches if required).

Send	Message	EventsMsg
	Parameters	List of event payloads

NewEventIDsMsg

Receive	Message	NewEventIDsMsg
	Parameters	List of event hashes

A peer notifies with this message that it has new Event hashes. This node scans the list and identifies if it contains any unknown hashes. If so, it will then request the Events represented by these hashes via GetEventsMsg messages.

Send	Message	GetEventMsg
	Parameters	List of events to get

FTM63 Messages

FTM63 is an upgrade on top of FTM62. It brings an improvement to FTM62 by allowing the DAG to be synced more efficiently. This logic is actually implemented as part of the lachesis code base.

The FTM63 messages follow a common pattern for syncing the internal state. For each:

• Events
• Block Votes
• Block Records
• Epoch Packs

There is a corresponding request message, and a response message. Each of these follow the same basic structure (all fields are RLP encoded).

Message	Request
Parameters	Session	ID	uint32
		Start	[]byte
		End	[]byte
	Limit	Num	uint32
		Size	uint64
	Type (IDs or Items)		uint8
	MaxChunks (max number requested)		uint32

Message	Response
Parameters	SessionID	uint32
	Done	bool
	IDs	[][32]byte
	Items	Num

The node maintains a list of sessions per peer (with the peer’s ID as key). These sessions are pruned of its oldest session at each new request notification, and the newest one is added to the list. The remote peer is supposed to maintain the state of these sessions too. Therefore, at each new request, the Session.Start provided in the request is compared with the locally maintained for the peer. If it doesn’t match, the peer is actually marked as misbehaving and the current request is aborted.

If session information matches, the node evaluates if IDs are being requested or actual items, and packs the appropriate data into the corresponding fields. After which the message is complete and is sent to the peer.

RequestEventStream

Receive	Message	RequestEventStream
	Parameters	List of events to get

There are two modes for syncing: "broadcasting" and via "streams". Streams are keeping track with the remote peer of what has been synced so far. It is a kind of pagination for retrieving events.

With this message, the peer is requesting a specific part of the DAG. This node notifies internally that this request has been received. It parses the message and evaluates session data. It then gathers the required information and sends it to peer via EventsStreamResponse messages (in batches if required).

Send	Message	EventsStreamResponse
	Parameters	Encoded list of events

EventsStreamResponse

Receive	Message	EventsStreamResponse
	Parameters	Encoded list of events

If this node receives an EventsStreamResponse message, it will:

• Handle the contained event hashes in the message just as in a NewEventIDsMsg, and request their corresponding events if it doesn’t locally have those event hashes.
• Handle the contained events in the message just as in a EventsMsg, and possibly request events with GetEventsMsg if required.

RequestBVsStream

Receive	Message	RequestBVsStream
	Parameters	Request Block Votes

The peer is requesting blockvotes information. The node will parse the message and evaluate session data. It then gathers the required information and sends it to the peer via BVsStreamResponse messages:

Send	Message	BVsStreamResponse
	Parameters	Encoded Block Votes

BVsSreamResponse

Receive	Message	BVsStreamResponse
	Parameters	Encoded Block Votes

The peer sent Block Votes information. This node will parse the contents of this message and process each of the elements in order to update its internal syncing data.

RequestBRsStream

Receive	Message	RequestBRsStream
	Parameters	Request Block Records

The peer requested Block Records information. The node will parse the message and evaluate session data. It then gathers the required information and sends it to the peer via BVsStreamResponse messages:

Send	Message	BRsStreamResponse
	Parameters	Encoded Block Records

BRsStreamResponse

Receive	Message	BRsStreamResponse
	Parameters	Encoded Block Records

The peer sent Block Record information. This node will parse the contents of this message and process each of the elements in order to update its internal syncing data.

RequestEPsStream

Receive	Message	RequestEPsStream
	Parameters	Request Epoch Packs

The peer requested Epoch Pack Records information. The node will parse the message and evaluate session data. It then gathers the required information and sends it to the peer via EPsStreamResponse messages:

Send	Message	EPsStreamResponse
	Parameters	Encoded Epoch Packs

EPsStreamResponse

Receive	Message	EPsStreamResponse
	Parameters	Encoded Epoch Packs

The peer sent Epoch Pack information. This node will parse the contents of this message and process each of the elements in order to update its internal syncing data.

Further readings:

• go-ethereum's networking layer documentation
• go-ethereum's devp2p
• go-ethereum's rlpx.md