Internal: TX processing is now event-based #492
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
odesenfans
force-pushed
the
od-event-based-tx-processor
branch
from
6e1134b to
61691ac
Compare
Problem: the pending TX processor uses a polling loop to determine which transactions must be processed. This leads to some latency when adding a new transaction when there is no currently pending transaction as the task sleeps while waiting for new transactions. Solution: make the TX processor event-based by using a RabbitMQ exchange + message queue. Upon receiving a new transaction from on-chain, we now publish a RabbitMQ message containing the hash of the transaction. The processor can then find the transaction in the DB. We still store an entry in the pending TXs table for monitoring purposes.
odesenfans
force-pushed
the
od-event-based-tx-processor
branch
from
61691ac to
befa366
Compare
hoh
pushed a commit
that referenced
this pull request
Dec 8, 2023
Problem: the pending TX processor uses a polling loop to determine which transactions must be processed. This leads to some latency when adding a new transaction when there is no currently pending transaction as the task sleeps while waiting for new transactions. Solution: make the TX processor event-based by using a RabbitMQ exchange + message queue. Upon receiving a new transaction from on-chain, we now publish a RabbitMQ message containing the hash of the transaction. The processor can then find the transaction in the DB. The RabbitMQ message is only used as a hint that something must be done. The TX processor still looks for pending TXs in the database.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem: the pending TX processor uses a polling loop to determine which transactions must be processed. This leads to some latency when adding a new transaction when there is no currently pending transaction as the task sleeps while waiting for new transactions.
Solution: make the TX processor event-based by using a RabbitMQ exchange
We still store an entry in the pending TXs table for monitoring purposes.