core/txpool/legacypool: add overflowpool for txs #2660
Conversation
@buddh0 if by replacement transaction you mean transaction with same nonce but different/higher gas price then if the first transaction is already in pool3 (which we are calling Overflow Pool now) then the Main Pool is unaware of it and might treat the replacement transaction as the only one and might include in a block. In that case when the original transaction finally gets from Overflow Pool to Main Pool, it will be discarded due to |
ok |
|
emailtovamos
changed the title
| } | ||
| item := old[n-1] | ||
| old[n-1] = nil // avoid memory leak | ||
| *h = old[0 : n-1] |
There was a problem hiding this comment.
It will pop the latest tx, right? but IMO, we wanna to pop the oldest tx.
So it is actually a Stack, not a Heap.
Should we change it to:
item := old[0]
old[0] = nil // avoid memory leak
*h = old[1 : n]
But, it will update the all item's index...
There was a problem hiding this comment.
It will pop the latest tx, right? but IMO, we wanna to pop the oldest tx. So it is actually a Stack, not a Heap. Should we change it to:
item := old[0] old[0] = nil // avoid memory leak *h = old[1 : n]But, it will update the all item's index...
heap.Pop will pop the oldest one:
heap.Pop will put the oldest item at the end firstly, compare by the func txHeap.Less
then call txHeap.Pop
There was a problem hiding this comment.
ok, got it. but I wonder if it would cost lots of CPU resource for each Pop operation, as it needs to do sorting first
)" This reverts commit 675449a.
Description
This PR incorporates an Overflow Pool which stores transactions in memory for such cases and time to time transfers them to the Main Pool when the Main Pool isn't full. It is not propagated or announced while it is in the Overflow Pool. But when it is finally transferred to the Main Pool, at that point it is considered as a transaction that just arrived at the Main Pool and is subject to the usual checks and propagation logic.
Overflow Pool's size can be chosen by a node depending on its hardware capacity.
Rationale
During high traffic, transactions are lost as the current legacy pool (Main Pool) has limits on its capacity and propagating them during high traffic also costs network congestions. So this is an attempt to solve 'tx lost' and get better UX.
Details
a. Main Pool
There is no new structure for Main Pool, actually it is just the current Transaction Pool (Legacy Pool). When this Pool isn't full, the received new transactions will be broadcasted to all the relevant peers, just the same as the current behaviour.
b. Overflow Pool: the local buffer (LRU or Heap)
When the Main Pool is overflowed during high traffic, then it would put lots of stress on the network if we keep broadcasting new transactions. So we put any new transaction into the Overflow Pool and don’t broadcast or announce.
The size of the Overflow Pool could be very large, in order to hold a large traffic volume, like 500K. Suppose the average transaction size is 500B, it will take around 256MB memory, which is acceptable.
How to flush transactions from Overflow Pool to Main Pool: