diff --git a/README.md b/README.md index fcca07b..991d70b 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,43 @@ ## Withdrawals API -How works withdrawal time calculation [here](how-calculation-works.md) +
+ Withdrawals API Logo +
-## Installation +The **Withdrawals API** service offers an utility for estimating the waiting time for [withdrawals](https://docs.lido.fi/guides/lido-tokens-integration-guide#withdrawals-unsteth) within the Lido protocol on Ethereum. + +The service is helpful for stakers, providing insights from the moment of withdrawal request [placement](https://docs.lido.fi/contracts/withdrawal-queue-erc721#request) to its [finalization](https://docs.lido.fi/contracts/withdrawal-queue-erc721#finalization) when the request becomes claimable. + +### Use Cases + +- Estimation before request: users can estimate the waiting time before placing a withdrawal request. +- Tracking the existing request: users can track the estimated waiting time for the already placed request. + +ℹ️ See also the [detailed explanation](how-estimation-works.md) of the estimation algorithm. + +### Prerequisites + +- Node.js (version 16.0 or higher) +- Yarn (version 1.22 or higher) + +### Installation ```bash $ yarn install ``` +### Configuration + +```bash +# Edit the newly created `.env` file to populate with proper values. +$ cp sample.env .env +``` + + + ## Running the app + ```bash # development $ yarn start @@ -46,4 +74,4 @@ To create a new release: ## License -API Template is [MIT licensed](LICENSE). +Withdawals API is [MIT licensed](LICENSE). diff --git a/how-calculation-works.md b/how-calculation-works.md deleted file mode 100644 index 6ea4435..0000000 --- a/how-calculation-works.md +++ /dev/null @@ -1,68 +0,0 @@ -## How withdrawal request time calculation works - -### The algorithm of searching for a first suitable calculation case: - -1. `bunker` mode is active. -2. `buffer` or `vaultsBalance` has enough funds. -3. select minimum frame from the following three cases of `rewardOnly`, *`validatorBalances`* or *`exitValidators`.* - -Each case of calculation finds epoch when Lido can finalize requests, but finalization happens during Accounting Oracle report and it happens in exact reference slot. Here are examples of reference slots for different networks: mainnet - 12:00:04 UTC, goerli - 05:28:00 UTC, holesky - each 2 hours. - -### Calculate next report time - -- calculation of epoch of next frame of report: - `epochOfNextReport = initialEpoch + nextFrame * epochsPerFrame` -- `initialEpoch` and `epochsPerFrame` are from `accountingOracleHashConsensus` contract [getFrameConfig](https://docs.lido.fi/contracts/hash-consensus#getframeconfig) method - -### Gap before next report - -- Withdrawal requests which are close to report (`getRequestTimestampMargin`) will not be in closest report, will be postponed to the next one if they are in that gap before report. -- `OracleReportSanityChecker` contract, get `limits.requestTimestampMargin` from [getOracleReportLimits](https://docs.lido.fi/contracts/oracle-report-sanity-checker#getoraclereportlimits) - -### Gap after report - -- it takes time to proceed report it usually it can take 20-30 min. But in emergency cases there is possibility that report won’t happen in current frame, but possibility is very low and it happened only around 5 time in history. - -## More about each calculation case - -### Case if there is enough tokens in buffer + withdrawals vaults for withdrawal: - -If next sources of tokens has enough funds for current request + all others unfinalized requests before this request, we can withdraw tokens without exit validators process. - -- lido buffer from [method](https://docs.lido.fi/contracts/lido#getbufferedether) -- balance of [WithdrawalVaults](https://docs.lido.fi/contracts/withdrawal-vault) and [ExecutionLayerRewardsVault](https://docs.lido.fi/contracts/lido-execution-layer-rewards-vault) - ---- - -### Case of rewards only - -Based on rewards from previous report we can consider this number as approximate future rewards. Formula to get epoch - -`onlyRewardPotentialEpoch = unfinalized / rewardsPerEpoch` - -where `unfinalized` is current request + all requests before it. - ---- - -### Case of validators with withdrawable epoch - -We can find Lido validators which is already in process of withdrawal and group them by `withdrawable_epoch` to `frameBalances`, then we can find frame and return it. - ---- - -### Case if there is NOT enough tokens for withdrawal: - -- We are going to calculate how much reports should be proceeded to fulfil current withdrawal -- Basic target is to find epoch for exit all needed validators -- Find epoch by this formula: `unfinalizedStethe / (32eth * churnLimit + rewardsPerEpoch)` -- `rewardsPerEpoch` calculation can be found [here](https://hackmd.io/@lido/r1fau3aJ3?type=view#Predict-available-ETH-before-next-withdrawn) -- And last thing to count is `sweepingMean`. More information can be found [here](https://consensys.net/shanghai-capella-upgrade/) in *Full Withdrawal Process* chapter*.* Shortly it is minimum 256 epoch (27.3 hour). But we use middle value which is 567 epoch (depending on total validators number). ** -- Next find next frame after found epoch potential epoch -- No gap before report but gap after report still exists - ---- - -### [Under development] Case of bunker active - -We check if bunker is active in contract [here](https://docs.lido.fi/contracts/withdrawal-queue-erc721#isbunkermodeactive) and predict that finalization will be in approximately 14 frames as constant if there are no associated slashings. More about bunker more [here](https://docs.lido.fi/guides/oracle-spec/accounting-oracle/#bunker-mode-activation). - diff --git a/how-estimation-works.md b/how-estimation-works.md new file mode 100644 index 0000000..d033711 --- /dev/null +++ b/how-estimation-works.md @@ -0,0 +1,121 @@ +## Understanding withdrawal waiting time estimation + +### Introduction + +This guide details the method used in the Withdrawals API service to estimate the time from when a withdrawal request is placed in the Lido protocol until it is finalized and becomes claimable. + +By demystifying this process, the document is aimed to provide both users and developers with a clear understanding of the steps involved. + +### The algorithm overview + +The estimation of the withdrawal waiting time begins by identifying the applicable base case, based on the following conditions: + +The algorithm estimates withdrawal waiting times by evaluating the current state and projecting future events. + +1. Check if the `BUNKER` mode is active. +2. Determine if the `buffer` or `vaultsBalance` holds enough ether to cover withdrawal request (`totalBuffer` is enough). +3. Choose the soonest possible case for covering the withdrawal request, which could be: + 1. `rewardsOnly`: incorporating expected rewards from consensus layer (CL) and execution layer (EL). + 2. `validatorBalances`: leveraging balances from validators scheduled for withdrawal + 3. `exitValidators`: considering additional validator exits necessary to accumulate sufficient funds. + +Regardless of the base case detected, every possible withdrawal request finalization still aligns with the [AccountingOracle](https://docs.lido.fi/contracts/accounting-oracle) report, occurring daily (with only rare exceptions), gathered for the exact reference slot (e.g., 12:00:04 UTC every day for Mainnet) happening with a submission delay about ~30 minutes (see the example [tx](https://etherscan.io/tx/0x569556dd4694408de8c8c0a164f4ace48273227c156b42969cd75034063f0907) for `AccountingOracle.submitReportData(...)`). + +Therefore, the withdrawal time estimation is affected by the quantization of the time scale based on the projected oracle report timestamps. + +### AccountingOracle report finalization quantization + +The section provides an overview of the `AccountingOracle` key timings and margins. + +#### Calculate next AccountingOracle report reference epoch + +Since the withdrawal finalization happens together with the AccountingOracle report, the first step needed is to calculate the nearest (next) report reference epoch. + +- calculation of epoch of the next frame of the `AccountingOracle` report: + `epochOfNextReport = initialEpoch + nextFrame * epochsPerFrame` +- `initialEpoch` and `epochsPerFrame` are taken from the `HashConsensus` contract instance deployed for `AccountingOracle` via the [getFrameConfig](https://docs.lido.fi/contracts/hash-consensus#getframeconfig) method, the frame length is 1 day on Mainnet. + +#### Finalization safe border before the next report + +- Withdrawal requests which were placed too close to the upcoming report will be postponed further to the closest suitable report following the upcoming one +- The margin (i.e., finalization safe border) is defined inside the `OracleReportSanityChecker` contract and can be retrieved as `limits.requestTimestampMargin` from [getOracleReportLimits](https://docs.lido.fi/contracts/oracle-report-sanity-checker#getoraclereportlimits), the default value on Mainnet is 2 hours if the protocol is in the `TURBO` (i.e., not `BUNKER`) mode, see the following [thread](https://research.lido.fi/t/withdrawals-for-lido-on-ethereum-bunker-mode-design-and-implementation/3890/4). + +### Report submission delay after the reference epoch arrival + +- It usually takes 20-30 minutes to prepare the oracle report (mostly to wait when the reference epoch is finalized). +- However, in rare cases it's possible that the report won’t happen in the current reporting frame, though it's unlikely and happened only a few times since the moment of the Lido protocol launch. + +## Base cases + +### 1. [WIP] Case of the active `BUNKER` mode (unlikely) + +The `BUNKER` mode is checked via the `WithdrawalQueueERC721` contract using the utility view method [here](https://docs.lido.fi/contracts/withdrawal-queue-erc721#isbunkermodeactive). + +> In the `BUNKER` mode, a fixed finalization time of 14 frames is assumed unless affected by slashing events, reflecting a more conservative approach to claimable ether accessibility. + +More about the `BUNKER` mode is [here](https://docs.lido.fi/guides/oracle-spec/accounting-oracle/#bunker-mode-activation). + +### 2. Case if there is enough buffered ether (including withdrawals and EL rewards vaults balances) + +If there is enough ether from the following sources to finalize all unfinalized requests placed before and included the provided one: + +- buffer ether balance retrieved from the [Lido.getBufferedEther()](https://docs.lido.fi/contracts/lido#getbufferedether) method +- ether balance of [WithdrawalVault](https://docs.lido.fi/contracts/withdrawal-vault) +- ether balance of [ExecutionLayerRewardsVault](https://docs.lido.fi/contracts/lido-execution-layer-rewards-vault) + +i.e, `totalBuffer = Lido.getBufferedEther() + balanceOf(WithdrawalVault) + balanceOf(ELRewardsVault)` + +Then the finalization is possible relying on the already existing `totalBuffer` ether amount in the nearest oracle report (including the safe board and submission delay into consideration). + +--- + +### 3.i. Case of the projected rewards + +If there is not enough ether to fulfill the withdrawal request (`unfinalized > totalBuffer`), the next case is to consider projected rewards for the future report. In this case, the projected rewards amount can be approximately derived from the previous report. + +The following formula is utilized to get epoch + +`projectedRewardsPotentialEpoch = (unfinalized - totalBuffer) / rewardsPerEpoch` + +where `unfinalized` is the amount of the withdrawal request considered summed with all of the unfinalized requests placed before. + +--- we can consider this number as approximate future rewards + +### 3.ii. Case of validators with withdrawable epoch + +If there is not enough ether to fulfill the withdrawal request (`unfinalized > totalBuffer`), the previous case might be appended with the known validators are to be withdrawn (when the `withdrawable_epoch` is assigned). + +It's needed to select the Lido-participating validators which are already in process of withdrawal and group them by `withdrawable_epoch` to `frameBalances`, allowing to find the oracle report frame containing enough funds from: + +- buffer (`totalBuffer`) +- projectedRewards (`rewardsPerEpoch * epochsTillTheFrame`) +- frameBalances (`object { [frame]: [sum of balances of validators with withdrawable_epoch for certain frame] }`) + +--- + +### 3.iii. Case when new validator exits are needed to finalize the withdrawal requests + +- The idea is to find the nearest epoch pretending that all needed validators were exited +- Find epoch by this formula: `unfinalizedStETH / (32ETH * churnLimit + rewardsPerEpoch)` +- `rewardsPerEpoch` is calculated as described in the provided [prediction model](https://hackmd.io/@lido/r1fau3aJ3?type=view#Predict-available-ETH-before-next-withdrawn) + +- Worth noting that the exited validators become withdrawable only after the withdrawal sweep approached them. +For simplicity, it's suggested to use the average time of the withdrawal sweep (`sweepingMean`) as a constant timeframe extension when estimating the withdrawal waiting time. + +More information can be found [here](https://consensys.net/shanghai-capella-upgrade/) in *Full Withdrawal Process* chapter*.* + +> it is takes 256 epoch (27.3 hours) at minimum on Mainnet, and the current avg value is 567 epochs (the values depend on the total validators number). + +--- + +### Conclusion + +This guide aims to offer a comprehensive understanding of the factors influencing the estimation of withdrawal request finalization times within the Lido protocol. For further details and updates, refer to the links provided in the References section. + +### References + +- [Withdrawals: Lido tokens integration guide](https://docs.lido.fi/guides/lido-tokens-integration-guide#withdrawals-unsteth) +- [AccountingOracle: withdrawals stage](https://docs.lido.fi/guides/oracle-spec/accounting-oracle#withdrawal-stage) +- [ValidatorExitBus specification](https://docs.lido.fi/guides/oracle-spec/validator-exit-bus) +- [Just how fast are Ethereum withdrawals using the Lido protocol](https://blog.lido.fi/just-how-fast-are-ethereum-withdrawals-using-the-lido-protocol/) +- [Lido Withdrawal Queue Stats: Dune dashboard](https://dune.com/lido/lido-v2) diff --git a/package.json b/package.json index 3681343..3b9f344 100644 --- a/package.json +++ b/package.json @@ -2,9 +2,13 @@ "name": "withdrawals-api", "version": "0.0.1", "description": "Withdrawals API", - "author": "Lido team", + "author": "Lido contributors", "private": true, "license": "MIT", + "engines": { + "node": ">=16", + "yarn": ">=1.22" + }, "scripts": { "prebuild": "rimraf dist", "build": "nest build", diff --git a/static/img/logo.jpg b/static/img/logo.jpg new file mode 100644 index 0000000..174d993 Binary files /dev/null and b/static/img/logo.jpg differ