diff --git a/README.md b/README.md index beb040f80..ecdaf7baa 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ Changes to the protocol specification and standards are called NEAR Enhancement ## NEPs | NEP # | Title | Author | Status | -| ----------------------------------------------------------------- | ---------------------------------------- | ------------------------------------------- | ------ | +| ----------------------------------------------------------------- |------------------------------------------|---------------------------------------------|--------| | [0001](https://github.com/near/NEPs/blob/master/neps/nep-0001.md) | NEP Purpose and Guidelines | @jlogelin | Living | | [0021](https://github.com/near/NEPs/blob/master/neps/nep-0021.md) | Fungible Token Standard (Deprecated) | @evgenykuzyakov | Final | | [0141](https://github.com/near/NEPs/blob/master/neps/nep-0141.md) | Fungible Token Standard | @evgenykuzyakov @oysterpack | Final | @@ -30,6 +30,8 @@ Changes to the protocol specification and standards are called NEAR Enhancement | [0399](https://github.com/near/NEPs/blob/master/neps/nep-0399.md) | Flat Storage | @Longarithm @mzhangmzz | Review | | [0448](https://github.com/near/NEPs/blob/master/neps/nep-0448.md) | Zero-balance Accounts | @bowenwang1996 | Final | | [0455](https://github.com/near/NEPs/blob/master/neps/nep-0455.md) | Parameter Compute Costs | @akashin @jakmeier | Final | +| [0514](https://github.com/near/NEPs/blob/master/neps/nep-0514.md) | Fewer Block Producer Seats in `testnet` | @nikurt | Final | + ## Specification diff --git a/nep-0000-template.md b/nep-0000-template.md index 772979545..47b65bdac 100644 --- a/nep-0000-template.md +++ b/nep-0000-template.md @@ -58,11 +58,11 @@ See example above --> [This technical section is required for Protocol proposals but optional for other categories. A draft implementation should demonstrate a minimal implementation that assists in understanding or implementing this proposal. Explain the design in sufficient detail that: -- Its interaction with other features is clear. -- Where possible, include a Minimum Viable Interface subsection expressing the required behavior and types in a target programming language. (ie. traits and structs for rust, interfaces and classes for javascript, function signatures and structs for c, etc.) -- It is reasonably clear how the feature would be implemented. -- Corner cases are dissected by example. -- For protocol changes: A link to a draft PR on nearcore that shows how it can be integrated in the current code. It should at least solve the key technical challenges. +* Its interaction with other features is clear. +* Where possible, include a Minimum Viable Interface subsection expressing the required behavior and types in a target programming language. (ie. traits and structs for rust, interfaces and classes for javascript, function signatures and structs for c, etc.) +* It is reasonably clear how the feature would be implemented. +* Corner cases are dissected by example. +* For protocol changes: A link to a draft PR on nearcore that shows how it can be integrated in the current code. It should at least solve the key technical challenges. The section should return to the examples given in the previous section, and explain more fully how the detailed proposal makes those examples work.] @@ -84,15 +84,15 @@ The section should return to the examples given in the previous section, and exp ### Positive -- p1 +* p1 ### Neutral -- n1 +* n1 ### Negative -- n1 +* n1 ### Backwards Compatibility @@ -102,9 +102,9 @@ The section should return to the examples given in the previous section, and exp [Explain any issues that warrant further discussion. Considerations -- What parts of the design do you expect to resolve through the NEP process before this gets merged? -- What parts of the design do you expect to resolve through the implementation of this feature before stabilization? -- What related issues do you consider out of scope for this NEP that could be addressed in the future independently of the solution that comes out of this NEP?] +* What parts of the design do you expect to resolve through the NEP process before this gets merged? +* What parts of the design do you expect to resolve through the implementation of this feature before stabilization? +* What related issues do you consider out of scope for this NEP that could be addressed in the future independently of the solution that comes out of this NEP?] ## Changelog @@ -118,8 +118,8 @@ The section should return to the examples given in the previous section, and exp > List of benefits filled by the Subject Matter Experts while reviewing this version: -- Benefit 1 -- Benefit 2 +* Benefit 1 +* Benefit 2 #### Concerns diff --git a/neps/nep-0514.md b/neps/nep-0514.md new file mode 100644 index 000000000..10a3bae80 --- /dev/null +++ b/neps/nep-0514.md @@ -0,0 +1,166 @@ +--- +NEP: 514 +Title: Reducing the number of Block Producer Seats in `testnet` +Authors: Nikolay Kurtov +Status: New +DiscussionsTo: https://github.com/nearprotocol/neps/pull/514 +Type: Protocol +Version: 1.0.0 +Created: 2023-10-25 +LastUpdated: 2023-10-25 +--- + + +## Summary + +This proposal aims to adjust the number of block producer seats on `testnet` in +order to ensure a positive number of chunk-only producers present in `testnet` +at all times. + +## Motivation + +The problem is that important code paths are not exercised in `testnet`. This +makes `mainnet` releases more risky than they have to be, and greatly slows +down development of features related to chunk-only producers, such as State +Sync. + +That is because `testnet` has fewer validating nodes than the number of block +producer seats configured. + +The number of validating nodes on `testnet` is somewhere in the range of +[26, 46], which means that all validating nodes are block producers and none of +them are chunk-only producers. [Grafana](https://nearinc.grafana.net/goto/7Kh81P7IR?orgId=1). + +`testnet` configuration is currently the following: + +* `"num_block_producer_seats": 100,` +* `"num_block_producer_seats_per_shard": [ 100, 100, 100, 100 ],` +* `"num_chunk_only_producer_seats": 200,` + +It's evident that the 100 block producer seats significantly outnumber the +validating nodes in `testnet`. + +An alternative solution to the problem stated above can be the following: + +1. Encourage the community to run more `testnet` validating nodes +1. Release owners or developers of features start a lot of validating nodes to +1. ensure `testnet` gets some chunk-only producing nodes. +1. Exercise the unique code paths in a separate chain, a-la `localnet`. + +Let's consider each of these options. + +### More community nodes + +This would be the ideal perfect situation. More nodes joining will make +`testnet` more similar to `mainnet`, which will have various positive effects +for protocol developers and dApp developers. + +However, this option is expensive, because running a validating node costs +money, and most community members can't afford spending that amount of money for +the good of the network. + +### More protocol developer nodes + +While this option may seem viable, it poses significant financial challenges for +protocol development. The associated computational expenses are exorbitantly +high, making it an impractical choice for sustainable development. + +### Test in separate chains + +That is the current solution, and it has significant drawbacks: + +* Separate chains are short-lived and may miss events critical to the unique + code paths of chunk-only producers +* Separate chains need special attention to be configured in a way that + accommodates for chunk-only producers. Most test cases are not concerned about + them, and don't exercise the unique code paths. +* Separate chains can't process real transaction traffic. The traffic must + either be synthetic or "inspired" by real traffic. +* Each such test has a significant cost of running multiple nodes, in some + cases, tens of nodes. + +## Specification + +The proposal suggests altering the number of block producer seats to ensure that +a portion of the `testnet` validating nodes become chunk-only producers. + +The desired `testnet` configuration is the following: + +* `"num_block_producer_seats": 20,` +* `"num_block_producer_seats_per_shard": [ 20, 20, 20, 20 ],` +* `"num_chunk_only_producer_seats": 100,` + +I suggest to implement the change for all networks that are not `mainnet` and +have `use_production_config` in the genesis file. `use_production_config` is a +sneaky parameter in `GenesisConfig` that lets protocol upgrades to change +network's `GenesisConfig`. + +I don't have a solid argument for lowering the number of chunk producer seats, +but that reflects the reality that we don't expect a lot of nodes joining +`testnet`. It also makes it easier to test the case of too many validating nodes +willing to join a network. + +## Reference Implementation + +[#9563](https://github.com/near/nearcore/pull/9563) + +If `use_production_config`, check whether `chain_id` is eligible, then change +the configuration as specified above. + +## Security Implications + +The block production in `testnet` becomes more centralized. It's not a new +concern as 50% of stake is already owned by nodes operated by the protocol +developers. + +## Alternatives + +See above. + +## Future possibilities + +Adjust the number of block and chunk producer seats according to the development +of the number of `testnet` validating nodes. + +## Consequences + +### Positive + +* Chunk-only production gets tested in `testnet` +* Development of State Sync and other features related to chunk-only producers accelerates + +### Neutral + +* `testnet` block production becomes more centralized + +### Negative + +* Any? + +### Backwards Compatibility + +During the protocol upgrade, some nodes will become chunk-only producers. + +The piece of code that updates `testnet` configuration value will need to be +kept in the database in case somebody wants to generate `EpochInfo` compatible +with the protocol versions containing the implementation of this NEP. + +## Changelog + +### 1.0.0 - Initial Version + +The Protocol Working Group members approved this NEP on Oct 26, 2023. + +[Zulip link](https://near.zulipchat.com/#narrow/stream/297873-pagoda.2Fnode/topic/How.20to.20test.20a.20chunk-only.20producer.20node.20in.20testnet.3F/near/396090090) + +#### Benefits + +See [Consequences](#consequences). + +#### Concerns + +See [Consequences](#consequences). + +## Copyright + +Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).