Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update feature/standalone 1706 with changes from upstream #1

Open
wants to merge 637 commits into
base: feature/standalone_1706
Choose a base branch
from

Conversation

zguesmi
Copy link
Member

@zguesmi zguesmi commented Feb 9, 2024

No description provided.

asdacap and others added 30 commits May 5, 2023 20:49
* Add ability to tune db

* Remove tuning

* Cleanup
* Receipt key with block number

* Thanks copilot.

* Fix hasblock not working

* Check first key first

* Fix crash when seeking to first on disposed iterator

* Missed a file

* Fix crash with rocksdb
Co-authored-by: Nikita Mescheryakov <[email protected]>
* Add readahead flag

* Fix caching store does not forward flags

* Make it column family safe

* Minor cleanup

* Make it configurable

* Adjust default

* Ok, lets try this

* Comment and calculate readahead threshold carefully

* Minor comment

* Adjust threshold

* Back to previous threshold

* Whitespace

* Minor unit test

* Whitespace

* Managed iterator

* Slight cleanup

* Whitespace

* Fix missed column family

* Add test

* Fix test can't run together

* Inherit ThreadLocal

* Remove unneeded usings

* Move iteration dispose to class

---------

Co-authored-by: benaadams <[email protected]>
* Fix & refactor

* static class

---------

Co-authored-by: Nikita Mescheryakov <[email protected]>
* Put tests in matrix

* Fix DotNetty memory leaks

* Revise test projects names

* Fix trace store test

* Put tests with coverage in matrix

* Remove repo owner check

* Combine test workflows

* Remove redundant flag

* Revise test name
* One that touches keccak.bytes

* Update src/Nethermind/Nethermind.Crypto/Keccak512.cs

---------

Co-authored-by: Ben Adams <[email protected]>
…5676)

* Allow specifying concurrent connection count and connect timeout

* Update src/Nethermind/Nethermind.Network/Config/INetworkConfig.cs

---------

Co-authored-by: Ben Adams <[email protected]>
* IsSyncing & error codes

* Fix spaces

* Sync degraded

* Fix spaces

* Fix build

* Add test

* Fix tests

* Remove ToArray

* Fix test
* Updating Fast Sync config files

* few fixes

---------

Co-authored-by: marcindsobczak <[email protected]>
Co-authored-by: Marcin Sobczak <[email protected]>
* Update pivot basing on message from CL

* cosmetic

* cosmetic

* add max attempts, refactor

* decrease log level when failed to update

* add check if new pivot is higher than old one

* adjust bodies and receipts sync feeds

* update pivot only when state sync not finished

* fix logger levels

* add validation of hash of pivot block header

* save new pivot for bodies and receipts feeds

* refactor, add storing in db

* enable update by default (for testing reasons)

* one more place

* cosmetics

* make disabled by default

* enable by default with timeout 900s (15min)

* fix Interlock to be thread safe

* use MetadataDb from interface

* add logger.Trace to ShouldBeInUpdatingPivot()

* add flag notInUpdatingPivot to all traces

* read pivots from DB until whole sync not finished (including receipts)

* Revert "read pivots from DB until whole sync not finished (including receipts)"

This reverts commit eccecab.

* compress pivot data to be under one key in db

* change some Warn logs to Info in PivotUpdator

* fix beacon headers in MultiSyncModeSelector

* add trace to ShouldBeInBeaconHeaders

* fix whitespaces

* print waiting log every 10s instead of 1s
Co-authored-by: Kamil Chodoła <[email protected]>
* remove codeDb from ReadOnlyTxProcessingEnv

* merge the UpdateCode and UpdateCodeHash function into InsertCode

* use only world state provider and remove storage and state provider

* fix whitespace

* fix
* Move to value keccak on snap response

* More conversion

* Minor adjustment

* in modifier
* No-warnings Nethermind.Serialization.Ssz.Test.csproj

* revert submodule
- Solves #5582
- Default address is still `http://localhost:8545`
* Remove AccountStartNonce

* Clean up Keccak use

* Reorg init

* Optimize IsContract

* Test new path

* Optimize "Is contract" solidity patterns

* Simpler construction

* Per sec counts aren't useful for Evm metrics

* Short-circuit code

* Revert "Per sec counts aren't useful for Evm metrics"

This reverts commit 7312b9c.

* Fix merge conflict

* Feedback
* Kch/add sync testnets pr check (#5685)

* Create sync-testnets.yml

* Update sync-testnets.yml

* Change a branch against which action is running

* Change a ref for which action is executed with all refs

* Rollback checkout step

* Add logs of sedge

* Remove -f flag to just show what was done up until now + add a 1 minute wait for some more logs

* Add sepolia

* Fix script issue

* Increase timeout with docker image to 20 just in case of GitHub degradation

* Change to sync only on master merge

* Fix warning + update step version

* Add test branch for testing purpose

* Fix branch reference

* Change to local docker image instead of pushing to docker hub

* Fix yaml issue

* Fix typo

* Fix docker build according to action

* remove arm and add load

* Update sync-testnets.yml

* Final cleanup

* Create sync-testnets.yml

Update sync-testnets.yml

Change a branch against which action is running

Change a ref for which action is executed with all refs

Rollback checkout step

Add logs of sedge

Remove -f flag to just show what was done up until now + add a 1 minute wait for some more logs

Add sepolia

Fix script issue

Increase timeout with docker image to 20 just in case of GitHub degradation

Change to sync only on master merge

Fix warning + update step version

Add test branch for testing purpose

Fix branch reference

Change to local docker image instead of pushing to docker hub

Fix yaml issue

Fix typo

Fix docker build according to action

remove arm and add load

Update sync-testnets.yml

Final cleanup

Create sync-testnets.yml

---------

Co-authored-by: Kamil Chodoła <[email protected]>

* Add additional basic health checks + temp branch for testing

* Add Verify for chiado and fix comment

* Bring back load?

* Add missing workspace

* Improve logging for failure

* Improve Validation of logs

* Ensure that logs will check only execution

* Change waiting for node to sync method + improve logs verifier

* Improve Verify logs

* Improve Verify since not working still

* Add a option to prevent of executing JsonRPC before container is properly created.

* Fix typo

* Update Verify logs with new approach

* Remove unnecessary exits

* Add check if service is alive

* Add +e to ensure that script will be executed fully

* Fix wc missreports empty result as 1

* Update sync-testnets.yml

---------

Co-authored-by: Kamil Chodoła <[email protected]>
* No-warnings Nethermind.Serialization.Ssz.Test.csproj

* revert submodule

* Chiado Fork tests

* revert submodule

* Chiado Fork tests

* change hardcoded value to Chaido.Shanghai

* changed blockNumber for future fork block hash calculation
* autodoc compatible with doxygen

added Docker file, scripts and readme to reproduce in tools.

* Removing unneeded files

Allow users to generate auto doc with more reliable GPT4-30k versions

* Making Doxyfile easier for review

Removed commented out lines

* Removing commented out lines
flcl42 and others added 30 commits August 18, 2023 15:55
* - Added Eip4788 to ReleaseSpecs
- Added Eip4788 wiring to ChainSpecs

* - Added passing state to Precompiles
- Added Eip4788 stateful precompile

* - Adds consensus logic of Eip4788

* - Tests update

* - Minor fixes, Missing migration code

* - Test improvements

* - Test progress, Cleanup

* - More tests fixes

* - Whitespace fixes

* - Attempt fix failing JsonRpc.Tests

* - Fix whistespace check failure

* - move setup precompile 4788 to instance method
- make Instance field in Precompile static generic to match class type

* - fix some build issue

* - fixed failing tests

* - Apply changes

* - Fix failing tests

* - ws fixes

* - Introduce IBeaconBlockRootHandler

* - Removed unnecessary "null" default value

* - Fixed some naming inconsistensies

* - Fix ws problems

* - Fix default behavior of beaconBlockRootHandler argument in BlockProcessor.ctor

* - fix

* - Fix ws

* small improvements

* Add EngineApiVersions

* - refactor PayloadAttributes.Validate

* - ws fix

* Init beacon root precompile state for genesis (#5982)

* - Applied suggested changes

* Fix merge

* Fix name, whitespaces

* Fix expected code

* Remove balance set

* genesis.json beacon root.

* jsonrpc: fix parentBeaconBlockRoot naming, add isEip4788.

* Catch args issue

* - ignore 0x0b from precompile checks

* Ignore for genesis

* Don't count RLP wrapper, when calc tx size for pooled txs msg

* Revert "Trie recovery (#5861)"

This reverts commit a099f0f.

* Use main EIP to detect fork

* Adding some workarounds

* Add system acc

* Another workaround

* What if zero bloc

* Another idea

* Another idea

* +

* +

* Another attempt

* 1) What

* No genesis

* 2) H

* Revert "Revert "Trie recovery (#5861)""

This reverts commit 1425d3c.

* Adhere a formal rule of encoding

* Decline payloads with v3 fields

* 2) The

* Fix 2) The

* clarification comment

* fix typo in file name

* Add RLP testing scenarios

* FIx tracer type

* - 4788 cleanup

* - Make Eip4788 settabe from chainspec

* Fix merge

* - other tests fixes

* Working on review comments

* fix build

* remove BeaconBlock from precompile list

* - moved ParentBeaconBlockRoot type from byte[] to Keccak
- removed commented precompile line from VirtualMachine

* - fixed some tests
- removed deprectated tests

* - more cleanup

* - adjusted expected size in blob tx size tests

* - ws fix

* Default addr

* more cosmetic stuff

* Remove worldState parameter

* Revert "more cosmetic stuff"

This reverts commit 872bce8.

---------

Co-authored-by: Demuirgos <[email protected]>
Co-authored-by: lukasz.rozmej <[email protected]>
Co-authored-by: Alexey <[email protected]>
Co-authored-by: spencer-tb <[email protected]>
Co-authored-by: smartprogrammer <[email protected]>
…Tests.sln (#6018)

* Removed unused using

* Remove from Benchmarks.sln
…6016)

* Rebroadcast all persistent txs if PeerNotificationThreshold is 100

* whitespace

* extend test to check tx order

* cosmetic

* modify config description

---------

Co-authored-by: MarekM25 <[email protected]>
* Deserialize work

* Enable slice memory

* Integrate with old bodies

* Fix missed dispose

* Unmanaged block bodies test

* Fix hash not working

* Minor cleanup

* Finalize pattern in metty buffer instead

* Fix memory prehash still pool

* Pool tx also

* Rename to OwnedBlockBodies

* Cleanup

* More cleanup

* Whitespace

* SuppressFinalize

* Increase retained tx
* Minor optimization on peek

* Fix peek

* Skip and read position without property
* Progress

* Capped array

* Unit test passed

* Should work now

* Tracking all buffer

* Separate the pool from the trie resolver

* Cleanup

* fix wrong calculate root

* Micro optimize

* Cleanup and receipts root also

* More tests

* Directly store array if possible

* Cleanup

* Test pool

* Whitespace

* Minor optimization on peek

* Fix peek

* Skip and read position without property

* Reduced memory usage

* Fix missed tests

* Ensure non nullable

* More unit tests

* Cleanup

* More cleanup

* Minor optimization

* More cleanup

* Fix minor mistake
* Handle `AccessList` differently depending on `TxType`

- Legacy does not have an `AccessList` field
- Others have an `AccessList` field and should be at least an empty array

* Remove all fields except serializer

- Can create issues if we don't properly reset the field

* Use raw strings everywhere

* Use file-scoped namespace

* Replace string comparisons with JSON equivalence

* Formatting

* Fix whitespace
* Add holesky configs and chainspec

* fixes

* Add tests and fix for 4788 address

* adjust shanghai timestamp for holeskey to be genesis + (256*12)
* Explicitly flush WAL on state sync finish

* Lock is unnecessary here
* Code cleanup

- Extract constants
- Use file-scoped namespace
- Use guards
- Code reorg

* Extract code into separate methods

* Add new tests

* Check if message pattern matches the exact length of 'output'

* Rename 'input' to 'hex'

* Add eip-140 test case for REVERT error message

* Remove .vscode settings

* Fixed explanation

* Whitespace
* Change EngineEnabledModules to be comma separated

* Change on holesky_Archive
* Move initializer code in its own file

* Fix missing state root on restart
* Fix old blocks downloaded beyond old barrier

* Fix comment
* Handle `AccessList` differently depending on `TxType`

- Legacy does not have an `AccessList` field
- Others have an `AccessList` field and should be at least an empty array

* Remove all fields except serializer

- Can create issues if we don't properly reset the field

* Use raw strings everywhere

* Use file-scoped namespace

* Replace string comparisons with JSON equivalence

* Formatting

* Fix whitespace

* Remove 'Data' field from 'TransactionForRpc'

- It's a duplicate of 'Input'
- Spec does not define it for any type of Tx

* Update tests

- Removed 'Data' form 'TransactionForRpc'
- Replaced string comparison with JSON equivalence
* Dont stop peer connection on any exception

* Make log debug

* Change to error

* Exclude cancelled

* Run on default scheduler

* Add comment

* Make sure task completion does not leak task scheduler

* Make sure no funny business with tasks
…1706

# Conflicts:
#	scripts/syncSettings.py
#	src/Nethermind/Nethermind.Blockchain/GenesisLoader.cs
#	src/Nethermind/Nethermind.Consensus/Processing/BlockProcessor.cs
#	src/Nethermind/Nethermind.Evm/TransactionExtensions.cs
#	src/Nethermind/Nethermind.Evm/VirtualMachine.cs
#	src/Nethermind/Nethermind.Runner/Properties/launchSettings.json
#	src/Nethermind/Nethermind.Specs/ChainSpecStyle/Json/ChainSpecGenesisJson.cs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.