Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync beacon execution update on demand #1140

Closed
wants to merge 82 commits into from
Closed
Show file tree
Hide file tree
Changes from 81 commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
387e085
Update for Dencun
yrong Sep 21, 2023
846065a
build geth&lodestar from source
yrong Sep 25, 2023
65272d4
Fix makeTrie
yrong Sep 25, 2023
d2f0d66
Merge branch 'main' into ron/dencun
yrong Nov 21, 2023
63b0f07
Update go ethereum to v1.13.5
yrong Nov 21, 2023
faf5d01
Update packages & Speedup e2e setup
yrong Nov 21, 2023
c179311
Update lodestar to v1.12.0
yrong Nov 21, 2023
d728f48
Remove unused
yrong Nov 21, 2023
28f23dd
Remove unrelated changes
yrong Nov 21, 2023
cce7714
Ignore build geth when binary exist
yrong Nov 21, 2023
48cc44f
Cleanup for deprecated ethashproof
yrong Nov 22, 2023
fe1e191
Update with VersionedExecutionPayloadHeader
yrong Nov 22, 2023
d37a736
Fix breaking tests
yrong Nov 22, 2023
1aa5a0a
Merge branch 'main' into ron/dencun
yrong Dec 1, 2023
545e267
Update node package
yrong Dec 1, 2023
1c5b33d
Update git modules
yrong Dec 1, 2023
647f57e
Sync packages with nix
yrong Dec 1, 2023
40daff1
Merge branch 'main' of https://github.com/Snowfork/snowbridge into ro…
yrong Dec 4, 2023
565cd83
Update sdk
yrong Dec 4, 2023
7414746
Update contract binding
yrong Dec 5, 2023
70d017b
More cleanup
yrong Dec 5, 2023
585dc78
For deneb(1)
yrong Dec 5, 2023
ba67bf3
For deneb(2)
yrong Dec 5, 2023
f9f7458
Update sdk
yrong Dec 5, 2023
d542799
For deneb(3)
yrong Dec 5, 2023
904e97a
Update sdk
yrong Dec 5, 2023
99c0304
For deneb(4)
yrong Dec 5, 2023
8a50ced
Enable deneb for local setup
yrong Dec 5, 2023
e5c2381
Fix tests
yrong Dec 5, 2023
29a2afe
Fix BeaconStateDeneb & more tests
yrong Dec 6, 2023
3f6cb38
Merge branch 'main' into ron/dencun
yrong Dec 6, 2023
4aae19a
Generate test fixture for Deneb and more tests accordingly
yrong Dec 7, 2023
79e79dd
Merge branch 'main' into ron/dencun
yrong Dec 7, 2023
ed69a35
Fix parse empty field
yrong Dec 7, 2023
1c1ca20
Setup for switchover test
yrong Dec 7, 2023
8f95e8a
Test workflow specific for deneb
yrong Dec 7, 2023
ba5957e
Ignore change detect building polkadot binary
yrong Dec 11, 2023
9673b21
Encode with error handling
yrong Dec 11, 2023
e767e6e
Fix format
yrong Dec 11, 2023
e5120e8
Merge branch 'main' into ron/dencun
yrong Dec 13, 2023
6a38f79
Update lodestar
yrong Dec 14, 2023
c246f70
Improve scripts setting up nodes for production
yrong Dec 14, 2023
6892678
Merge branch 'main' into ron/dencun
yrong Dec 27, 2023
b76a4dd
Sync execution update on demand
yrong Dec 31, 2023
bdec747
Merge branch 'main' into ron/dencun
yrong Jan 11, 2024
99f5a6d
Fix import
yrong Jan 11, 2024
3580c4d
Reorganize deneb primitives
yrong Jan 11, 2024
d1a5b86
Update go mod
yrong Jan 12, 2024
ad65ebd
Fix format
yrong Jan 12, 2024
53e4fa6
Remove obsolete
yrong Jan 12, 2024
d069f97
Fix import
yrong Jan 12, 2024
da3d338
Merge branch 'main' into ron/dencun
yrong Jan 12, 2024
58a28bd
Merge branch 'main' into ron/dencun
yrong Jan 17, 2024
61dadc9
Fix the deneb ForkVersion
yrong Jan 17, 2024
328d541
Upgrade geth and lodestar
yrong Jan 17, 2024
9d633e2
Merge branch 'ron/dencun' into ron/execution-update-on-demand
yrong Jan 18, 2024
d37ee40
Download geth release to replace the nix default
yrong Jan 18, 2024
d4bc62b
Naming consistently
yrong Jan 18, 2024
0c14e49
Update lodestar with the mock hack
yrong Jan 18, 2024
0ed243a
Fix generate mainnet fixture
yrong Jan 18, 2024
810fa47
Merge branch 'ron/dencun' into ron/execution-update-on-demand
yrong Jan 18, 2024
53c6fb8
Remove fixture unused
yrong Jan 18, 2024
4cdfa96
Backward compatible
yrong Jan 18, 2024
b5ea6b2
Check execution_branch
yrong Jan 18, 2024
6bd6723
Remove build geth
yrong Jan 18, 2024
b892f77
Remove cross check
yrong Jan 18, 2024
a646a4e
sync all lagging periods
yrong Jan 18, 2024
ee71d1a
Clean up
yrong Jan 18, 2024
79247a2
More refactoring
yrong Jan 18, 2024
49800bd
Cleanup scripts
yrong Jan 18, 2024
cdc7165
Update sdk
yrong Jan 18, 2024
072c3f0
Merge branch 'main' into ron/execution-update-on-demand
yrong Jan 19, 2024
c42d0af
Ignore go.work
yrong Jan 19, 2024
5bc2526
Rename function
yrong Jan 19, 2024
f465995
Merge branch 'main' into ron/execution-update-on-demand
yrong Jan 24, 2024
dd866bb
Fix test
yrong Jan 24, 2024
afe2b43
Fix populate checkpoint
yrong Jan 24, 2024
018c565
Fix smoke test
yrong Jan 24, 2024
775e56a
Merge branch 'main' into ron/execution-update-on-demand
yrong Jan 28, 2024
3f0f027
Merge branch 'main' into ron/execution-update-on-demand
yrong Jan 30, 2024
1d67967
Merge branch 'main' into ron/execution-update-on-demand
yrong Feb 6, 2024
7b5e1e3
Update lodestar & polkadot-sdk
yrong Feb 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,4 @@ contracts/beefy-state.json

go/
gocache/
go.work*
2 changes: 1 addition & 1 deletion lodestar
Submodule lodestar updated 848 files
6 changes: 5 additions & 1 deletion parachain/pallets/ethereum-client/fixtures/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1125,6 +1125,8 @@ pub fn make_sync_committee_update() -> Box<Update> {
hex!("5340ad5877c72dca689ca04bc8fedb78d67a4801d99887937edd8ccd29f87e82").into(),
hex!("f5ff4b0c6190005015889879568f5f0d9c40134c7ec4ffdda47950dcd92395ad").into(),
],
execution_header: None,
execution_branch: None,
})
}

Expand Down Expand Up @@ -1165,7 +1167,9 @@ pub fn make_finalized_header_update() -> Box<Update> {
hex!("ecea7e1d3152d8130e83afdfe34b4de4ba2b69a33c9471991096daf454de9cf5").into(),
hex!("b2bf1758e50b2bfff29169fbc70fdb884b2b05bb615dbc53567574da6f4f1ae2").into(),
hex!("cd87069daf70975779126d6af833b7d636c75ca4d5e750ebcad0e76408a5e5bf").into(),
]
],
execution_header: None,
execution_branch: None,
})
}

Expand Down
89 changes: 46 additions & 43 deletions parachain/pallets/ethereum-client/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,7 @@ mod tests;
#[cfg(feature = "runtime-benchmarks")]
mod benchmarking;

use frame_support::{
dispatch::DispatchResult, pallet_prelude::OptionQuery, traits::Get, transactional,
};
use frame_support::{dispatch::DispatchResult, pallet_prelude::OptionQuery, traits::Get};
use frame_system::ensure_signed;
use primitives::{
fast_aggregate_verify, verify_merkle_branch, verify_receipt_proof, BeaconHeader, BlsError,
Expand Down Expand Up @@ -214,8 +212,7 @@ pub mod pallet {
#[pallet::call]
impl<T: Config> Pallet<T> {
#[pallet::call_index(0)]
#[pallet::weight(T::WeightInfo::force_checkpoint())]
#[transactional]
#[pallet::weight((T::WeightInfo::force_checkpoint(), DispatchClass::Operational))]
/// Used for pallet initialization and light client resetting. Needs to be called by
/// the root origin.
pub fn force_checkpoint(
Expand All @@ -234,7 +231,6 @@ pub mod pallet {
Some(_) => T::WeightInfo::submit_with_sync_committee(),
}
})]
#[transactional]
/// Submits a new finalized beacon header update. The update may contain the next
/// sync committee.
pub fn submit(origin: OriginFor<T>, update: Box<Update>) -> DispatchResult {
Expand All @@ -246,7 +242,6 @@ pub mod pallet {

#[pallet::call_index(2)]
#[pallet::weight(T::WeightInfo::submit_execution_header())]
#[transactional]
/// Submits a new execution header update. The relevant related beacon header
/// is also included to prove the execution header, as well as ancestry proof data.
pub fn submit_execution_header(
Expand Down Expand Up @@ -330,32 +325,11 @@ pub mod pallet {
}

pub(crate) fn process_update(update: &Update) -> DispatchResult {
Self::cross_check_execution_state()?;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the cross_check we don't need it any more.

Self::verify_update(update)?;
Self::apply_update(update)?;
Ok(())
}

/// Cross check to make sure that execution header import does not fall too far behind
/// finalised beacon header import. If that happens just return an error and pause
/// processing until execution header processing has caught up.
pub(crate) fn cross_check_execution_state() -> DispatchResult {
let latest_finalized_state =
FinalizedBeaconState::<T>::get(LatestFinalizedBlockRoot::<T>::get())
.ok_or(Error::<T>::NotBootstrapped)?;
let latest_execution_state = Self::latest_execution_state();
// The execution header import should be at least within the slot range of a sync
// committee period.
let max_latency = config::EPOCHS_PER_SYNC_COMMITTEE_PERIOD * config::SLOTS_PER_EPOCH;
ensure!(
latest_execution_state.beacon_slot == 0 ||
latest_finalized_state.slot <
latest_execution_state.beacon_slot + max_latency as u64,
Error::<T>::ExecutionHeaderTooFarBehind
);
Ok(())
}

/// References and strictly follows <https://github.com/ethereum/consensus-specs/blob/dev/specs/altair/light-client/sync-protocol.md#validate_light_client_update>
/// Verifies that provided next sync committee is valid through a series of checks
/// (including checking that a sync committee period isn't skipped and that the header is
Expand Down Expand Up @@ -479,6 +453,28 @@ pub mod pallet {
)
.map_err(|e| Error::<T>::BLSVerificationFailed(e))?;

// Execution payload header corresponding to `beacon.body_root` (from Capella onward)
// https://github.com/ethereum/consensus-specs/blob/dev/specs/capella/light-client/sync-protocol.md#modified-lightclientheader
if let Some(version_execution_header) = &update.execution_header {
let execution_header_root: H256 = version_execution_header
.hash_tree_root()
.map_err(|_| Error::<T>::BlockBodyHashTreeRootFailed)?;
ensure!(
&update.execution_branch.is_some(),
Error::<T>::InvalidExecutionHeaderProof
);
ensure!(
verify_merkle_branch(
execution_header_root,
&update.execution_branch.clone().unwrap(),
config::EXECUTION_HEADER_SUBTREE_INDEX,
config::EXECUTION_HEADER_DEPTH,
update.finalized_header.body_root
),
Error::<T>::InvalidExecutionHeaderProof
);
}

Ok(())
}

Expand Down Expand Up @@ -530,6 +526,19 @@ pub mod pallet {
)?;
}

if let Some(version_execution_header) = &update.execution_header {
let finalized_block_root: H256 = update
.finalized_header
.hash_tree_root()
.map_err(|_| Error::<T>::HeaderHashTreeRootFailed)?;
Self::store_execution_header(
version_execution_header.block_hash(),
version_execution_header.clone().into(),
update.finalized_header.slot,
finalized_block_root,
);
}

Ok(())
}

Expand All @@ -548,15 +557,6 @@ pub mod pallet {
Error::<T>::HeaderNotFinalized
);

// Checks that we don't skip execution headers, they need to be imported sequentially.
let latest_execution_state: ExecutionHeaderState = Self::latest_execution_state();
ensure!(
latest_execution_state.block_number == 0 ||
update.execution_header.block_number() ==
latest_execution_state.block_number + 1,
Error::<T>::ExecutionHeaderSkippedBlock
);

Comment on lines -551 to -559
Copy link
Contributor Author

@yrong yrong Feb 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the consecutive check we don't need it any more.

// Gets the hash tree root of the execution header, in preparation for the execution
// header proof (used to check that the execution header is rooted in the beacon
// header body.
Expand Down Expand Up @@ -709,12 +709,15 @@ pub mod pallet {
block_number
);

LatestExecutionState::<T>::mutate(|s| {
s.beacon_block_root = beacon_block_root;
s.beacon_slot = beacon_slot;
s.block_hash = block_hash;
s.block_number = block_number;
});
let latest_execution_state = LatestExecutionState::<T>::get();
if beacon_slot > latest_execution_state.beacon_slot {
LatestExecutionState::<T>::mutate(|s| {
s.beacon_block_root = beacon_block_root;
s.beacon_slot = beacon_slot;
s.block_hash = block_hash;
s.block_number = block_number;
});
}

Self::deposit_event(Event::ExecutionHeaderImported { block_hash, block_number });
}
Expand Down
114 changes: 2 additions & 112 deletions parachain/pallets/ethereum-client/src/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
use crate::{
functions::compute_period, pallet::ExecutionHeaders, sync_committee_sum, verify_merkle_branch,
BeaconHeader, CompactBeaconState, Error, ExecutionHeaderBuffer, FinalizedBeaconState,
LatestExecutionState, LatestFinalizedBlockRoot, NextSyncCommittee, SyncCommitteePrepared,
LatestFinalizedBlockRoot, NextSyncCommittee, SyncCommitteePrepared,
};

use crate::mock::{
Expand All @@ -18,10 +18,7 @@ pub use crate::mock::*;
use crate::config::{EPOCHS_PER_SYNC_COMMITTEE_PERIOD, SLOTS_PER_EPOCH};
use frame_support::{assert_err, assert_noop, assert_ok};
use hex_literal::hex;
use primitives::{
CompactExecutionHeader, ExecutionHeaderState, Fork, ForkVersions, NextSyncCommitteeUpdate,
VersionedExecutionPayloadHeader,
};
use primitives::{CompactExecutionHeader, Fork, ForkVersions, NextSyncCommitteeUpdate};
use rand::{thread_rng, Rng};
use snowbridge_core::{
inbound::{VerificationError, Verifier},
Expand Down Expand Up @@ -348,34 +345,6 @@ fn find_present_keys() {
});
}

#[test]
fn cross_check_execution_state() {
new_tester().execute_with(|| {
let header_root: H256 = TEST_HASH.into();
<FinalizedBeaconState<Test>>::insert(
header_root,
CompactBeaconState {
// set slot to period 5
slot: ((EPOCHS_PER_SYNC_COMMITTEE_PERIOD * SLOTS_PER_EPOCH) * 5) as u64,
block_roots_root: Default::default(),
},
);
LatestFinalizedBlockRoot::<Test>::set(header_root);
<LatestExecutionState<Test>>::set(ExecutionHeaderState {
beacon_block_root: Default::default(),
// set slot to period 2
beacon_slot: ((EPOCHS_PER_SYNC_COMMITTEE_PERIOD * SLOTS_PER_EPOCH) * 2) as u64,
block_hash: Default::default(),
block_number: 0,
});

assert_err!(
EthereumBeaconClient::cross_check_execution_state(),
Error::<Test>::ExecutionHeaderTooFarBehind
);
});
}

/* SYNC PROCESS TESTS */

#[test]
Expand Down Expand Up @@ -608,40 +577,6 @@ fn submit_update_with_skipped_sync_committee_period() {
});
}

#[test]
fn submit_update_execution_headers_too_far_behind() {
let checkpoint = Box::new(load_checkpoint_update_fixture());
let finalized_header_update = Box::new(load_finalized_header_update_fixture());
let execution_header_update = Box::new(load_execution_header_update_fixture());
let next_update = Box::new(load_next_sync_committee_update_fixture());

new_tester().execute_with(|| {
let far_ahead_finalized_header_slot = finalized_header_update.finalized_header.slot +
(EPOCHS_PER_SYNC_COMMITTEE_PERIOD * SLOTS_PER_EPOCH * 2) as u64;
assert_ok!(EthereumBeaconClient::process_checkpoint_update(&checkpoint));
assert_ok!(EthereumBeaconClient::submit(RuntimeOrigin::signed(1), finalized_header_update));
assert_ok!(EthereumBeaconClient::submit_execution_header(
RuntimeOrigin::signed(1),
execution_header_update
));

let header_root: H256 = TEST_HASH.into();
<FinalizedBeaconState<Test>>::insert(
header_root,
CompactBeaconState {
slot: far_ahead_finalized_header_slot,
block_roots_root: Default::default(),
},
);
LatestFinalizedBlockRoot::<Test>::set(header_root);

assert_err!(
EthereumBeaconClient::submit(RuntimeOrigin::signed(1), next_update),
Error::<Test>::ExecutionHeaderTooFarBehind
);
});
}

#[test]
fn submit_irrelevant_update() {
let checkpoint = Box::new(load_checkpoint_update_fixture());
Expand Down Expand Up @@ -764,51 +699,6 @@ fn submit_execution_header_update_invalid_execution_header_proof() {
});
}

#[test]
fn submit_execution_header_update_that_skips_block() {
let checkpoint = Box::new(load_checkpoint_update_fixture());
let finalized_header_update = Box::new(load_finalized_header_update_fixture());
let execution_header_update = Box::new(load_execution_header_update_fixture());
let mut skipped_block_execution_header_update =
Box::new(load_execution_header_update_fixture());
let mut skipped_execution_header =
skipped_block_execution_header_update.execution_header.clone();

skipped_execution_header = match skipped_execution_header {
VersionedExecutionPayloadHeader::Capella(execution_payload_header) => {
let mut mut_execution_payload_header = execution_payload_header.clone();
mut_execution_payload_header.block_number = execution_payload_header.block_number + 2;
VersionedExecutionPayloadHeader::Capella(mut_execution_payload_header)
},
VersionedExecutionPayloadHeader::Deneb(execution_payload_header) => {
let mut mut_execution_payload_header = execution_payload_header.clone();
mut_execution_payload_header.block_number = execution_payload_header.block_number + 2;
VersionedExecutionPayloadHeader::Deneb(mut_execution_payload_header)
},
};

skipped_block_execution_header_update.execution_header = skipped_execution_header;

new_tester().execute_with(|| {
assert_ok!(EthereumBeaconClient::process_checkpoint_update(&checkpoint));
assert_ok!(EthereumBeaconClient::submit(RuntimeOrigin::signed(1), finalized_header_update));
assert_ok!(EthereumBeaconClient::submit_execution_header(
RuntimeOrigin::signed(1),
execution_header_update.clone()
));
assert!(<ExecutionHeaders<Test>>::contains_key(
execution_header_update.execution_header.block_hash()
));
assert_err!(
EthereumBeaconClient::submit_execution_header(
RuntimeOrigin::signed(1),
skipped_block_execution_header_update
),
Error::<Test>::ExecutionHeaderSkippedBlock
);
});
}

#[test]
fn submit_execution_header_update_that_is_also_finalized_header_which_is_not_stored() {
let checkpoint = Box::new(load_checkpoint_update_fixture());
Expand Down
6 changes: 6 additions & 0 deletions parachain/primitives/beacon/src/types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -450,6 +450,12 @@ impl VersionedExecutionPayloadHeader {
}
}

impl Default for VersionedExecutionPayloadHeader {
fn default() -> Self {
VersionedExecutionPayloadHeader::Deneb(deneb::ExecutionPayloadHeader::default())
}
}

#[cfg(test)]
mod tests {
use super::*;
Expand Down
23 changes: 8 additions & 15 deletions parachain/primitives/beacon/src/updates.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ use sp_std::prelude::*;

use crate::types::{BeaconHeader, SyncAggregate, SyncCommittee, VersionedExecutionPayloadHeader};

#[derive(Encode, Decode, CloneNoBound, PartialEqNoBound, RuntimeDebugNoBound, TypeInfo)]
#[derive(
Default, Encode, Decode, CloneNoBound, PartialEqNoBound, RuntimeDebugNoBound, TypeInfo,
)]
#[cfg_attr(
feature = "std",
derive(serde::Serialize, serde::Deserialize),
Expand All @@ -23,26 +25,13 @@ pub struct CheckpointUpdate<const COMMITTEE_SIZE: usize> {
pub block_roots_branch: Vec<H256>,
}

impl<const COMMITTEE_SIZE: usize> Default for CheckpointUpdate<COMMITTEE_SIZE> {
fn default() -> Self {
CheckpointUpdate {
header: Default::default(),
current_sync_committee: Default::default(),
current_sync_committee_branch: Default::default(),
validators_root: Default::default(),
block_roots_root: Default::default(),
block_roots_branch: Default::default(),
}
}
}

#[derive(
Default, Encode, Decode, CloneNoBound, PartialEqNoBound, RuntimeDebugNoBound, TypeInfo,
)]
#[cfg_attr(
feature = "std",
derive(serde::Deserialize),
serde(deny_unknown_fields, bound(serialize = ""), bound(deserialize = ""))
serde(bound(serialize = ""), bound(deserialize = ""))
)]
pub struct Update<const COMMITTEE_SIZE: usize, const COMMITTEE_BITS_SIZE: usize> {
/// A recent header attesting to the finalized header, using its `state_root`.
Expand All @@ -64,6 +53,10 @@ pub struct Update<const COMMITTEE_SIZE: usize, const COMMITTEE_BITS_SIZE: usize>
pub block_roots_root: H256,
/// The merkle path to prove the `block_roots_root` value.
pub block_roots_branch: Vec<H256>,
/// The execution header to be imported
pub execution_header: Option<VersionedExecutionPayloadHeader>,
/// The merkle proof for the execution_header
pub execution_branch: Option<Vec<H256>>,
}

#[derive(
Expand Down
2 changes: 1 addition & 1 deletion polkadot-sdk
Submodule polkadot-sdk updated 0 files
Loading
Loading