Fresh benchmarks

[bkontur]

subweight compare commits \
>          --path-pattern "./system-parachains/bridge-hubs/**/weights/**/*.rs" \
>          --format markdown --no-color  \
>          --change added changed removed  \
>         --method asymptotic --ignore-errors  \
>         --strip-path-prefix="system-parachains/bridge-hubs/" \
>         --threshold 0.5 \
>          remotes/Snowfork/ethereum-headers-on-demand \
>          origin/bko-snowbridge-benchmarks

File	Extrinsic	Old	New	Change [%]
bridge-hub-polkadot/src/weights/snowbridge_pallet_ethereum_client.rs	submit	25.40ms	34.32ms	+35.11
bridge-hub-polkadot/src/weights/snowbridge_pallet_ethereum_client.rs	submit_with_sync_committee	127.21ms	167.58ms	+31.73
bridge-hub-polkadot/src/weights/snowbridge_pallet_ethereum_client.rs	force_checkpoint	102.44ms	133.80ms	+30.61
bridge-hub-kusama/src/weights/snowbridge_pallet_ethereum_client.rs	submit	26.22ms	33.01ms	+25.89
bridge-hub-kusama/src/weights/snowbridge_pallet_ethereum_client.rs	submit_with_sync_committee	131.31ms	162.24ms	+23.56
bridge-hub-kusama/src/weights/snowbridge_pallet_ethereum_client.rs	force_checkpoint	105.80ms	129.70ms	+22.58
bridge-hub-kusama/src/weights/snowbridge_pallet_inbound_queue.rs	submit	481.28us	584.43us	+21.43
bridge-hub-polkadot/src/weights/snowbridge_pallet_inbound_queue.rs	submit	482.86us	585.13us	+21.18

[github-actions]

Review required! Latest push from author must always be reviewed

[claravanstaden]

Thanks for the fresh benchmarks @bkontur! I am concerned about the increase in weight, especially for all the ethereum client extrinsics. They should start more or less the same.

The diff shows that there were mostly minor removals in bridges/snowbridge/parachain/pallets/ethereum-client/src/lib.rs. Have you seen weight increased like this before, for code that didn't really change?

[acatangiu]

@claravanstaden can you run short benchmark for just one of them with old version and new, on your local machine and see if you are getting any change between them?

if so, someone on your team can also "debug" the benchmarking code for said extrinsics and see if it is actual real-world cost change or benchmarking issue

[bkontur]

Thanks for the fresh benchmarks @bkontur! I am concerned about the increase in weight, especially for all the ethereum client extrinsics. They should start more or less the same.

The diff shows that there were mostly minor removals in bridges/snowbridge/parachain/pallets/ethereum-client/src/lib.rs. Have you seen weight increased like this before, for code that didn't really change?

I pushed another commit from different machines, results are a little bit better, but still.

bparity@bkontur-ThinkPad-P14s-Gen-2i:~/parity/polkadot-fellows/runtimes$ subweight compare commits \
>           --path-pattern "./system-parachains/bridge-hubs/**/weights/**/*.rs" \
>           --format markdown --no-color  \
>           --change added changed removed  \
>          --method asymptotic --ignore-errors  \
>          --strip-path-prefix="system-parachains/bridge-hubs/" \
>          --threshold 0.5 \
>           remotes/Snowfork/ethereum-headers-on-demand \
>           origin/bko-snowbridge-benchmarks

File	Extrinsic	Old	New	Change [%]
bridge-hub-polkadot/src/weights/snowbridge_pallet_ethereum_client.rs	submit	25.40ms	31.51ms	+24.03
bridge-hub-kusama/src/weights/snowbridge_pallet_ethereum_client.rs	submit	26.22ms	31.82ms	+21.36
bridge-hub-polkadot/src/weights/snowbridge_pallet_ethereum_client.rs	submit_with_sync_committee	127.21ms	152.93ms	+20.22
bridge-hub-kusama/src/weights/snowbridge_pallet_inbound_queue.rs	submit	481.28us	577.03us	+19.90
bridge-hub-polkadot/src/weights/snowbridge_pallet_inbound_queue.rs	submit	482.86us	578.29us	+19.76
bridge-hub-polkadot/src/weights/snowbridge_pallet_ethereum_client.rs	force_checkpoint	102.44ms	122.11ms	+19.20
bridge-hub-kusama/src/weights/snowbridge_pallet_ethereum_client.rs	submit_with_sync_committee	131.31ms	154.03ms	+17.31
bridge-hub-kusama/src/weights/snowbridge_pallet_ethereum_client.rs	force_checkpoint	105.80ms	123.00ms	+16.25

I see that your benchmarks were regenerated the last time here:

-//! DATE: 2024-03-25, STEPS: `50`, REPEAT: `20`, LOW RANGE: `[]`, HIGH RANGE: `[]`
+//! DATE: 2024-07-01, STEPS: `50`, REPEAT: `20`, LOW RANGE: `[]`, HIGH RANGE: `[]`
 //! WORST CASE MAP SIZE: `1000000`
-//! HOSTNAME: `ggwpez-ref-hw`, CPU: `Intel(R) Xeon(R) CPU @ 2.60GHz`
+//! HOSTNAME: `bkontur-ref-hw`, CPU: `AMD EPYC 7232P 8-Core Processor`

Previously we used some google cloud VM instance, but we don't have them anymore. We have new baremetal servers.

[claravanstaden]

@bkontur @acatangiu thanks for running the benchmarks again! It looks like the diff might be from moving the hash tree root of the finalized header: https://github.com/paritytech/polkadot-sdk/compare/e046d3b2caaca6c43a60f26594052c5c806b980e..6dba07c0bc0c14c2caa33dc8fc385c849175f6d5#diff-98c747bd18857fe76701570ede9bc3bef5350ed0f9808daea2af1e769c1a8effR502

I'm going to merge the PR.

[yrong]

can you run short benchmark for just one of them with old version and new, on your local machine and see if you are getting any change between them?

Run against old version and the new on same machine, seems no difference.

File	Extrinsic	Old	New	Change [%]
bridge-hub-polkadot/src/weights/snowbridge_pallet_ethereum_client.rs	submit	35.886ms	35.813ms	-0.2
bridge-hub-polkadot/src/weights/snowbridge_pallet_ethereum_client.rs	submit_with_sync_committee	181.850ms	181.573ms	-0.1

[claravanstaden]

@yrong you're right, I saw the same when I tested locally. So might be the machine difference. Thanks for checking. 🙏🏻