Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create benchmark #14916

Merged
merged 1 commit into from
Jan 15, 2025
Merged

Create benchmark #14916

merged 1 commit into from
Jan 15, 2025

Conversation

runtian-zhou
Copy link
Contributor

@runtian-zhou runtian-zhou commented Oct 9, 2024

Description

Added new transaction types for APT token transfers, including a permissioned transfer option that uses the permissioned signer framework. This enables both standard APT transfers and permissioned transfers through the transaction generator and e2e benchmark framework.

How Has This Been Tested?

The functionality is integrated into the existing e2e benchmark framework and transaction generator test infrastructure. The new transfer types are added to the test suite alongside other transaction types.

Key Areas to Review

  • New Move module permissioned_transfer.move implementing both standard and permissioned transfer functions
  • Integration of new transaction types into the entry points enum and transaction type arguments
  • Permissioned transfer implementation using the permissioned signer framework

Type of Change

  • New feature
  • Tests

Which Components or Systems Does This Change Impact?

  • Testing Infrastructure

Checklist

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

Copy link

trunk-io bot commented Oct 9, 2024

⏱️ 1h 59m total CI duration on this PR
Slowest 15 Jobs Cumulative Duration Recent Runs
rust-targeted-unit-tests 24m 🟥
rust-move-tests 11m 🟩
check-dynamic-deps 10m 🟩🟩🟩🟩🟩 (+2 more)
rust-cargo-deny 10m 🟩🟩🟩🟩🟩 (+1 more)
rust-move-tests 10m 🟥
rust-move-tests 10m 🟩
rust-move-tests 10m 🟩
rust-move-tests 10m 🟥
rust-move-tests 10m 🟩
file_change_determinator 5m 🟩🟩🟩🟩🟥 (+1 more)
rust-lints 4m 🟥
general-lints 3m 🟩🟩🟩🟩🟩 (+1 more)
semgrep/ci 2m 🟩🟩🟩🟩🟩 (+1 more)
rust-check-merge-base 1m 🟩
permission-check 21s 🟩🟩🟩🟩🟩 (+1 more)

🚨 2 jobs on the last run were significantly faster/slower than expected

Job Duration vs 7d avg Delta
file_change_determinator 4m 11s +1821%
check-dynamic-deps 4m 1m +204%

settingsfeedbackdocs ⋅ learn more about trunk.io

Copy link
Contributor Author

runtian-zhou commented Oct 9, 2024

This stack of pull requests is managed by Graphite. Learn more about stacking.

@runtian-zhou runtian-zhou mentioned this pull request Oct 9, 2024
21 tasks
@rahxephon89 rahxephon89 force-pushed the 09-11-permission_for_framework branch from 43f2715 to d468098 Compare October 9, 2024 21:45
@runtian-zhou runtian-zhou force-pushed the 09-11-permission_for_framework branch from d468098 to 43f2715 Compare October 9, 2024 21:48
@runtian-zhou runtian-zhou force-pushed the 10-09-create_benchmark branch 3 times, most recently from 6c002e4 to 44e71be Compare October 9, 2024 23:23
@runtian-zhou runtian-zhou force-pushed the 09-11-permission_for_framework branch from 43f2715 to e036793 Compare October 15, 2024 19:04
@runtian-zhou runtian-zhou force-pushed the 10-09-create_benchmark branch from 44e71be to 9161080 Compare October 15, 2024 19:04
@runtian-zhou runtian-zhou force-pushed the 09-11-permission_for_framework branch from e036793 to e4239a8 Compare October 15, 2024 20:05
@runtian-zhou runtian-zhou force-pushed the 10-09-create_benchmark branch from 9161080 to b728e81 Compare October 15, 2024 20:06
@runtian-zhou runtian-zhou force-pushed the 09-11-permission_for_framework branch from e4239a8 to d92055d Compare November 15, 2024 20:31
@runtian-zhou runtian-zhou force-pushed the 10-09-create_benchmark branch 2 times, most recently from ef3c832 to e2fcd91 Compare November 18, 2024 21:33
@runtian-zhou runtian-zhou force-pushed the 09-11-permission_for_framework branch from d92055d to 881aa71 Compare November 18, 2024 21:56
@runtian-zhou runtian-zhou force-pushed the 10-09-create_benchmark branch from e2fcd91 to 4caabaf Compare November 18, 2024 21:56
@runtian-zhou runtian-zhou force-pushed the 09-11-permission_for_framework branch from 881aa71 to e569dd2 Compare November 18, 2024 22:01
@runtian-zhou runtian-zhou force-pushed the 10-09-create_benchmark branch from 4caabaf to e5d6e5d Compare November 18, 2024 22:01
@runtian-zhou runtian-zhou force-pushed the 09-11-permission_for_framework branch from e569dd2 to 7723f05 Compare November 18, 2024 22:18
@runtian-zhou runtian-zhou force-pushed the 10-09-create_benchmark branch from e5d6e5d to bf6221c Compare November 18, 2024 22:18
@runtian-zhou runtian-zhou force-pushed the 09-11-permission_for_framework branch from 7723f05 to 54bc94c Compare November 19, 2024 18:13
@runtian-zhou runtian-zhou force-pushed the 10-09-create_benchmark branch from bf6221c to 115f8f6 Compare November 19, 2024 18:13
@runtian-zhou runtian-zhou force-pushed the 09-11-permission_for_framework branch from 54bc94c to 5d15334 Compare November 19, 2024 18:39
@runtian-zhou runtian-zhou force-pushed the 10-09-create_benchmark branch from 115f8f6 to 124c67e Compare November 19, 2024 18:39
@runtian-zhou runtian-zhou force-pushed the 09-11-permission_for_framework branch from d287494 to 939ead6 Compare January 15, 2025 07:08
@runtian-zhou runtian-zhou force-pushed the 10-09-create_benchmark branch from b4f354a to 2f74ae7 Compare January 15, 2025 07:08
@runtian-zhou runtian-zhou force-pushed the 09-11-permission_for_framework branch from 939ead6 to f6d98cc Compare January 15, 2025 07:35
@runtian-zhou runtian-zhou force-pushed the 10-09-create_benchmark branch from 2f74ae7 to b082a3c Compare January 15, 2025 07:35
Copy link
Contributor

@georgemitenkov georgemitenkov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks for adding! You need to regenerate package/module bytes though, to get this included in the benchmark?

  1. Regenerate framework_usacases package:
    // Run "cargo run -p module-publish" to generate the file `raw_module_data.rs`.
  2. Add a test to check it works (manually is ok):
    fn test_native_vm_benchmark_transaction() {

@igor-aptos should know more about this workflow

@@ -78,6 +78,8 @@ pub enum TransactionTypeArg {
SmartTablePicture1MWith1KChangeExceedsLimit,
DeserializeU256,
SimpleScript,
PermissionedTransfer,
APTTransfer,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few things:

  • Use Apt like in other enum variants
  • How is this related to AptFaTransfer? Isn't it the same? I am fine having both, I think it would be nice to unify transfer under one use cases: coin, fa, permissioned, possibly veiled in the future

crates/transaction-workloads-lib/src/args.rs Outdated Show resolved Hide resolved
@@ -232,6 +232,8 @@ pub enum EntryPoints {
/// there to slow down deserialization & verification, effectively making it more expensive to
/// load it into code cache.
SimpleScript,
APTPermissionedTransfer,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can we add these variants next to other transfers?

crates/transaction-workloads-lib/src/move_workloads.rs Outdated Show resolved Hide resolved
@runtian-zhou runtian-zhou force-pushed the 10-09-create_benchmark branch 2 times, most recently from 5135b7b to d74365f Compare January 15, 2025 15:25
@runtian-zhou runtian-zhou changed the base branch from 09-11-permission_for_framework to 09-04-add_permission_check_to_account January 15, 2025 15:25
@runtian-zhou runtian-zhou force-pushed the 09-04-add_permission_check_to_account branch from 9b05820 to 1230cf4 Compare January 15, 2025 15:34
@runtian-zhou runtian-zhou force-pushed the 10-09-create_benchmark branch 2 times, most recently from 8514dc2 to 5d7f12a Compare January 15, 2025 17:44
@runtian-zhou runtian-zhou changed the base branch from 09-04-add_permission_check_to_account to main January 15, 2025 17:45
Comment on lines +13 to +16
primary_fungible_store::grant_apt_permission(source, &permissioned_signer, amount);
aptos_account::transfer(&permissioned_signer, to, amount);

permissioned_signer::destroy_permissioned_handle(handle);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The permissioned_signer retains its APT transfer permissions after the transfer completes. To maintain proper permission hygiene, call primary_fungible_store::revoke_apt_permission(source, &permissioned_signer) before destroying the handle. This ensures no permissions persist beyond their intended use.

Spotted by Graphite Reviewer

Is this helpful? React 👍 or 👎 to let us know.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

@runtian-zhou runtian-zhou force-pushed the 10-09-create_benchmark branch from de618cd to aa7cdfe Compare January 15, 2025 18:25
@runtian-zhou runtian-zhou force-pushed the 10-09-create_benchmark branch from aa7cdfe to 145eaa1 Compare January 15, 2025 18:31

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite compat success on 6593fb81261f25490ffddc2252a861c994234c2a ==> 145eaa1572a47d7890905e5091bb8208a6d74503

Compatibility test results for 6593fb81261f25490ffddc2252a861c994234c2a ==> 145eaa1572a47d7890905e5091bb8208a6d74503 (PR)
1. Check liveness of validators at old version: 6593fb81261f25490ffddc2252a861c994234c2a
compatibility::simple-validator-upgrade::liveness-check : committed: 17425.66 txn/s, latency: 2010.22 ms, (p50: 2100 ms, p70: 2100, p90: 2200 ms, p99: 3300 ms), latency samples: 559840
2. Upgrading first Validator to new version: 145eaa1572a47d7890905e5091bb8208a6d74503
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 4853.00 txn/s, latency: 6352.19 ms, (p50: 7100 ms, p70: 7700, p90: 7900 ms, p99: 8000 ms), latency samples: 97680
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 4773.62 txn/s, latency: 7237.42 ms, (p50: 7900 ms, p70: 8000, p90: 8000 ms, p99: 8200 ms), latency samples: 172620
3. Upgrading rest of first batch to new version: 145eaa1572a47d7890905e5091bb8208a6d74503
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 4837.69 txn/s, latency: 6307.39 ms, (p50: 7200 ms, p70: 7600, p90: 8000 ms, p99: 8200 ms), latency samples: 98460
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 4930.11 txn/s, latency: 6997.11 ms, (p50: 7700 ms, p70: 7800, p90: 8100 ms, p99: 8400 ms), latency samples: 175020
4. upgrading second batch to new version: 145eaa1572a47d7890905e5091bb8208a6d74503
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 8704.40 txn/s, latency: 3498.67 ms, (p50: 4000 ms, p70: 4300, p90: 4600 ms, p99: 4800 ms), latency samples: 159160
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 8838.30 txn/s, latency: 3865.75 ms, (p50: 4100 ms, p70: 4400, p90: 4500 ms, p99: 4800 ms), latency samples: 291920
5. check swarm health
Compatibility test for 6593fb81261f25490ffddc2252a861c994234c2a ==> 145eaa1572a47d7890905e5091bb8208a6d74503 passed
Test Ok

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on 145eaa1572a47d7890905e5091bb8208a6d74503

two traffics test: inner traffic : committed: 14854.24 txn/s, latency: 2674.38 ms, (p50: 2700 ms, p70: 2700, p90: 2900 ms, p99: 3200 ms), latency samples: 5647880
two traffics test : committed: 100.00 txn/s, latency: 1349.30 ms, (p50: 1300 ms, p70: 1400, p90: 1500 ms, p99: 1600 ms), latency samples: 1700
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 1.618, avg: 1.527", "ConsensusProposalToOrdered: max: 0.296, avg: 0.288", "ConsensusOrderedToCommit: max: 0.305, avg: 0.297", "ConsensusProposalToCommit: max: 0.597, avg: 0.585"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.86s no progress at version 40610 (avg 0.19s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.52s no progress at version 2654968 (avg 0.52s) [limit 16].
Test Ok

@runtian-zhou runtian-zhou merged commit c7452c7 into main Jan 15, 2025
46 checks passed
@runtian-zhou runtian-zhou deleted the 10-09-create_benchmark branch January 15, 2025 19:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants