[block-stm] Do not set incorrect use on storage errors

[georgemitenkov]

Description

When there is a storage error, propagate it up with alert, but do not set incorrect use to avoid fallback to sequential and scary invariant violation logs.

How Has This Been Tested?

Key Areas to Review

Type of Change

• New feature
• Bug fix
• Breaking change
• Performance improvement
• Refactoring
• Dependency update
• Documentation update
• Tests

Which Components or Systems Does This Change Impact?

• Validator Node
• Full Node (API, Indexer, etc.)
• Move/Aptos Virtual Machine
• Aptos Framework
• Aptos CLI/SDK
• Developer Infrastructure
• Move Compiler
• Other (specify)

Checklist

• I have read and followed the CONTRIBUTING doc
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I identified and added all stakeholders and component owners affected by this change as reviewers
• I tested both happy and unhappy path of the functionality
• I have made corresponding changes to the documentation

[trunk-io]

⏱️ 58m total CI duration on this PR

Slowest 15 Jobs	Cumulative Duration	Recent Runs
execution-performance / single-node-performance	29m	🟥 🟩
rust-doc-tests	7m	🟩
test-target-determinator	5m	🟩
execution-performance / test-target-determinator	5m	🟩
check-dynamic-deps	5m	🟩 🟩
rust-cargo-deny	3m	🟩 🟩
fetch-last-released-docker-image-tag	2m	🟩
general-lints	55s	🟩 🟩
semgrep/ci	49s	🟩 🟩
file_change_determinator	24s	🟩 🟩
file_change_determinator	11s	🟩
permission-check	5s	🟩 🟩
permission-check	4s	🟩
permission-check	4s	🟩 🟩
determine-docker-build-metadata	3s	🟩

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

[georgemitenkov]

• [block-stm] Do not set incorrect use on storage errors #15855 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[github-actions]

✅ Forge suite compat success on 60f7ca8827f5d64a148c3b163dc4126b0879279b ==> 953d3e8e30df256cb976c6252891f21e656528bb

Compatibility test results for 60f7ca8827f5d64a148c3b163dc4126b0879279b ==> 953d3e8e30df256cb976c6252891f21e656528bb (PR)
1. Check liveness of validators at old version: 60f7ca8827f5d64a148c3b163dc4126b0879279b
compatibility::simple-validator-upgrade::liveness-check : committed: 12610.53 txn/s, latency: 2489.35 ms, (p50: 2500 ms, p70: 2700, p90: 2800 ms, p99: 3000 ms), latency samples: 410440
2. Upgrading first Validator to new version: 953d3e8e30df256cb976c6252891f21e656528bb
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 4262.46 txn/s, latency: 7335.65 ms, (p50: 8100 ms, p70: 8700, p90: 9200 ms, p99: 9400 ms), latency samples: 91560
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 4257.54 txn/s, latency: 7992.50 ms, (p50: 8900 ms, p70: 9100, p90: 9300 ms, p99: 9500 ms), latency samples: 149600
3. Upgrading rest of first batch to new version: 953d3e8e30df256cb976c6252891f21e656528bb
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 4316.01 txn/s, latency: 7184.70 ms, (p50: 8000 ms, p70: 8600, p90: 9000 ms, p99: 9100 ms), latency samples: 92200
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 4291.32 txn/s, latency: 7933.48 ms, (p50: 8900 ms, p70: 9000, p90: 9300 ms, p99: 9300 ms), latency samples: 151280
4. upgrading second batch to new version: 953d3e8e30df256cb976c6252891f21e656528bb
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 7939.40 txn/s, latency: 3880.69 ms, (p50: 4500 ms, p70: 4700, p90: 4800 ms, p99: 4900 ms), latency samples: 147000
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 7634.48 txn/s, latency: 4446.49 ms, (p50: 4800 ms, p70: 4900, p90: 5000 ms, p99: 5100 ms), latency samples: 259380
5. check swarm health
Compatibility test for 60f7ca8827f5d64a148c3b163dc4126b0879279b ==> 953d3e8e30df256cb976c6252891f21e656528bb passed
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite realistic_env_max_load success on 953d3e8e30df256cb976c6252891f21e656528bb

two traffics test: inner traffic : committed: 14284.25 txn/s, submitted: 14291.35 txn/s, expired: 7.10 txn/s, latency: 2756.43 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 3600 ms), latency samples: 5431540
two traffics test : committed: 99.97 txn/s, latency: 1454.83 ms, (p50: 1400 ms, p70: 1500, p90: 1700 ms, p99: 2200 ms), latency samples: 1980
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 1.528, avg: 1.398", "ConsensusProposalToOrdered: max: 0.308, avg: 0.299", "ConsensusOrderedToCommit: max: 0.460, avg: 0.440", "ConsensusProposalToCommit: max: 0.762, avg: 0.739"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.87s no progress at version 13607 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.74s no progress at version 6157068 (avg 0.73s) [limit 16].
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite framework_upgrade success on 60f7ca8827f5d64a148c3b163dc4126b0879279b ==> 953d3e8e30df256cb976c6252891f21e656528bb

Compatibility test results for 60f7ca8827f5d64a148c3b163dc4126b0879279b ==> 953d3e8e30df256cb976c6252891f21e656528bb (PR)
Upgrade the nodes to version: 953d3e8e30df256cb976c6252891f21e656528bb
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1609.13 txn/s, submitted: 1613.96 txn/s, failed submission: 4.62 txn/s, expired: 4.83 txn/s, latency: 1757.82 ms, (p50: 1600 ms, p70: 2000, p90: 2700 ms, p99: 4200 ms), latency samples: 146321
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1504.85 txn/s, submitted: 1510.13 txn/s, failed submission: 5.28 txn/s, expired: 5.28 txn/s, latency: 1898.06 ms, (p50: 1800 ms, p70: 2100, p90: 3100 ms, p99: 4200 ms), latency samples: 136903
5. check swarm health
Compatibility test for 60f7ca8827f5d64a148c3b163dc4126b0879279b ==> 953d3e8e30df256cb976c6252891f21e656528bb passed
Upgrade the remaining nodes to version: 953d3e8e30df256cb976c6252891f21e656528bb
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1924.93 txn/s, submitted: 1931.74 txn/s, failed submission: 6.81 txn/s, expired: 6.81 txn/s, latency: 1480.65 ms, (p50: 1500 ms, p70: 1700, p90: 2100 ms, p99: 3000 ms), latency samples: 175221
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking