[RK] Batch upstream merge

[x100111010]

This pull request merges updates from the Rusty-Kaspa repository into Rusty-Spectre. It includes several improvements to the mempool, updates for TN-11, as well as enhancements to PSKT/PSST and WRPCv2, as we discussed.

Important Notes and To-Dos:

• currently build fails due to openssl. This will be resolved in a later pull request, which will replace openssl with rusttls
• lints is failing because of unused code in:
  • consensus/core/src/tx/script_public_key.rs
  • crypto/addresses/src/lib.rs

While this can be bypassed this with #[allow(dead_code)], we should investigate why this code is unused and if it's needed.
I have already addressed the Rust 1.81 lints in 052e65c

• get_server_info():

  • broken due to data structure changes

• wRPC v2 Breaking Changes:

  • new public node infrastructure deployment
  • custom serializers for elastic data structure serialization

• enter our own scripted transactions details in (we can already take some details from wallet\psst\examples\multisig.rs and https://explorer.spectre-network.org/txs/76ab09236a7aa13fe8f5389c74545c0110d4bba1ca778e49367d4d8de827834b):

  • consensus/src/processes/transaction_validator/transaction_validator_populated.rs
  • crypto/txscript/src/standard/multisig.rs
  • wallet/psst/src/bundle.rs

• fixed readme layout

kaspanet/rusty-kaspa#499

• replace by fee on mempool 1b0956d

kaspanet/rusty-kaspa#479

• input signature_script checking to submitTransaction 817a3db

kaspanet/rusty-kaspa#512

• hint message for P2P reject reason block not found a7ff129

kaspanet/rusty-kaspa#518

• lazy load origin children 91f8117

kaspanet/rusty-kaspa#513

• O(k log n) mempool transaction sampler + Fee estimation API 93fdc31

kaspanet/rusty-kaspa#519

• update mimalloc dep ec3b064
  • fixes compiling on debian

kaspanet/rusty-kaspa#523

• feat: next_block_template_feerate f24bbb7

kaspanet/rusty-kaspa#526

• semaphore tracing feature (for tracing prune readers vs writers time) 411bc42

kaspanet/rusty-kaspa#476

• TN11 bug fix: activate mass hashing when modifying a cached block template 6b46dc6
  • fix transaction hash merkle root calculation in modify block template to consider storage mass activation
  • avoid similar future errors: expose only a single calc_hash_merkle_root function with include_mass_field arg and update all test usages explicitly
  • move subnet checks to inner location
  • reorganize cache mem estimators

kaspanet/rusty-kaspa#527

• optimizations related to multi-level relations 694da52
  • In ParentsBuilder optimize the execution path of the common case for high levels
  • Prune relations for all levels below the affiliated proof level for block
  • Keep relations for multi-level parents of pruning point and its anticone

kaspanet/rusty-kaspa#530

• query all DNS seeders if missing many connections 4e35e0b

kaspanet/rusty-kaspa#506

• wRPC v2, integrations, typescript-v2, resolver v2 052e65c

  • refactored Borsh serialization (wRPC v2)
  • refactored watch-only
  • fix Rust v1.81 lints
  • using latest resolver APIs (guards)
  • updated TypeScript bindings (typescript-v2)
  • fixes related to "insufficient-funds"
  • Hex View for ScriptBuilder:
  • formatted hex view for scripts

• Kip9 Updates:

  • improvements to mass calculator (Kip9::Beta variant)

New RPC Methods

• get_sync_status():
  • node sync status polling
• get_connection_count():
  • total TCP sockets in use
• get_system_info():
  • unique ID, git commit hash, total RAM, CPU cores, internal FD limit
• get_metrics():
  • provisional dictionary*based map for custom metrics

kaspanet/rusty-kaspa#528

• get_current_block_color RPC utility 8d49801
  • getCurrentBlockColor algorithm and RPC functions
  • move get_current_block_color to consensus and apply standard sorting
  • check for block existence on get_current_block_color
  • add extra DAG order checks to ensure about children
  • Includes:
    1. Stylistic changes using ?
    2. is_dag_ancestor_of(a, b) is different than !is_dag_ancestor_of(b, a)
  • Bug fix:
    1. Hash -> child
    2. Make store calls only where they are actually used (within the if)
  • Style:
    1. Use struct unfolding syntax
    2. Use a name such as decedent which reflects the relation to hash
  • remove Borsh derivations
  • fix gRPC message codes
  • fix gRPC getCurrentBlockColorResponse
  • improve tests

kaspanet/rusty-kaspa#524

• mempool tweaks 1ced37a
  • add case to test_evict
  • remove explicit check for too big transactions
  • raise DEFAULT_MEMPOOL_SIZE_LIMIT to 1GB
  • move to estimated size
  • remove estimated_size field

kaspanet/rusty-kaspa#535

• transaction pool -- fix rare cases where byte size tracking was inaccurate 509e900
  • remove crate-level mutable access to inner mempool tx
  • fix bytes size tracking
  • update update_revalidated_transaction only if validation result is OK
  • remove transaction if validation fails

kaspanet/rusty-kaspa#537

• feerate estimator outliers: capture block slot by the specific mass of the outlier transaction 7ecb3f4
  • deals with outliers (high feerate transactions)
  • captures block space
  • estimates feerate for remaining block space
  • inaccurate average tx mass for outliers
  • fix: Capture block space by specific mass of outlier transaction
  • includes test to verify changes

kaspanet/rusty-kaspa#538

• refactoring of mass calculator utilities c759870
  • rename verbose data mass to compute_mass
  • fix TS interface verbose transaction data property name
  • relocate consensus core processes MassCalculator to consensus core
  • WASM fix ECDSA pubkey generation
  • error variant related to ECDSA from PublicKey creation
  • refactor client mass APIs
    • propagate minimum_signatures via pending tx
    • rename mass calc functions to unsigned for clarity
  • introduce change index to PTX
  • fix missing transfer of client Transaction mass to RpcTransaction
  • introduce IRawBlock and IRawHeader TS types used in GetBlockTemplateResponse and SubmitBlockRequest
  • fix docs
  • refactor mass calculator and extract global storage mass function that operates on iterators + use from wallet mass calculator

[0xA001113]

Great, will create the additional PR for the rest of upstream fixes