Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deferred calls #4723

Draft
wants to merge 102 commits into
base: mainnet_2_3
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
102 commits
Select commit Hold shift + click to select a range
19e89b4
Add structure async call and async call id
AurelienFT May 9, 2024
4407e86
progress
damip May 9, 2024
b6948d0
progress
damip May 9, 2024
bcfaed3
todos
damip May 9, 2024
eb59c84
Improve general behavior
damip May 10, 2024
5315ea4
typo
damip May 10, 2024
d4300c3
reformat registry
damip May 10, 2024
60cbf28
make things compile
damip May 10, 2024
42c47cd
updates
damip May 21, 2024
37a1961
Merge remote-tracking branch 'aurelien/new_asc' into deferred_calls
modship Jul 8, 2024
94016cb
update new interface
modship Jul 8, 2024
d62dbee
deferred calls first step
modship Jul 9, 2024
d18ed1b
rename massa_asc package to massa_deferred_calls
modship Jul 9, 2024
8b3b2da
TDD for ser_deser validation
modship Jul 10, 2024
7d7e850
Test(DeferredRegistrySlotChanges) : Ser/deser
modship Jul 10, 2024
f945386
Test(DeferredRegistryChanges) : Ser/Deser
modship Jul 10, 2024
9814537
update StateChanges Deserializer
modship Jul 10, 2024
90202d3
create macros, put_entry , delete_entry on DB
modship Jul 12, 2024
698a033
refactor + get_total_gas
modship Jul 12, 2024
cfe824a
get_slot_calls && get_slot_base_fee
modship Jul 15, 2024
1a86834
add mod tests / deferred_call_quote
modship Jul 16, 2024
9d74423
get_deferred_call_quote (interface_impl)
modship Jul 16, 2024
e1f2eca
register call
modship Jul 17, 2024
1bcc026
register set gas
modship Jul 17, 2024
14ca8d5
deferred_call_cancel
modship Jul 17, 2024
4a0c7db
execute deferred call, advance_slot, snapshot
modship Jul 24, 2024
ee901f1
execution trace && execution info
modship Jul 24, 2024
693cf0e
delete call from DB
modship Jul 24, 2024
43923d1
slot remaining gas for async msg
modship Jul 25, 2024
e94c57f
in case of fail, refound sender
modship Jul 25, 2024
6297693
get_slot from DeferredCallId
modship Jul 25, 2024
8bbb687
Fix(ser/deser): error and update test
modship Jul 26, 2024
801e65c
Fix(exec_deferred_call) : Fix lock
modship Jul 26, 2024
d59a556
Test(deferred_call) : first execution test
modship Jul 26, 2024
cd44aa1
fix(deferred_call) : remove double delete_call
modship Jul 26, 2024
cc14cdc
Test(abi) : First test for register deferred_call
modship Jul 29, 2024
1668f4d
Test(register) : more assert
modship Jul 30, 2024
24e09fe
Fix(set_total_gas) : fix set / keep
modship Jul 31, 2024
64f6c24
Test(deferred_call) : Register call fail (total_gas)
modship Jul 31, 2024
9a022ef
Fix(execution) : cancel deferred on check target sc + test refound
modship Jul 31, 2024
08bbef2
update wasm file
modship Jul 31, 2024
9fb3c0e
Fix(deferred_call) : fix compute call fee
modship Aug 1, 2024
82938ec
Fix(deferred_call) : some fix
modship Aug 1, 2024
3ce418f
Test(deferred_call) : Add slot base fee
modship Aug 1, 2024
baa1c3a
rename DEFERRED_CALLS_SLOT_PREFIX to DEFERRED_CALLS_PREFIX
modship Aug 14, 2024
87bb3a2
deferred calls ser/deser error msg
modship Aug 14, 2024
1dcab48
deferred_call_id deserializer limit
modship Aug 14, 2024
ac7ccde
models error
modship Aug 14, 2024
ebe33e1
update max_async_gas for async msg
modship Aug 16, 2024
12971b5
remove useless condition
modship Aug 16, 2024
1d61d14
overbooking_fee return Result
modship Aug 16, 2024
8f82d3e
fix operator
modship Aug 16, 2024
6c1caec
add penalty constant
modship Aug 16, 2024
f3331b5
pr comment # 1
modship Aug 26, 2024
9664ca7
Update massa-execution-worker/src/context.rs
modship Aug 26, 2024
a08664e
pr comment # 2
modship Aug 26, 2024
b53c77f
create DeferredCallsConfig
modship Aug 26, 2024
91ac5e3
use config struct
modship Aug 27, 2024
fe347bb
clippy
modship Aug 27, 2024
07863fc
allow unused_macros
modship Aug 27, 2024
5f28f44
fix : massa-bootstrap import
modship Aug 27, 2024
fd69b2e
wrap deferred call execution into a closure
modship Aug 28, 2024
67ad55f
update sc runtime && wasm test file
modship Aug 28, 2024
6815f44
update interface params from bytes to string
modship Aug 29, 2024
db87465
first metrics
modship Aug 29, 2024
d49dda7
Test: Add check on total_gas after execution
modship Aug 29, 2024
f43fdea
Test : Add 2nd call in scenario and check total_gas
modship Aug 30, 2024
5a58f5f
move type Set Update Delete from Ledger to massa_models
modship Sep 2, 2024
3f5f3cc
remove todo
modship Sep 3, 2024
4df7053
add vm allocation gas cost
modship Sep 4, 2024
c8ca2a8
remove todo
modship Sep 5, 2024
3dcd4ab
Fee: calc relu_occupancy_before
modship Sep 5, 2024
b5dae7c
update sc runtime
modship Sep 5, 2024
89c4d1d
add abi gas cost
modship Sep 5, 2024
039e9ba
Fix : total gas booked
modship Sep 6, 2024
562c16f
BugFix && add nb_call_registered
modship Sep 18, 2024
9583fc7
API : Jsonrpc DeferredCallQuote
modship Sep 20, 2024
341db02
API : JsonRpc get_deferred_call_info && list_deferred_calls_by_slot
modship Sep 23, 2024
4f7524d
API : openrpc spec
modship Sep 24, 2024
5b68c00
grpc api
modship Sep 26, 2024
815ed4c
Fix grpc log error
modship Sep 27, 2024
fd9d945
API : Update list call ids by slot
modship Sep 27, 2024
ed88731
rename openrpc method
modship Sep 27, 2024
7cd95b3
Fix : reset_to_snapshot
modship Oct 1, 2024
08be8a9
Fix: reset_to_snapshot
modship Oct 1, 2024
bc225ec
FIX : Api double alloc vm cst
modship Oct 2, 2024
a4e21a3
Fix : compute_fee get_effective_total_gas()
modship Oct 2, 2024
a4f3a6d
Update runtime
modship Oct 2, 2024
34d9975
Fix abi gas cost
modship Oct 2, 2024
9e57b53
Test : compute_call_fee setup && first basic test
modship Oct 2, 2024
6d25ee6
Test : add test for deferred call exists
modship Oct 3, 2024
2aea5a0
clippy
modship Oct 3, 2024
46fccbe
Fix : fee calculation
modship Oct 7, 2024
8ebf906
add storage cost for deferred call
modship Oct 16, 2024
2af6725
Merge remote-tracking branch 'origin/mainnet_2_3' into deferred_calls
modship Oct 18, 2024
88ecb0d
Storage: Refund sender (storage cost for params)
modship Oct 18, 2024
ff28375
Test : deferred call quote
modship Oct 21, 2024
64f4cdb
update runtime
modship Oct 21, 2024
95e1bc8
Fix all feature && update runtime
modship Oct 21, 2024
c8e45d8
fix doc
modship Oct 21, 2024
2453a80
Update storage cost for deferred calls and set MAX_FUNCTION_NAME_LENG…
modship Oct 30, 2024
1ef9d71
Fix comment
modship Oct 30, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -107,8 +107,8 @@ massa_versioning = { path = "./massa-versioning" }
massa_wallet = { path = "./massa-wallet" }

# Massa projects dependencies
massa-proto-rs = { git = "https://github.com/massalabs/massa-proto-rs", "rev" = "1cce8ec75e0df66af6ce3c5a86350adaea8ac463" }
massa-sc-runtime = { git = "https://github.com/massalabs/massa-sc-runtime", "rev" = "f5fd64c9e05c6d7cc01a30d1ef1dc0c202456a4b" }
massa-proto-rs = { git = "https://github.com/massalabs/massa-proto-rs", "rev" = "67c331c10a1a3f0a9985b3b16c50e31d63074b83" }
massa-sc-runtime = { git = "https://github.com/massalabs/massa-sc-runtime", "rev" = "7289367b2cb490ae1c6c74190d6d36b8d65b30cf" }


peernet = { git = "https://github.com/massalabs/PeerNet", "rev" = "04b05ddd320fbe76cc858115af7b5fc28bdb8310" }
Expand Down
2 changes: 2 additions & 0 deletions massa-api-exports/src/execution.rs
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,8 @@ pub struct DeferredCallsQuoteRequest {
pub target_slot: Slot,
/// The maximum gas requested.
pub max_gas_request: u64,
/// Size of parameters
pub params_size: u64,
}

/// The response to a request for a deferred call quote.
Expand Down
1 change: 1 addition & 0 deletions massa-api/src/public.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1169,6 +1169,7 @@ impl MassaRpcServer for API<Public> {
.map(|call| ExecutionQueryRequestItem::DeferredCallQuote {
target_slot: call.target_slot,
max_gas_request: call.max_gas_request,
params_size: call.params_size,
})
.collect();

Expand Down
5 changes: 4 additions & 1 deletion massa-deferred-calls/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ use massa_models::{
DEFERRED_CALL_GLOBAL_OVERBOOKING_PENALTY, DEFERRED_CALL_MAX_ASYNC_GAS,
DEFERRED_CALL_MAX_FUTURE_SLOTS, DEFERRED_CALL_MAX_POOL_CHANGES, DEFERRED_CALL_MIN_GAS_COST,
DEFERRED_CALL_MIN_GAS_INCREMENT, DEFERRED_CALL_SLOT_OVERBOOKING_PENALTY,
MAX_FUNCTION_NAME_LENGTH, MAX_PARAMETERS_SIZE, THREAD_COUNT,
LEDGER_COST_PER_BYTE, MAX_FUNCTION_NAME_LENGTH, MAX_PARAMETERS_SIZE, THREAD_COUNT,
},
};
use serde::Deserialize;
Expand Down Expand Up @@ -37,6 +37,8 @@ pub struct DeferredCallsConfig {
pub max_pool_changes: u64,

pub max_gas: u64,

pub ledger_cost_per_byte: Amount,
}

impl Default for DeferredCallsConfig {
Expand All @@ -54,6 +56,7 @@ impl Default for DeferredCallsConfig {
global_overbooking_penalty: DEFERRED_CALL_GLOBAL_OVERBOOKING_PENALTY,
slot_overbooking_penalty: DEFERRED_CALL_SLOT_OVERBOOKING_PENALTY,
call_cst_gas_cost: DEFERRED_CALL_CST_GAS_COST,
ledger_cost_per_byte: LEDGER_COST_PER_BYTE,
}
}
}
3 changes: 2 additions & 1 deletion massa-execution-exports/src/mapping_grpc.rs
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,8 @@ pub fn to_querystate_filter(
"target slot is required".to_string(),
))?
.into(),
max_gas_request: value.max_gas_request,
max_gas_request: value.max_gas,
params_size: value.params_size,
})
}
exec::RequestItem::DeferredCallInfo(info) => {
Expand Down
2 changes: 2 additions & 0 deletions massa-execution-exports/src/types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,8 @@ pub enum ExecutionQueryRequestItem {
target_slot: Slot,
/// gas request
max_gas_request: u64,
/// params size
params_size: u64,
},
/// get info of deferred calls
DeferredCallInfo(DeferredCallId),
Expand Down
11 changes: 8 additions & 3 deletions massa-execution-worker/src/context.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1140,15 +1140,20 @@ impl ExecutionContext {
self.speculative_deferred_calls.advance_slot(current_slot)
}

/// Get the price it would cost to reserve "gas" at target slot "slot".
/// Get the price it would cost to reserve "gas" with params at target slot "slot".
pub fn deferred_calls_compute_call_fee(
&self,
target_slot: Slot,
max_gas_request: u64,
current_slot: Slot,
params_size: u64,
) -> Result<Amount, ExecutionError> {
self.speculative_deferred_calls
.compute_call_fee(target_slot, max_gas_request, current_slot)
self.speculative_deferred_calls.compute_call_fee(
target_slot,
max_gas_request,
current_slot,
params_size,
)
}

pub fn deferred_call_register(
Expand Down
7 changes: 6 additions & 1 deletion massa-execution-worker/src/controller.rs
Original file line number Diff line number Diff line change
Expand Up @@ -334,8 +334,13 @@ impl ExecutionController for ExecutionControllerImpl {
ExecutionQueryRequestItem::DeferredCallQuote {
target_slot,
max_gas_request,
params_size,
} => {
let result = execution_lock.deferred_call_quote(target_slot, max_gas_request);
let result = execution_lock.deferred_call_quote(
target_slot,
max_gas_request,
params_size,
);
Ok(ExecutionQueryResponseItem::DeferredCallQuote(
result.0, result.1, result.2, result.3,
))
Expand Down
8 changes: 7 additions & 1 deletion massa-execution-worker/src/execution.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2442,12 +2442,18 @@ impl ExecutionState {
&self,
target_slot: Slot,
max_request_gas: u64,
params_size: u64,
) -> (Slot, u64, bool, Amount) {
let gas_request =
max_request_gas.saturating_add(self.config.deferred_calls_config.call_cst_gas_cost);
let context = context_guard!(self);

match context.deferred_calls_compute_call_fee(target_slot, gas_request, context.slot) {
match context.deferred_calls_compute_call_fee(
target_slot,
gas_request,
context.slot,
params_size,
) {
Ok(fee) => (target_slot, gas_request, true, fee),
Err(_) => (target_slot, gas_request, false, Amount::zero()),
}
Expand Down
11 changes: 9 additions & 2 deletions massa-execution-worker/src/interface_impl.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1345,6 +1345,7 @@ impl Interface for InterfaceImpl {
&self,
target_slot: (u64, u8),
gas_limit: u64,
params_size: u64,
) -> Result<(bool, u64)> {
// write-lock context

Expand All @@ -1357,7 +1358,12 @@ impl Interface for InterfaceImpl {
let gas_request =
gas_limit.saturating_add(self.config.deferred_calls_config.call_cst_gas_cost);

match context.deferred_calls_compute_call_fee(target_slot, gas_request, current_slot) {
match context.deferred_calls_compute_call_fee(
target_slot,
gas_request,
current_slot,
modship marked this conversation as resolved.
Show resolved Hide resolved
params_size,
) {
Ok(fee) => Ok((true, fee.to_raw())),
Err(_) => Ok((false, 0)),
}
Expand Down Expand Up @@ -1402,7 +1408,8 @@ impl Interface for InterfaceImpl {
}

// check fee, slot, gas
let (available, fee_raw) = self.get_deferred_call_quote(target_slot, max_gas)?;
let (available, fee_raw) =
self.get_deferred_call_quote(target_slot, max_gas, params.len() as u64)?;
if !available {
bail!("The Deferred call cannot be registered. Ensure that the target slot is not before/at the current slot nor too far in the future, and that it has at least max_gas available gas.");
}
Expand Down
73 changes: 70 additions & 3 deletions massa-execution-worker/src/speculative_deferred_calls.rs
Original file line number Diff line number Diff line change
Expand Up @@ -422,6 +422,7 @@ impl SpeculativeDeferredCallRegistry {
target_slot: Slot,
max_gas_request: u64,
current_slot: Slot,
params_size: u64,
) -> Result<Amount, ExecutionError> {
// Check that the slot is not in the past
if target_slot <= current_slot {
Expand Down Expand Up @@ -450,6 +451,12 @@ impl SpeculativeDeferredCallRegistry {
));
}

if params_size > self.config.max_parameter_size.into() {
return Err(ExecutionError::DeferredCallsError(
"Parameters size is too big.".into(),
));
}

// We perform Dynamic Pricing of slot gas booking using a Proportional-Integral controller (https://en.wikipedia.org/wiki/Proportional–integral–derivative_controller).
// It regulates the average slot async gas usage towards `target_async_gas` by adjusting fees.

Expand Down Expand Up @@ -486,10 +493,22 @@ impl SpeculativeDeferredCallRegistry {
self.config.slot_overbooking_penalty, // total_initial_coin_supply/10000
)?;

// Storage cost for the parameters
let storage_cost = self
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add total cost of call

.config
.ledger_cost_per_byte
.checked_mul_u64(params_size)
.ok_or_else(|| {
ExecutionError::DeferredCallsError(
"overflow when calculating storage cost".to_string(),
)
})?;

// return the fee
Ok(integral_fee
.saturating_add(global_overbooking_fee)
.saturating_add(slot_overbooking_fee))
.saturating_add(slot_overbooking_fee)
.saturating_add(storage_cost))
}

/// Register a new call
Expand Down Expand Up @@ -562,13 +581,13 @@ impl SpeculativeDeferredCallRegistry {

#[cfg(test)]
mod tests {
use std::sync::Arc;
use std::{str::FromStr, sync::Arc};

use massa_db_exports::{MassaDBConfig, MassaDBController};
use massa_db_worker::MassaDB;
use massa_deferred_calls::{config::DeferredCallsConfig, DeferredCallRegistry};
use massa_final_state::MockFinalStateController;
use massa_models::{config::THREAD_COUNT, slot::Slot};
use massa_models::{amount::Amount, config::THREAD_COUNT, slot::Slot};
use parking_lot::RwLock;
use tempfile::TempDir;

Expand Down Expand Up @@ -627,6 +646,7 @@ mod tests {
period: 1,
thread: 1,
},
1_000
)
.is_err());

Expand All @@ -642,6 +662,7 @@ mod tests {
period: 5,
thread: 1,
},
1000
)
.is_err());

Expand All @@ -658,9 +679,55 @@ mod tests {
period: 1,
thread: 1,
},
1000
)
.is_err());

// params too big
assert!(speculative
.compute_call_fee(
good_slot,
1_000_000,
Slot {
period: 1,
thread: 1,
},
50_000_000
)
.is_err());

// no params
assert_eq!(
speculative
.compute_call_fee(
good_slot,
200_000,
Slot {
period: 1,
thread: 1,
},
0,
)
.unwrap(),
Amount::from_str("0.000000079").unwrap()
);

// 10Ko params size
assert_eq!(
speculative
.compute_call_fee(
good_slot,
200_000,
Slot {
period: 1,
thread: 1,
},
10_000,
)
.unwrap(),
Amount::from_str("1.000000079").unwrap()
);

// TODO : add more tests with different values and check the results of the fee
}
}
6 changes: 5 additions & 1 deletion massa-node/base_config/openrpc.json
Original file line number Diff line number Diff line change
Expand Up @@ -2155,14 +2155,18 @@
"type": "object",
"required": [
"target_slot",
"max_gas_request"
"max_gas_request",
"params_size"
],
"properties": {
"target_slot": {
"$ref": "#/components/schemas/Slot"
},
"max_gas_request": {
"type": "number"
},
"params_size": {
"type": "number"
}
}
},
Expand Down
1 change: 1 addition & 0 deletions massa-node/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,7 @@ async fn launch(
global_overbooking_penalty: DEFERRED_CALL_GLOBAL_OVERBOOKING_PENALTY,
slot_overbooking_penalty: DEFERRED_CALL_SLOT_OVERBOOKING_PENALTY,
call_cst_gas_cost: DEFERRED_CALL_CST_GAS_COST,
ledger_cost_per_byte: LEDGER_COST_PER_BYTE,
};
let final_state_config = FinalStateConfig {
ledger_config: ledger_config.clone(),
Expand Down
Loading