Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WEB3-322: feat: Add HostEvmEnv::extend method for merging environments #420

Merged
merged 8 commits into from
Feb 1, 2025
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 44 additions & 3 deletions crates/steel/src/host/db/proof.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,15 @@ use alloy::{
transports::Transport,
};
use alloy_primitives::{
map::{hash_map, AddressHashMap, B256HashMap, B256HashSet, HashSet},
map::{hash_map, AddressHashMap, B256HashMap, B256HashSet, HashMap, HashSet},
Address, BlockNumber, Bytes, StorageKey, StorageValue, B256, U256,
};
use anyhow::{ensure, Context, Result};
use revm::{
primitives::{AccountInfo, Bytecode},
Database,
};
use std::hash::{BuildHasher, Hash};

/// A simple revm [Database] wrapper that records all DB queries.
pub struct ProofDb<D> {
Expand All @@ -41,23 +42,28 @@ pub struct ProofDb<D> {
inner: D,
}

#[derive(Clone, Debug, PartialEq, Eq)]
struct AccountProof {
/// The inclusion proof for this account.
account_proof: Vec<Bytes>,
/// The MPT inclusion proofs for several storage slots.
storage_proofs: B256HashMap<StorageProof>,
}

#[derive(Clone, Debug, PartialEq, Eq)]
struct StorageProof {
/// The value that this key holds.
value: StorageValue,
/// In MPT inclusion proof for this particular slot.
proof: Vec<Bytes>,
}

impl<D: Database> ProofDb<D> {
impl<D> ProofDb<D> {
/// Creates a new ProofDb instance, with a [Database].
pub fn new(db: D) -> Self {
pub fn new(db: D) -> Self
where
D: Database,
{
Self {
accounts: Default::default(),
contracts: Default::default(),
Expand All @@ -83,6 +89,15 @@ impl<D: Database> ProofDb<D> {
pub fn inner(&self) -> &D {
&self.inner
}

/// Extends the `ProofDb` with the contents of another `ProofDb`. It panics if they are not
/// consistent.
Comment on lines +93 to +94
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A ProofDb is linked to a single state (commitment) ya? So trying to merge the ProofDb for two different blocks will panic here. I can see how, if written correctly, this will never fail. It also seems like an common enough mistake that panicking is perhaps too extreme, and could result in DoS in cases where e.g. someone is running a service that produces Steel proofs, if an error can cause the service to go down.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it so that the public extend now returns an error if the header config does not match. This could be a user error.
If the actual hash maps are extended, the panic is kept because this should not be possible without fraudulent RPC nodes. Also, this would be unrecoverable since the DB has already been modified.

pub fn extend(&mut self, other: ProofDb<D>) {
extend_checked(&mut self.accounts, other.accounts);
extend_checked(&mut self.contracts, other.contracts);
extend_checked(&mut self.proofs, other.proofs);
self.block_hash_numbers.extend(other.block_hash_numbers);
}
}

impl<T: Transport + Clone, N: Network, P: Provider<T, N>> ProofDb<AlloyDb<T, N, P>> {
Expand Down Expand Up @@ -265,6 +280,32 @@ impl<DB: Database> Database for ProofDb<DB> {
}
}

/// Extends a `HashMap` with the contents of an iterator.
fn extend_checked<K, V, S, T>(map: &mut HashMap<K, V, S>, iter: T)
where
K: Eq + Hash,
V: PartialEq,
S: BuildHasher,
T: IntoIterator<Item = (K, V)>,
{
let iter = iter.into_iter();
let (lower_bound, _) = iter.size_hint();
map.reserve(lower_bound);

for (k, v) in iter {
match map.entry(k) {
hash_map::Entry::Vacant(entry) => {
entry.insert(v);
}
hash_map::Entry::Occupied(entry) => {
if entry.get() != &v {
panic!("mismatching values for key")
}
}
}
}
}

fn filter_existing_keys(account_proof: Option<&AccountProof>) -> impl Fn(&StorageKey) -> bool + '_ {
move |key| {
!account_proof
Expand Down
60 changes: 60 additions & 0 deletions crates/steel/src/host/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,66 @@ impl<D, H: EvmBlockHeader, C> HostEvmEnv<D, H, C> {

self
}

/// Extends the environment with the contents of another compatible environment.
///
/// ### Panics
///
/// Panics if the environments are inconsistent, specifically if:
/// - The configurations don't match
/// - The headers don't match
/// - The database states conflict
///
/// ### Use Cases
///
/// This method is particularly useful for combining results from parallel preflights,
/// allowing you to execute multiple independent operations and merge their environments.
///
/// ### Example
/// ```rust
/// # use risc0_steel::{ethereum::EthEvmEnv, Contract};
/// # use alloy_primitives::address;
/// # use alloy_sol_types::sol;
///
/// # #[tokio::main(flavor = "current_thread")]
/// # async fn main() -> anyhow::Result<()> {
/// # sol! {
/// # interface IERC20 {
/// # function balanceOf(address account) external view returns (uint);
/// # }
/// # }
/// let call =
/// IERC20::balanceOfCall { account: address!("F977814e90dA44bFA03b6295A0616a897441aceC") };
/// # let usdt_addr = address!("dAC17F958D2ee523a2206206994597C13D831ec7");
/// # let usdc_addr = address!("A0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48");
///
/// let url = "https://ethereum-rpc.publicnode.com".parse()?;
/// let builder = EthEvmEnv::builder().rpc(url);
///
/// let mut env1 = builder.clone().build().await?;
/// let mut contract1 = Contract::preflight(usdt_addr, &mut env1);
/// let mut env2 = builder.clone().build().await?;
/// let mut contract2 = Contract::preflight(usdc_addr, &mut env2);
///
/// tokio::join!(contract1.call_builder(&call).call(), contract2.call_builder(&call).call());
///
/// env1.extend(env2);
/// let evm_input = env1.into_input().await?;
///
/// # Ok(())
/// # }
/// ```
pub fn extend(&mut self, other: Self) {
assert_eq!(self.cfg_env, other.cfg_env, "mismatching configuration");
assert_eq!(
self.header.seal(),
other.header.seal(),
"mismatching header"
);
// the commitments do not need to match as long as the cfg_env is consistent

self.db_mut().extend(other.db.unwrap());
}
}

impl<T, N, P, H> HostEvmEnv<AlloyDb<T, N, P>, H, ()>
Expand Down
11 changes: 7 additions & 4 deletions crates/steel/src/verifier.rs
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,10 @@ impl<'a, H: EvmBlockHeader> SteelVerifier<&'a GuestEvmEnv<H>> {
#[cfg(feature = "host")]
mod host {
use super::*;
use crate::host::db::ProofDb;
use crate::{history::beacon_roots, host::HostEvmEnv};
use crate::{
history::beacon_roots,
host::{db::ProofDb, HostEvmEnv},
};
use anyhow::Context;
use revm::Database;

Expand Down Expand Up @@ -119,8 +121,9 @@ mod host {
///
/// It panics if the closure panics.
/// This function is necessary because mutable references to the database cannot be passed
/// directly to `tokio::task::spawn_blocking`. Instead, the database is temporarily taken out of
/// the `HostEvmEnv`, moved into the blocking task, and then restored after the task completes.
/// directly to `tokio::task::spawn_blocking`. Instead, the database is temporarily taken
/// out of the `HostEvmEnv`, moved into the blocking task, and then restored after
/// the task completes.
async fn spawn_with_db<F, R>(&mut self, f: F) -> R
where
F: FnOnce(&mut ProofDb<D>) -> R + Send + 'static,
Expand Down
3 changes: 2 additions & 1 deletion crates/steel/tests/corruption.rs
Original file line number Diff line number Diff line change
Expand Up @@ -421,7 +421,8 @@ mod history {
use super::*;
use test_log::test;

/// Creates `EthEvmInput::History` using live RPC nodes preflighting `IERC20(USDT).balanceOf(0x0)`.
/// Creates `EthEvmInput::History` using live RPC nodes preflighting
/// `IERC20(USDT).balanceOf(0x0)`.
async fn rpc_usdt_history_input() -> anyhow::Result<EthEvmInput> {
let mut env = EthEvmEnv::builder()
.rpc(RPC_URL.parse()?)
Expand Down
Loading