Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Getrandom issue even though local execution works #9109

Closed
drewstone opened this issue Jun 15, 2021 · 15 comments
Closed

Getrandom issue even though local execution works #9109

drewstone opened this issue Jun 15, 2021 · 15 comments

Comments

@drewstone
Copy link
Contributor

drewstone commented Jun 15, 2021

We've pushed some new runtime modules to our testnet and are getting some new errors in execution. These modules deal with zero-knowledge verification and hashing. In particular, an OsRng is instantiated to seed random values for the verifier here.

Perhaps it is related to --wasm-execution Compiled that we have some nodes running with though we're a bit unsure. Any ideas? We're seeing errors such as:

Jun 14 23:49:13 berlin6 run_node.sh[22489]: 2021-06-14 23:49:13.677 ERROR tokio-runtime-worker runtime: panicked at 'Error: getrandom: this target is not supported', /cargo-home/registry/src/github.com-1ecc6299db9ec823/rand_core-0.5.1/src/os.rs:63:13

An example node execution is:

target/release/edgeware \
    --base-path /mnt/blockstorage/edgeware-db/edgeware \
    --node-key=$KEY \
    --chain=$CHAINSPEC \
    --validator \
    --unsafe-rpc-external \
    --unsafe-ws-external \
    --rpc-cors="all" \
    --name $NODE_NAME \
    --wasm-execution Compiled \
    -lrpc=trace

The system works locally and without problems. Transactions occur in a timely fashion however on the live testnet they take up to 30 seconds and potentially 60 seconds while slowing the chain down throughout the process. Any insight or ways to debug would be appreciated.

@drewstone
Copy link
Contributor Author

I'll try out --wasm-execution interpreted-i-know-what-i-do and then without it to see if we achieve any speedup. I was under the assumption that our usage of getrandom was compatible in certain WASM environments.

@drewstone
Copy link
Contributor Author

drewstone commented Jun 15, 2021

@burdges maybe you have an idea, inspecting old discussion here using the same transcripting system?

@rphmeier
Copy link
Contributor

Hey Drew,

The Wasm execution of the runtime needs to be completely deterministic, so getrandom should not be allowed in any scenario. The real issue here is that it works at all in any case.

@drewstone
Copy link
Contributor Author

Ok, glad that hopefully this provides good feedback @rphmeier. How then could I reasonably instantiate an prng for the verification? I was thinking it would be great if the RandomnessCoinFlip module could implement the RngCore + CryptoRng traits since I think those can work in no_std. Any thoughts on this? Or is this lib basically unusable unless I expose host functions to do the job.

Beyond that, I'm guessing that even if I use host functions that this functionality would never work in a parachain runtime right?

@drewstone
Copy link
Contributor Author

Or at least if anyone knows of a prng module that can take a deterministic random seed through the RandomnessCoinFlip module and provide these traits so that verification can take place in a pallet.

@burdges
Copy link

burdges commented Jun 15, 2021

Do not use RandomnessCoinFlip in production. It's basically not secure for anything.. just useful for tests.

@burdges
Copy link

burdges commented Jun 15, 2021

What is the actual problem? Is this randomness fora proover or a verifier? As a rule, one cannot run proovers from PVFs anyways, but native off chain workers could run them just fine. Assuming it's a batch verifier.. I've not looked into bulletproofs closely recently, and never too closely at the batching, but usually protocols with random batching could actually be deterministically batched via Fiat-Shamir. It requires being careful however..

@rphmeier
Copy link
Contributor

rphmeier commented Jun 15, 2021

If you need a PRNG you could run ChaCha32 on the BABE VRF or epoch randomness. But this is public info.

@drewstone
Copy link
Contributor Author

This is randomness for a verifier @burdges and we're using Aura so we don't have access to any BABE randomness @rphmeier. I hear you that that randomness may not be terribly secure, but does it really matter if I just need to instantiate a PRNG for verifier randomness?

@burdges
Copy link

burdges commented Jun 15, 2021

It's verifiers I see.. We should discuss this with someone who looked more closely at bulletproofs, but..

I doubt VRFs help here, he needs Fiat-Shamir.

It's likely they're using merlin's transcript rng builder, which takes another Rng+CryptoRng, so folks give it OsRng. At this stage, the transcript much be built identically in the proover and the verifier, so maybe it maybe lacks the complete picture. It's likely save to do roughly

let mut seed_block_hash = [u8; 32]::default();
merlin::Transcript::new(b"foo")  // or whatever hash you like
.append_message(b"bar", all_transactions_bodies_with_proof.as_bytes())
.challenge_bytes(b"baz", &mut seed);
let mut fs_rng = ChaChaRng::from_seed(seed_block_hash);

You then pass fs_rng into bulletproof so it gets used in code that looks like

let mut rng = transcript
    .build_rng()
    .rekey_with_witness_bytes(b"witness1", witness_data)
    .rekey_with_witness_bytes(b"witness2", more_witness_data)
    .finalize(&mut fs_rng);

What this is doing is making sure that the whole all_transactions_bodies_with_proof aka block hash gets taken into account in the Fiat-Shamir transform.

@burdges
Copy link

burdges commented Jun 15, 2021

Anyways your question is: How do I derandomize a bulletproofs batch verifier with Fiat-Shamir?

It probably just works but I cannot dig into the bullet proofs paper right now.

@drewstone
Copy link
Contributor Author

Ok, I'm not using the batch verifier or range proofs, rather using their R1CS interfaces. I will read through this and continue to debug.. I think I'm finding ChaCha to be in the right direction too.

Thanks for the feedback so quickly!

@zhenfeizhang
Copy link

Ok, glad that hopefully this provides good feedback @rphmeier. How then could I reasonably instantiate an prng for the verification? I was thinking it would be great if the RandomnessCoinFlip module could implement the RngCore + CryptoRng traits since I think those can work in no_std. Any thoughts on this? Or is this lib basically unusable unless I expose host functions to do the job.

Beyond that, I'm guessing that even if I use host functions that this functionality would never work in a parachain runtime right?

I think I may have encountered this bug before.
IIRC, my solution was to use ark_std::rand::{RngCore, SeedableRng};. It worked for arkworks version 0.2.0 (whose rand was a wrapper for rand version 0.7.x), rand_chacha version 0.2.0.
Not sure this will solve your problem.

@burdges
Copy link

burdges commented Jun 16, 2021

I'm not using the batch verifier or range proofs, rather using their R1CS interfaces

I think provers require real randomness for zero-knowledge so if you hit https://doc.dalek.rs/src/bulletproofs/r1cs/prover.rs.html#343 then you'll need it to run native on the parachain nodes.

You've presumably hit https://doc.dalek.rs/src/bulletproofs/r1cs/verifier.rs.html#374 so likely you could fork the bulletproofs crate and make Verifier::verify take an impl Rng or some seed for ChaChaRng, and then derive the seed from the block hash, or maybe just the transaction in the unbatched case.

Actually Verifier::verify never touches self.transcript after this point, so maybe even ChaChaRng::from_seed([083; 32]) turns out secure, but do not do this because we're really not sure. ;)

I only say likely because I've never checked whether this turns out secure. It's ages since I've looked into Bootle's paper or the bulletproofs paper that adds batching. In fact, it'll depend somewhat upon what your circuit actually does. @AlistairStewart could tell you stories about smart contracts that compute their own hash, but that's a story for another day.

You're likely find if all your doing is simple asset transfers, not scripting ala smart contracts.

@bkchr
Copy link
Member

bkchr commented Oct 18, 2022

I think this can be closed, if not please reopen.

@bkchr bkchr closed this as completed Oct 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants