From ab6fa6ae1bbbd1b10071a92ec209b381b5d82511 Mon Sep 17 00:00:00 2001 From: Mamy Ratsimbazafy Date: Mon, 24 Jun 2024 02:22:06 +0200 Subject: [PATCH] docs: add PLANNING.md and update protocol readme --- PLANNING.md | 208 ++++++++++++++++++++++++++++++++++++++++++ constantine/README.md | 85 +++++++++++++++-- 2 files changed, 287 insertions(+), 6 deletions(-) create mode 100644 PLANNING.md diff --git a/PLANNING.md b/PLANNING.md new file mode 100644 index 00000000..2efee2d1 --- /dev/null +++ b/PLANNING.md @@ -0,0 +1,208 @@ +# Constantine's planning + +This document is current as of June 23, 2024. + +This splits Constantine's axis of development under various tracks. +Priority is given to Ethereum, proof systems and optimization tracks. + +Other tracks are stretch goals, contributions towards them are accepted. + +## Table of Contents + + + +- [Constantine's planning](#constantines-planning) + - [Table of Contents](#table-of-contents) + - [Tracks](#tracks) + - [Tech debt track](#tech-debt-track) + - [Ethereum Consensus Track](#ethereum-consensus-track) + - [Ethereum Execution Track](#ethereum-execution-track) + - [Proving Ethereum track](#proving-ethereum-track) + - [Optimization track](#optimization-track) + - [User Experience track](#user-experience-track) + - [Technical marketing track](#technical-marketing-track) + - [ZK and proof systems track](#zk-and-proof-systems-track) + - [Multi-party computation MPC track](#multi-party-computation-mpc-track) + - [Core crypto track](#core-crypto-track) + - [Fully-Homomorphic encryption FHE track](#fully-homomorphic-encryption-fhe-track) + - [Post-Quantum cryptography PQC track](#post-quantum-cryptography-pqc-track) + + + +## Tracks + +### Tech debt track + +- Endomorphism splitting bounds guarantee: i.e. division-based vs lattice-based splitting +- Fix optimized MSM + +Internal API breakage, which might leak to others using internal APIs. + +- Expand the library to support named fields and not just curves + - move and rename the config & constant folder + - use field instead of curve + - "named_math/fields_and_curves" + - matchingOrderBigInt -> BigInt[foo.getOrder()] + - Delete Mod*(), replace by fieldMod + +- Twisted Edwards identity/neutral element are on the curve + and are NOT point at infinity, though they are named like that. + - isInf -> isNeutral + +### Ethereum Consensus Track + +- Implement cryptography and erasure codes EIP-7594 PeerDAS + - https://github.com/mratsim/constantine/issues/341 + - Spec: + - crypto: https://github.com/ethereum/consensus-specs/blob/29d3a24/specs/_features/eip7594/polynomial-commitments-sampling.md + - erasure codes: https://github.com/ethereum/consensus-specs/blob/29d3a24/specs/_features/eip7594/das-core.md#recover_matrix + - executive summary: 2-dimensional data availability sampling for KZG polynomial commitments + - Prerequisites: + - Coset FFT + - KZG multiproofs + - Polynomial interpolation +- Fuzzing + - BLS signatures + - KZG in https://github.com/jtraglia/kzg-fuzz + +- Long-term project, unspecified: + - Secret Shared Leader Election + - Single Slot Finality + - enshrined DVT (distributed validator technology) + +### Ethereum Execution Track + +- Keccak + - with hardware acceleration +- Hash functions precompiles: + - RIPEMD-160, Blake2 +- KZG point precompile +- Verkle Tries + - Finish IPA for Verkle Tries: + - Fix verification outside AddressSanitizer + - Full test suite coverage https://github.com/mratsim/constantine/issues/396 + - Fix multiproofs + - Add IPA and multiproofs to benchmark to compare with other implementations + +- Fast MSM for fixed base like Trusted Setups and Ethreum Verkle Tries + - Notes on MSMs with precomputation https://hackmd.io/WfIjm0icSmSoqy2cfqenhQ + - Verkle Trees - Another iteration of VKTs MSMs https://hackmd.io/@jsign/vkt-another-iteration-of-vkt-msms + +### Proving Ethereum track + +- Proof-of-equivalence for Ethereum KZG: + - https://notes.ethereum.org/@vbuterin/proto_danksharding_faq#Moderate-approach-works-with-any-ZK-SNARK + - https://notes.ethereum.org/@dankrad/kzg_commitments_in_proofs + - https://ethresear.ch/t/easy-proof-of-equivalence-between-multiple-polynomial-commitment-schemes-to-the-same-data/8188 + + - Prerequisites: + - ZK friendly hash function like Poseidon (there are 2 versions !): https://github.com/mratsim/constantine/issues/294 + +- Groth16 + on-chain verifier code-generator (solidity/huff/yul) + +- Long-term project, unspecified: + - Snarkified EVM + +### Optimization track + +- ARM assembly +- Finish Nvidia GPU codegenerator up to MSM +- Implement an unsaturated finite fields backend for Risc-V, WASM, WebGPU, AMD GPU, Apple Metal, Vulkan, ... + - ideally in LLVM IR so that pristine Risc-V assembly can be generated + and used in zkVMs without any risk of C stdlib or syscalls being used + and without depending on the Nim compiler at build time. +- introduce batchAffine_vartime + +### User Experience track + +- Provide curves_sugar and fields_sugar higher-level with + out-of-place functions like `+` and `*`. + - They should be tagged {.inline, noInit.} and just forward to the in-place function + to guarantee copy elision. + - Mention lack of control over stack space + - Guaranteed optimization issue + - with even up to a dramatic 20% perf: + - https://www.reddit.com/r/rust/comments/kfs0oe/comment/ggc0dui/ + - past Constantine large copy bug: + - out-of-place function lead to bad codegen: + - https://github.com/mratsim/constantine/issues/145 + - https://github.com/nim-lang/Nim/issues/16897 + - Mention Rust GCE and NRVO / RVO + - https://github.com/rust-lang/rust/pull/76986 + - https://github.com/rust-lang/rfcs/pull/2884 + +- Shorter names: + - Rename to EC_ShortW + - Rename to EC_TwEdw + +### Technical marketing track + +- Create Python bindings + - provide primitives appealing to cryptography researchers and enabling fast prototyping + +- Create a Constantine benchmark CLI and UI. + - Make it easy-to-use from tools like Phoronix test suite + - Give a single-threaded/multi-threaded, for use in say EthDocker to rank hardware. + - Integrate building it in CI + - Goal: the reference cryptographic benchmark + +- Participate in secp256k1 programming language benchmark: + - https://programming-language-benchmarks.vercel.app/problem/secp256k1 + - outline: https://github.com/mratsim/constantine/issues/285 + +### ZK and proof systems track + +- Transcripts (Halo2, Merlin) + - https://zcash.github.io/halo2/design/implementation/proofs.html + - https://merlin.cool/transcript/index.html + +- SNARKS: + - Polynomial IOP (Interactive Oracle Proof) + Implement BabySpartan (Spartan+Lasso) or Spartan or Spartan2 + + - Lookup Argument + One that commits to only small field elements if the witness contains small field elements + Example: Lasso or LogUp+GKR + + - Multilinear Polynomial Commitment Schemes + For efficiency when commiting to small values (for example coming from bit manipulation in hash functions) + Example: KZG+Gemini/Zeromorph, Dory, Hyrax, Binius, ... + +- STARKS: + - Implement small fields: + - Mersenne31: 2^31-1 + - BabyBear + - Goldilocks + - Optimize small fields with Neon / Avx512 + - Implement FRI and/or STIR + - Prerequisites: + - Erasure codes + - Merkle Trees + +Long-term, unspecified: +- zkML + +### Multi-party computation (MPC) track + +- Implement Shamir Secret Sharing +- Threshold signatures and Distributed Key Generation for DVT (Distributed Validator Technology) + +### Core crypto track + +- Implement NaCl / libsodium API: +- Implement the Signal Protocol: + - https://signal.org/docs/ +- Implement TLSv3: + - https://datatracker.ietf.org/doc/html/rfc8446 +- Json Web Tokens + +### Fully-Homomorphic encryption (FHE) track + +- Implement lattice-based RLWE: Ring-Learning-With-Errors + +Long-term, unspecified: +- Privacy-perserving machine learning + +### Post-Quantum cryptography (PQC) track + +- Implement a lattice-based cryptography scheme diff --git a/constantine/README.md b/constantine/README.md index 1daafb4f..1eede738 100644 --- a/constantine/README.md +++ b/constantine/README.md @@ -2,18 +2,91 @@ This folder stores protocol implemented on top of Constantine. -## Ethereum Virtual Machine + + +- [Constantine-backed protocols](#constantine-backed-protocols) + - [Ethereum Consensus Layer](#ethereum-consensus-layer) + - [BLS signatures](#bls-signatures) + - [Performance](#performance) + - [BLS12-381 Key Derivation for wallets](#bls12-381-key-derivation-for-wallets) + - [KZG commitments for EIP-4844](#kzg-commitments-for-eip-4844) + - [Performance](#performance) + - [Ethereum Execution Layer](#ethereum-execution-layer) + - [Ethereum Virtual Machine](#ethereum-virtual-machine) + - [Performance](#performance) + - [IPA for Verkle Tries](#ipa-for-verkle-tries) + + + +## Ethereum Consensus Layer + +### BLS signatures + +Constantine implements the full BLS signatures used in CL clients. +Batch verification is also parallelized. + +Specs: +- https://github.com/ethereum/consensus-specs/blob/v1.4.0/specs/phase0/beacon-chain.md#bls-signatures +- https://www.ietf.org/archive/id/draft-irtf-cfrg-bls-signature-05.html + +#### Performance + +source, serial bench from https://github.com/mratsim/constantine/pull/279#issuecomment-1746433431 +- 1.19x faster signing than BLST +- 1.25x faster verification + +### BLS12-381 Key Derivation for wallets + +Specs: +- https://eips.ethereum.org/EIPS/eip-2333 + +### KZG commitments for EIP-4844 + +Constantine implements the full Ethereum KZG API in Nim, C, Rust, Go for CL clients. +It is also fully parallelized. + +Specs: +- https://github.com/ethereum/consensus-specs/blob/v1.4.0/specs/deneb/polynomial-commitments.md + +#### Performance + +https://github.com/mratsim/constantine/pull/304#issuecomment-1844795359 + +| Bench | c-kzg-4844 (serial) | go-kzg-4844 (serial) | go-kzg-4844 (parallel) | constantine (serial) | constantine (parallel) | +|:------------------------------:|:-------------------:|:--------------------:|:----------------------:|:--------------------:|:----------------------:| +| blob_to_kzg_commitment | 37.773 ms | - | 5.823 ms | 23.765 ms | 4.425 ms | +| compute_kzg_proof | 39.945 ms | - | 7.146 ms | 24.255 ms | 4.710 ms | +| compute_blob_kzg_proof | 40.212 ms | - | 7.205 ms | 24.288 ms | 4.794 ms | +| verify_kzg_proof | 0.915 ms | 0.923 ms | - | 0.782 ms | - | +| verify_blob_kzg_proof | 1.531 ms | - | 1.390 ms | 1.266 ms | 1.113 ms | +| verify_blob_kzg_proof_batch 1 | 1.528 ms | 1.392 ms | 1.405 ms | 1.286 ms | 1.130 ms | +| verify_blob_kzg_proof_batch 2 | 2.589 ms | 3.233 ms | 1.591 ms | 2.006 ms | 1.152 ms | +| verify_blob_kzg_proof_batch 4 | 4.553 ms | 4.671 ms | 1.914 ms | 3.437 ms | 1.250 ms | +| verify_blob_kzg_proof_batch 8 | 8.446 ms | 7.410 ms | 2.738 ms | 6.115 ms | 1.891 ms | +| verify_blob_kzg_proof_batch 16 | 16.228 ms | 12.734 ms | 3.542 ms | 11.567 ms | 3.091 ms | +| verify_blob_kzg_proof_batch 32 | 32.016 ms | 23.048 ms | 7.215 ms | 21.779 ms | 6.764 ms | +| verify_blob_kzg_proof_batch 64 | 63.415 ms | 43.224 ms | 14.438 ms | 43.099 ms | 11.538 ms | + +## Ethereum Execution Layer + +### Ethereum Virtual Machine Constantine implements precompiles primitives for the Ethereum virtual machine +- SHA256 - ECADD on BN254_Snarks (called `alt_bn128` in Ethereum), address 0x6, spec [EIP-196](https://eips.ethereum.org/EIPS/eip-196) and pricing [EIP-1108](https://eips.ethereum.org/EIPS/eip-1108) - ECMUL on BN254_Snarks (called `alt_bn128` in Ethereum), address 0x7, spec [EIP-196](https://eips.ethereum.org/EIPS/eip-196) and pricing [EIP-1108](https://eips.ethereum.org/EIPS/eip-1108) - ECPAIRING on BN254_Snarks (called `alt_bn128` in Ethereum), address 0x8, spec [EIP-197](https://eips.ethereum.org/EIPS/eip-197) and pricing [EIP-1108](https://eips.ethereum.org/EIPS/eip-1108) +- MODEXP, arbitrary precision modular exponentiation, spec [EIP-198](https://eips.ethereum.org/EIPS/eip-198) +- BLS12-381 precompiles (addition, scalar multiplication, MSM, pairing, hashing-to-curve), spec [EIP-2537](https://eips.ethereum.org/EIPS/eip-2537) + +#### Performance -Their main use-case is for use zero-knowledge proofs and zkRollups and be compatible with work on Zcash +- SHA256 implementation is faster than OpenSSL's for messages less than 65kB: https://github.com/mratsim/constantine/pull/206 + and 16% faster for 32 bytes. +- EIP-2537: https://github.com/mratsim/constantine/pull/368 +- https://ethereum-magicians.org/t/eip-2537-bls12-precompile-discussion-thread/4187/76 -- Succinct Non-Interactive Zero Knowledge -for a von Neumann Architecture\ - Eli Ben-Sasson, Alessandro Chiesa, Eran Tromer, Madars Virza\ - https://eprint.iacr.org/2013/879.pdf +### IPA for Verkle Tries +Those are currently WIP