RFC: clean up repos

[0xaatif]

Here's a table of all the Rust repos we have in our org.
I think this surface area is too big. If we have active code, let's keep it in our active repos, where we can keep it current, tested, audited etc. Otherwise let's archive it.
e.g smt_utils, eth_trie_tools should actually just be (crates or modules) in the zk_evm repo, where there's already trie diffing stuff.

Here's a strawman proposal

proposal	repo	stale (months)	description
archive or move to plonky	plonky2-u32	6	u32 arithmetic gadgets for plonky2
archive or move to plonky2	CircleStark 🔒	9	Plonky3 fork for circle stark development
archive or move to plonky2	plonky-edge-block-trace-parser	12	Logic for parsing a block trace for edge into txn generation payloads for plonky (2 & 3).
archive or move to plonky2	plonky2-ecdsa	13	ECDSA gadgets for plonky2
archive or move to zk_evm	eth_trie_utils	9	Types and utility functions for building/working with partial Ethereum tries
archive or move to zk_evm	eth-trie-tools	5	Various debugging tools for working with tries for eth_trie_utils.
archive or move to zk_evm	eth-tx-proof	3	Proving historical Ethereum transactions using the Plonky2 zkEVM
archive or move to zk_evm	proof-protocol-decoder	9	Common protocol for generating ZK proofs for blocks on different blockchains.
archive or move to zk_evm	smt_utils	12	Utilities related to the Sparse Merkle Tree used for type2 provers
archive? who uses this?	evm-tests	1	Parser and test runner for testing compatable common Ethereum full node tests against Polygon Zero's EVM.
archive? who uses this?	fast-mds-check 🔒	8	Code to check if a matrix has the MDS property.
archive? who uses this?	nums	5	Number theoretic functions, such as primality testing and factorization, for BigUints
archive	plonky2-insertion	20	list-insertion gadget for plonky2
archive	plonky2-waksman	20	permutation gadgets using the Waksman network for plonky2
archive	system-zero	19	An obsolete zkVM from Polygon Zero
archive	zero-provers 🔒	13	Prover clients for Polygon Zero
keep	paladin	1	Bringing divine order to remote task execution.
keep	plonky2	1
keep	zk_evm	1
keep archived	hyperstark 📥	18	Early prototype.
keep archived	plonky-block-proof-gen 📥	10	Generates block proofs from proof IR
keep archived	zero-bin 📥	5	plonky2 proving over paladin

Bootstrapping script

#!/usr/bin/env python

import json
import subprocess
import sys
from datetime import datetime
from dateutil.rrule import rrule, MONTHLY
from dateutil.parser import parse as parse_date
from typing import TypedDict, Literal

def eprintln(s: str):
    print(s, file = sys.stderr)

class Repo(TypedDict):
    description: str
    isArchived: bool
    isFork: bool
    isMirror: bool
    name: str
    pushedAt: str
    url: str
    visibility: Literal["PUBLIC"] | Literal["PRIVATE"] | Literal["INTERNAL"]


completed = subprocess.run(
    [
        "gh", "repo", "list", "0xPolygonZero",
        "--json", "description,isArchived,isFork,isMirror,name,primaryLanguage,pushedAt,url,visibility",
        "--jq", '[ .[] | select(.primaryLanguage.name == "Rust") | del(.primaryLanguage) ]'
    ],
    check=True,
    capture_output=True,
)

now = datetime.now()

repos: list[Repo] = json.loads(completed.stdout)
eprintln(f"{len(repos)=}")


print("| repo | stale (months) | description |")
print("|------|----------------|-------------|")

for repo in repos:
    name = " ".join([
        f"[{repo['name']}]({repo['url']})",
        "".join([
            "📥" if repo["isArchived"] else "",
            "🍴" if repo["isFork"] else "",
            "🪞" if repo["isMirror"] else "",
            "🔒" if repo["visibility"] in ("PRIVATE, INTERNAL") else "",
        ])
    ])
    months = sum(1 for _ in rrule(MONTHLY, dtstart=parse_date(repo["pushedAt"]).date(), until=now.date()))
    description = repo["description"]

    print(f"| {name}| {months} | {description} |")

Clone all repos

#/usr/bin/env bash
set -euxo pipefail

pushd 0xPolygonZero
    gh repo list 0xPolygonZero \
        --limit 1000 \
        --json 'sshUrl,primaryLanguage' \
        --jq '.[] | select(.primaryLanguage.name == "Rust") | .sshUrl' \
        | while read -r ssh_url _; do
            git clone "$ssh_url" &
        done
    wait
popd

[Nashtare]

A few notes:

• The plonky2-foo repos were initially part of the plonky2 workspace and then moved away into individual crates, though mostly outdated / unmaintained as of today.
• CircleStark is unrelated to plonky2, could eventually be moved to Plonky3 github org (cc @dlubarov)
• plonky-edge-block-trace-parser, eth_trie_utils, eth-tx-proof, proof-protocol-decoder, smt_utils and zero-provers can (should) definitely be archived
• evm-tests should probably be kept. @praetoriansentry created a matrix of all tests and deployed them in a test chain, but we still use them for easy, reliable testing and easier debugging + make it a testing reference to anyone who wants to check completeness of the zkEVM (these tests are the standard for EVM implementations).

[BGluth]

Just to add a few other notes:

• eth_trie_utils ended up becoming mpt_trie in this repo, so yeah we can archive this.
• I'm a bit out of the loop, but eth_trie_tools may have been partially integrated into zk_evm (have all 3 tools been moved over?).