From a9048b2f19e321abcdd3c14c3a3c9095ec5edbc4 Mon Sep 17 00:00:00 2001 From: Paul Lange Date: Wed, 2 Aug 2023 15:46:06 +0200 Subject: [PATCH] Add `celo-migrate` script This works by loading the database of a celo node. It then removes all existing blocks and generates a new genesis block including the existing state tree. Migrate to urfave/cli/v2 Update op-chain-ops/cmd/op-migrate/main.go Co-authored-by: Karl Bartel Combine Cel2 migration scripts (#148) * Initial script to play with celo DB history migration * Can Read All the headers Co-authored-by: Alec Schaefer * Adds new command to migrate ancients db * Adds comment * Adds extension methods for transformation * Implements Transform CeloBody * Adds impl that runs steps in a concurrent pipeline * Adds transformHead, verify hashing works cleanup * add migration for non-frozen blocks * copy over entire db and modify in place, works with op-geth at piersy/minimal-data-migration * remove unecessary copying, cleanup code * close and reopen DBs * migrate newdb in place * saving progress Co-authored-by: Mariano Cortesi * Refactor code to improve database migration process * better logging * refactor: inline parMigrateAncientRange * Remove frozen blocks from nonAncient DB * check hash matches on nonAncients migration * clean up branch Removes unused code, move code for better separation of concerns. * decode into new types * fix transformHeader * make old freezer not readonly so that .meta files are created * add configurable memory limit * add comment about memory * Added celo-dbmigrate Makefile target * Added dockerfile for celo-dbmigrate and celo-migrate tools * Workflow for running cel2-migration-tool * Update cel2-migration-tool image registry * update op-geth to point to https://github.com/celo-org/op-geth/commits/piersy/for-use-with-migrated-celo-datadir-use-gas-limit-differentiation-rebased-celo6/ * add celo6 logging * rename scripts to celo-migrate-state and celo-migrate-blocks * first pass at combining scripts * saving progress on testing * fix lint error, use %w to fmt errors * add updated state migration input files to testdata * add ability to run block and state migration seperately or together * add option for migrating only frozen blocks * remove old scripts * minor logging improvements in block migrations * invert clearNonAncients flag logic --> keepNonAncients, make dry-run flag only apply to state migration * adds README, improves logging * fix lint err * Fix Makefile and Dockerfile * move createNewDbIfNotExists * rename keep-non-ancients * update TODO to add more context and state changes * Remove channel buffers from ancients migration Co-authored-by: Valentin Rodygin * bump default batch size to 100000 * add back extended usage string * add info on state migration to README * remove --state-dry-run flag * update default batch size to 50k * Adding building for op images * Setting our values for image registry and repository * update README * fix logging when newAncients > oldAncients * fix return value when skipping ancients * skip transforming block bodies that have already been transformed * misc. fixes to get re-runs with --keep-non-ancients working * adds TODO * addresses cosmetic feedback * add flag for specifying a buffer * Show progress on rsync * Update to latest op-geth * state-migration: Refactor subtask * state-migration: Use EIP1559 settings from deploy config Fixes https://github.com/celo-org/optimism/issues/135 * state-migration: Enable Fjord hardfork during migration Fixes https://github.com/celo-org/optimism/issues/160 * state-migration: Deterministicly set migration block timestamp Fixes https://github.com/celo-org/optimism/issues/157 Sets the timestamp to be 5s after the last block. * state-migration: Set WithdrawalsHash in Cel2 migration block * fixup! Fix Makefile and Dockerfile * add note to README about using snapshots for pre-migration * Set blob gas header fields for transition block These are now required to be set since cancun was activated. * Use InitialBaseFee for pre-gingerbread transitionb * Fix warnings about capitalized error strings * Output chain config as marshalled JSON * state-migration: Handle accounts with existing balance Fixes https://github.com/celo-org/optimism/issues/158 * remove allocs file, add instructions for how to generate allocs file to README, update TODOs --------- Co-authored-by: Mariano Cortesi Co-authored-by: Alec Schaefer Co-authored-by: Mariano Cortesi Co-authored-by: Javier Cortejoso Co-authored-by: Paul Lange Co-authored-by: Valentin Rodygin Co-authored-by: Piers Powlesland Set balance of `CeloDistributionSchedule` contract (#162) * state-migration: Initialize CeloDistributionSchedule Fixes https://github.com/celo-org/optimism/issues/155 * state-migration: Don't fail when distribution schedule update errors * Review comments state-migration: Set ParentBeaconRoot (#176) This allows header validation to pass during snap sync state-migration: Set address of distribution schedule (#177) state-migration: Read total supply directly from state (#182) * state-migration: Read totalSupply directly from storage * Added trigger for updated dependencies * Removen token bindings --------- Co-authored-by: Javier Cortejoso Fix l2 block older than l1 origin error (#184) (#187) * Revert to using time.Now() for migration block Instead of simply adding 5 to the parent block time. We really do need a deterministic time for the migration block so that all parties that run the migration arrive at the same migration block but the problem is that op-geth requires that the L2 migration block (aka l2 origin) occurs after the l1 origin (I guess the point where you deploy the bridge contracts to the l1). When we migrate a partially synced datadir the block before the transition block will be very old, up to 4 years old! So of course it occurs before the l1 origin. So a fix just to get things working is to use time.Now(), but probably we should make this a configurable parameter. * add flag to specify timestamp * Update op-chain-ops/cmd/celo-migrate/main.go --------- Co-authored-by: piersy Migration script fixes (#179) * Fixed migration for datadirs without ancients The script was assuming that ancients would have been migrated and was considering the numAncients-1 to be the next block to migrate but when numAncients is zero that's a problem. Also remved logic for picking up where db migration left of for the level db since it was complicating the logic and that process takes a few seconds, which is nothing compared with the minutes taken to migrate the ancients. * Ensure that we set gas limit if migrating at pre-gingerbread point Fix migration script gap in migrated blocks (#189) * Fix migration script gap in migrated blocks The range of ancient blocks to remove from the non ancients database was off by one and resulted in a gap between ancients and non ancients. Also corrected some log statements that were off by one. Add pre-migration command to migration script (#192) * add pre-migration command, rsync and ancients run in parallel, remove onlyAncients flag * remove block and state migration sub-commands * make non ancient migration its own step, add flag to measure time * add more granular timers * open db without freezer in state migration, remove clearAll * fix error * remove update flag from rsync command, add rsync comments * delete commented out versions of checkForPrevFullMigration * remove aliases * remove clearNonAncients flag * remove measureTime flag, always log time measurements * remove logging from help text * remove db reset * move scan for extra ancients into pre-migration * update README * rename extraAncientNumHashes to strayAncientBlocks state-migration: Fail if account would be overwritten (#202) * state-migration: Fail if account would be overwritten * Review changes * Review changes 2 * Fail in unclear state * more changes * Use whitelist to decide if nonce and state are overwritten Cosmetic changes to the migration script - Use more lists for added readability - Capitalize Alfajores and Celo - Reorder scripting instructions to fit the actual order or operations - Use GitHub callouts migration: Add tests (#217) * migration: Add tests for state migration * migration: Fix issues shown by tests * migration: pass allowlist into state migration Allows for easier testing * migration: Add test with allowlist * Correct overwrite counter * Use in memory DB migration: Add working allowlist for Alfajores (#220) * migration: Simplify tests * migration: Add working allowlist for Alfajores Adapt migration code to changes in StateDB StateDB.CreateAccount used to copy existing balance, now it does not any more. migration: Set fields correctly for migration block (#212) migration: Enable Granite (#226) Write genesis file in state migration (#219) * squash of https://github.com/celo-org/optimism/pull/167 * add writeGenesis * open old freezer in readonly mode, fix locking error * remove devAlloc * Revert "open old freezer in readonly mode, fix locking error" This reverts commit e3fddeac9c4569a19dcd15d801da844b0b2ab125. * fix locking error * fix lint error, check errors, add comment * remove comment * filter extra genesis fields * fix issue with genesis extra data * update testdata --------- Co-authored-by: Javier Cortejoso migration: Overwrite create2deployer code (#233) migration: Allow 'createx' preinstall (#238) The code already exists on Alfajores and matches the one that would be deployed, therefore we just allow this address. add migration-block-number flag (#245) * add migration-block-number flag * address feedback * move migration-block-number flag out of state migration options Fixes for re-running migration script on same destination db (#246) * add reset flag * add --checksum to rsync options --- .github/workflows/docker-build-scan.yaml | 172 +++-- .gitignore | 3 + op-chain-ops/Dockerfile | 29 + op-chain-ops/Makefile | 3 + op-chain-ops/cmd/celo-migrate/README.md | 136 ++++ op-chain-ops/cmd/celo-migrate/ancients.go | 232 +++++++ op-chain-ops/cmd/celo-migrate/db.go | 133 ++++ op-chain-ops/cmd/celo-migrate/genesis.go | 649 ++++++++++++++++++ op-chain-ops/cmd/celo-migrate/main.go | 454 ++++++++++++ op-chain-ops/cmd/celo-migrate/non-ancients.go | 118 ++++ op-chain-ops/cmd/celo-migrate/state.go | 451 ++++++++++++ op-chain-ops/cmd/celo-migrate/state_test.go | 147 ++++ .../testdata/deploy-config-dango.json | 91 +++ .../deploy-config-holesky-alfajores.json | 89 +++ .../testdata/deployment-l1-dango.json | 37 + .../testdata/deployment-l1-holesky.json | 34 + .../testdata/rollup-config-dango.json | 42 ++ .../celo-migrate/testdata/rollup-config.json | 36 + op-chain-ops/cmd/celo-migrate/transform.go | 109 +++ op-chain-ops/justfile | 16 + packages/contracts-bedrock/foundry.toml | 1 + 21 files changed, 2895 insertions(+), 87 deletions(-) create mode 100644 op-chain-ops/Dockerfile create mode 100644 op-chain-ops/cmd/celo-migrate/README.md create mode 100644 op-chain-ops/cmd/celo-migrate/ancients.go create mode 100644 op-chain-ops/cmd/celo-migrate/db.go create mode 100644 op-chain-ops/cmd/celo-migrate/genesis.go create mode 100644 op-chain-ops/cmd/celo-migrate/main.go create mode 100644 op-chain-ops/cmd/celo-migrate/non-ancients.go create mode 100644 op-chain-ops/cmd/celo-migrate/state.go create mode 100644 op-chain-ops/cmd/celo-migrate/state_test.go create mode 100644 op-chain-ops/cmd/celo-migrate/testdata/deploy-config-dango.json create mode 100644 op-chain-ops/cmd/celo-migrate/testdata/deploy-config-holesky-alfajores.json create mode 100644 op-chain-ops/cmd/celo-migrate/testdata/deployment-l1-dango.json create mode 100644 op-chain-ops/cmd/celo-migrate/testdata/deployment-l1-holesky.json create mode 100644 op-chain-ops/cmd/celo-migrate/testdata/rollup-config-dango.json create mode 100644 op-chain-ops/cmd/celo-migrate/testdata/rollup-config.json create mode 100644 op-chain-ops/cmd/celo-migrate/transform.go create mode 100644 op-chain-ops/justfile diff --git a/.github/workflows/docker-build-scan.yaml b/.github/workflows/docker-build-scan.yaml index 30ad5196c7fe..709a65fafcb8 100644 --- a/.github/workflows/docker-build-scan.yaml +++ b/.github/workflows/docker-build-scan.yaml @@ -1,92 +1,90 @@ name: Docker Build Scan on: + pull_request: + branches: + - 'master' + - 'celo*' workflow_dispatch: jobs: - Build-Scan-Container-op-ufm: - uses: celo-org/reusable-workflows/.github/workflows/container-cicd-local.yaml@v1.11.2 - with: - dockerfile: op-ufm/Dockerfile - - Build-Scan-Container-ops-bedrock-l1: - uses: celo-org/reusable-workflows/.github/workflows/container-cicd-local.yaml@v1.11.2 - with: - dockerfile: ops-bedrock/Dockerfile.l1 - context: ops-bedrock - - Build-Scan-Container-ops-bedrock-l2: - uses: celo-org/reusable-workflows/.github/workflows/container-cicd-local.yaml@v1.11.2 - with: - dockerfile: ops-bedrock/Dockerfile.l2 - context: ops-bedrock - - Build-Scan-Container-indexer: - uses: celo-org/reusable-workflows/.github/workflows/container-cicd-local.yaml@v1.11.2 - with: - dockerfile: indexer/Dockerfile - - Build-Scan-Container-op-heartbeat: - uses: celo-org/reusable-workflows/.github/workflows/container-cicd-local.yaml@v1.11.2 - with: - dockerfile: op-heartbeat/Dockerfile - - Build-Scan-Container-op-exporter: - uses: celo-org/reusable-workflows/.github/workflows/container-cicd-local.yaml@v1.11.2 - with: - dockerfile: op-exporter/Dockerfile - - Build-Scan-Container-op-program: - uses: celo-org/reusable-workflows/.github/workflows/container-cicd-local.yaml@v1.11.2 - with: - dockerfile: op-program/Dockerfile - - Build-Scan-Container-ops-bedrock: - uses: celo-org/reusable-workflows/.github/workflows/container-cicd-local.yaml@v1.11.2 - with: - dockerfile: ops-bedrock/Dockerfile.stateviz - - Build-Scan-Container-ci-builder: - uses: celo-org/reusable-workflows/.github/workflows/container-cicd-local.yaml@v1.11.2 - with: - dockerfile: ops/docker/ci-builder/Dockerfile - - Build-Scan-Container-proxyd: - uses: celo-org/reusable-workflows/.github/workflows/container-cicd-local.yaml@v1.11.2 - with: - dockerfile: proxyd/Dockerfile - - Build-Scan-Container-op-node: - uses: celo-org/reusable-workflows/.github/workflows/container-cicd-local.yaml@v1.11.2 - with: - dockerfile: op-node/Dockerfile - - Build-Scan-Container-op-batcher: - uses: celo-org/reusable-workflows/.github/workflows/container-cicd-local.yaml@v1.11.2 - with: - dockerfile: op-batcher/Dockerfile - - Build-Scan-Container-indexer-ui: - uses: celo-org/reusable-workflows/.github/workflows/container-cicd-local.yaml@v1.11.2 - with: - dockerfile: indexer/ui/Dockerfile - - Build-Scan-Container-op-proposer: - uses: celo-org/reusable-workflows/.github/workflows/container-cicd-local.yaml@v1.11.2 - with: - dockerfile: op-proposer/Dockerfile - - Build-Scan-Container-op-challenger: - uses: celo-org/reusable-workflows/.github/workflows/container-cicd-local.yaml@v1.11.2 - with: - dockerfile: op-challenger/Dockerfile - - Build-Scan-Container-endpoint-monitor: - uses: celo-org/reusable-workflows/.github/workflows/container-cicd-local.yaml@v1.11.2 - with: - dockerfile: endpoint-monitor/Dockerfile - - Build-Scan-Container-opwheel: - uses: celo-org/reusable-workflows/.github/workflows/container-cicd-local.yaml@v1.11.2 - with: - dockerfile: op-wheel/Dockerfile - + detect-files-changed: + runs-on: ubuntu-latest + outputs: + files-changed: ${{ steps.detect-files-changed.outputs.all_changed_files }} + steps: + - uses: actions/checkout@v4 + - name: Detect files changed + id: detect-files-changed + uses: tj-actions/changed-files@v44 + with: + separator: ',' + + build-cel2-migration-tool: + runs-on: ubuntu-latest + needs: detect-files-changed + if: | + contains(needs.detect-files-changed.outputs.files-changed, 'go.sum') || + contains(needs.detect-files-changed.outputs.files-changed, 'op-chain-ops/cmd/celo-migrate') || + contains(needs.detect-files-changed.outputs.files-changed, 'op-chain-ops/Dockerfile') + permissions: + contents: read + id-token: write + security-events: write + steps: + - uses: actions/checkout@v4 + - name: Login at GCP Artifact Registry + uses: celo-org/reusable-workflows/.github/actions/auth-gcp-artifact-registry@v2.0 + with: + workload-id-provider: 'projects/1094498259535/locations/global/workloadIdentityPools/gh-optimism/providers/github-by-repos' + service-account: 'celo-optimism-gh@devopsre.iam.gserviceaccount.com' + docker-gcp-registries: us-west1-docker.pkg.dev + - name: Build and push container + uses: celo-org/reusable-workflows/.github/actions/build-container@v2.0 + with: + platforms: linux/amd64 + registry: us-west1-docker.pkg.dev/devopsre/dev-images/cel2-migration-tool + tags: ${{ github.sha }} + context: ./ + dockerfile: ./op-chain-ops/Dockerfile + push: true + trivy: false + + # Build op-node op-batcher op-proposer using docker-bake + build-op-stack: + runs-on: ubuntu-latest + needs: detect-files-changed + if: | + contains(needs.detect-files-changed.outputs.files-changed, 'go.sum') || + contains(needs.detect-files-changed.outputs.files-changed, 'ops/docker') || + contains(needs.detect-files-changed.outputs.files-changed, 'op-node/') || + contains(needs.detect-files-changed.outputs.files-changed, 'op-batcher/') || + contains(needs.detect-files-changed.outputs.files-changed, 'op-proposer/') || + contains(needs.detect-files-changed.outputs.files-changed, 'op-service/') + permissions: + contents: read + id-token: write + security-events: write + env: + GIT_COMMIT: ${{ github.sha }} + GIT_DATE: ${{ github.event.head_commit.timestamp }} + IMAGE_TAGS: ${{ github.sha }},latest + REGISTRY: us-west1-docker.pkg.dev + REPOSITORY: blockchaintestsglobaltestnet/dev-images + steps: + - uses: actions/checkout@v4 + - name: Login at GCP Artifact Registry + uses: celo-org/reusable-workflows/.github/actions/auth-gcp-artifact-registry@v2.0 + with: + workload-id-provider: 'projects/1094498259535/locations/global/workloadIdentityPools/gh-optimism/providers/github-by-repos' + service-account: 'celo-optimism-gh@devopsre.iam.gserviceaccount.com' + docker-gcp-registries: us-west1-docker.pkg.dev + # We need a custom steps as it's using docker bake + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@v3 + - name: Build and push + uses: docker/bake-action@v5 + with: + push: true + source: . + files: docker-bake.hcl + targets: op-node,op-batcher,op-proposer diff --git a/.gitignore b/.gitignore index 9751cc608985..6cae778cfe6f 100644 --- a/.gitignore +++ b/.gitignore @@ -46,3 +46,6 @@ __pycache__ # Ignore echidna artifacts crytic-export + +# vscode +.vscode/ diff --git a/op-chain-ops/Dockerfile b/op-chain-ops/Dockerfile new file mode 100644 index 000000000000..5aa278ed6b27 --- /dev/null +++ b/op-chain-ops/Dockerfile @@ -0,0 +1,29 @@ +FROM golang:1.21.1-alpine3.18 AS builder + +RUN apk --no-cache add make + +COPY ./go.mod /app/go.mod +COPY ./go.sum /app/go.sum + +WORKDIR /app + +RUN go mod download + +COPY ./op-service /app/op-service +COPY ./op-node /app/op-node +COPY ./op-alt-da /app/op-alt-da +COPY ./op-chain-ops /app/op-chain-ops +WORKDIR /app/op-chain-ops +RUN make celo-migrate + +FROM alpine:3.18 +RUN apk --no-cache add ca-certificates bash rsync + +# RUN addgroup -S app && adduser -S app -G app +# USER app +WORKDIR /app + +COPY --from=builder /app/op-chain-ops/bin/celo-migrate /app +ENV PATH="/app:${PATH}" + +ENTRYPOINT ["/app/celo-migrate"] diff --git a/op-chain-ops/Makefile b/op-chain-ops/Makefile index fd3cc9ad67b3..6a173c051031 100644 --- a/op-chain-ops/Makefile +++ b/op-chain-ops/Makefile @@ -32,6 +32,9 @@ ecotone-scalar: receipt-reference-builder: go build -o ./bin/receipt-reference-builder ./cmd/receipt-reference-builder/*.go +celo-migrate: + go build -o ./bin/celo-migrate ./cmd/celo-migrate/*.go + test: go test ./... diff --git a/op-chain-ops/cmd/celo-migrate/README.md b/op-chain-ops/cmd/celo-migrate/README.md new file mode 100644 index 000000000000..807a4f0b1606 --- /dev/null +++ b/op-chain-ops/cmd/celo-migrate/README.md @@ -0,0 +1,136 @@ +# Celo L2 Migration Script + +## Overview + +This script migrates a Celo L1 database (old datadir) into a new database compatible with Celo L2 (new datadir). It consists of 3 main processes that respectively migrate ancient blocks, non-ancient blocks and state. Migrated data is copied into a new datadir, leaving the old datadir unchanged. + +To minimize migration downtime, the script is designed to run in two stages: +1. The `pre migration` stage can be run ahead of the `full migration` and will process as much of the migration as possible up to that point. +2. The `full migration` can then be run to finish migrating new blocks that were created after the `pre migration` and apply necessary state changes on top of the migration block. + +### Pre migration + +The `pre migration` consists of two parts that are run in parallel: +- Copy and transform the ancient / frozen blocks (i.e. all blocks before the last 90000). +- Copy over the rest of the database using `rsync`. + +The ancients db is migrated sequentially because it is append-only, while the rest of the database is copied and then transformed in-place. We use `rsync` because it has flags for ignoring the ancients directory, skipping any already copied files and deleting any extra files in the new db, ensuring that we can run the script multiple times and only copy over actual updates. + +The `pre migration` step is still run during a `full migration` but it will be much quicker as only newly frozen blocks and recent file changes need to be migrated. + +### Full migration + +During the `full migration`, we re-run the `pre migration` step to capture any updates since the last `pre migration` and then apply in-place changes to non-ancient blocks and state. While this is happening, the script also checks for any stray ancient blocks that have remained in leveldb despite being frozen and removes them from the new db. Non-ancient blocks are then transformed to ensure compatibility with the L2 codebase. + +Finally after all blocks have been migrated, the script performs a series of modifications to the state db: +1. First, it deploys the L2 smart contracts by iterating through the genesis allocs passed to the script and setting the nonce, balance, code and storage for each address accordingly, overwritting existing data if necessary. +2. Finally, these changes are committed to the state db to produce a new state root and create the first Celo L2 block. + +### Notes + +> [!TIP] +> See `--help` for how to run each portion of the script individually, along with other configuration options. + +The longest running section of the script is the ancients migration, followed by the `rsync` command. By running these together in a `pre migration` we greatly reduce how long they will take during the `full migration`. Changes made to non-ancient blocks and state during a `full migration` are erased by the next `rsync` command. + +The script outputs a `rollup-config.json` file that is passed to the sequencer in order to start the L2 network. + +### Running the script + +> [!NOTE] +> You will need `rsync` to run this script if it's not already installed. + +From the `op-chain-ops` directory, first build the script by running: + +```bash +make celo-migrate +``` + +You can then run the script as follows: + +```bash +go run ./cmd/celo-migrate --help +``` + +#### Running with local test setup (Alfajores / Holesky) + +To test the script locally, we can migrate an Alfajores database and use Holesky as our L1. The input files needed for this can be found in `./testdata`. The necessary smart contracts have already been deployed on Holesky. + +##### Pull down the latest Alfajores database snapshot + +```bash +gcloud alpha storage cp gs://celo-chain-backup/alfajores/chaindata-latest.tar.zst alfajores.tar.zst +``` + +Unzip and rename + +```bash +tar --use-compress-program=unzstd -xvf alfajores.tar.zst +mv chaindata ./data/alfajores_old +``` + +##### Generate test allocs file + +The state migration takes in an allocs file that specifies the l2 state changes to be made during the migration. This file can be generated from the deploy config and l1 contract addresses by running the following from the `contracts-bedrock` directory. + +```bash +CONTRACT_ADDRESSES_PATH=../../op-chain-ops/cmd/celo-migrate/testdata/deployment-l1-dango.json \ +DEPLOY_CONFIG_PATH=../../op-chain-ops/cmd/celo-migrate/testdata/deploy-config-dango.json \ +STATE_DUMP_PATH=../../op-chain-ops/cmd/celo-migrate/testdata/l2-allocs-dango.json \ +forge script ./scripts/L2Genesis.s.sol:L2Genesis \ +--sig 'runWithStateDump()' +``` + +This should output the allocs file to `./testdata/l2-allocs-dango.json`. If you encounter difficulties with this and want to just continue testing the script, you can alternatively find the allocs file [here](https://storage.googleapis.com/cel2-rollup-files/alfajores-mvp/l2-allocs.json). + +##### Run script with test configuration + +```bash +go run ./cmd/celo-migrate pre \ +--old-db ./data/alfajores_old \ +--new-db ./data/alfajores_new +``` + +Running the pre-migration script should take ~5 minutes. This script copies and transforms ancient blocks and, in parallel, copies over all other chaindata without transforming it. This can be re-run mutliple times leading up to the full migration, and should only migrate updates to the old db between re-runs. + +```bash +go run ./cmd/celo-migrate full \ +--deploy-config ./cmd/celo-migrate/testdata/deploy-config-dango.json \ +--l1-deployments ./cmd/celo-migrate/testdata/deployment-l1-dango.json \ +--l1-rpc https://ethereum-holesky-rpc.publicnode.com \ +--l2-allocs ./cmd/celo-migrate/testdata/l2-allocs-dango.json \ +--outfile.rollup-config ./cmd/celo-migrate/testdata/rollup-config-dango.json \ +--old-db ./data/alfajores_old \ +--new-db ./data/alfajores_new +``` + +Running the full migration script re-runs the pre-migration script once to migrate any new changes to the old db that have occurred since the last pre-migration. It then performs in-place transformations on the non-ancient blocks and performs the state migration as well. + +#### Running for Cel2 migration + +##### Generate allocs file + +You can generate the allocs file needed to run the migration with the following script in `contracts-bedrock` + +```bash +CONTRACT_ADDRESSES_PATH= \ +DEPLOY_CONFIG_PATH= \ +STATE_DUMP_PATH= \ +forge script scripts/L2Genesis.s.sol:L2Genesis \ +--sig 'runWithStateDump()' +``` + +##### Dry-run / pre-migration + +To minimize downtime caused by the migration, node operators can prepare their Cel2 databases by running the pre-migration command a day ahead of the actual migration. This will pre-populate the new database with most of the ancient blocks needed for the final migration and copy over other chaindata without transforming it. + +If node operators would like to practice a `full migration` they can do so and reset their databases to the correct state by running another `pre migration` afterward. + +> [!IMPORTANT] +> The pre-migration should be run using a chaindata snapshot, rather than a db that is being used by a node. To avoid network downtime, we recommend that node operators do not stop any nodes in order to perform the pre-migration. + +Node operators should inspect their migration logs after the dry-run to ensure the migration completed succesfully and direct any questions to the Celo developer community on Discord before the actual migration. + +##### Final migration + +On the day of the actual Cel2 migration, the `full migration` script can be run using the datadir of a Celo L1 node that has halted on the migration block. Far in advance of the migration, a version of `celo-blockchain` will be distributed where a flag can specify a block to halt on. When the Celo community aligns on a migration block, node operators will start / restart their nodes with this flag specifying the migration block. Their nodes will halt when this block is reached, at which point they will be able to run `full migration` and begin syncing with the Celo L2 network. diff --git a/op-chain-ops/cmd/celo-migrate/ancients.go b/op-chain-ops/cmd/celo-migrate/ancients.go new file mode 100644 index 000000000000..183c81ed50de --- /dev/null +++ b/op-chain-ops/cmd/celo-migrate/ancients.go @@ -0,0 +1,232 @@ +package main + +import ( + "context" + "fmt" + "path/filepath" + + "github.com/ethereum/go-ethereum/common" + "github.com/ethereum/go-ethereum/core/rawdb" + "github.com/ethereum/go-ethereum/ethdb" + "github.com/ethereum/go-ethereum/log" + "golang.org/x/sync/errgroup" +) + +// RLPBlockRange is a range of blocks in RLP format +type RLPBlockRange struct { + start uint64 + hashes [][]byte + headers [][]byte + bodies [][]byte + receipts [][]byte + tds [][]byte +} + +// NewChainFreezer is a small utility method around NewFreezer that sets the +// default parameters for the chain storage. +func NewChainFreezer(datadir string, namespace string, readonly bool) (*rawdb.Freezer, error) { + const freezerTableSize = 2 * 1000 * 1000 * 1000 + // chainFreezerNoSnappy configures whether compression is disabled for the ancient-tables. + // Hashes and difficulties don't compress well. + var chainFreezerNoSnappy = map[string]bool{ + rawdb.ChainFreezerHeaderTable: false, + rawdb.ChainFreezerHashTable: true, + rawdb.ChainFreezerBodiesTable: false, + rawdb.ChainFreezerReceiptTable: false, + rawdb.ChainFreezerDifficultyTable: true, + } + return rawdb.NewFreezer(datadir, namespace, readonly, freezerTableSize, chainFreezerNoSnappy) +} + +func migrateAncientsDb(ctx context.Context, oldDBPath, newDBPath string, batchSize, bufferSize uint64) (uint64, uint64, error) { + defer timer("ancients")() + + oldFreezer, err := NewChainFreezer(filepath.Join(oldDBPath, "ancient"), "", false) // Can't be readonly because we need the .meta files to be created + if err != nil { + return 0, 0, fmt.Errorf("failed to open old freezer: %w", err) + } + defer oldFreezer.Close() + + newFreezer, err := NewChainFreezer(filepath.Join(newDBPath, "ancient"), "", false) + if err != nil { + return 0, 0, fmt.Errorf("failed to open new freezer: %w", err) + } + defer newFreezer.Close() + + numAncientsOld, err := oldFreezer.Ancients() + if err != nil { + return 0, 0, fmt.Errorf("failed to get number of ancients in old freezer: %w", err) + } + + numAncientsNewBefore, err := newFreezer.Ancients() + if err != nil { + return 0, 0, fmt.Errorf("failed to get number of ancients in new freezer: %w", err) + } + + if numAncientsNewBefore >= numAncientsOld { + log.Info("Ancient Block Migration Skipped", "process", "ancients", "ancientsInOldDB", numAncientsOld, "ancientsInNewDB", numAncientsNewBefore) + return numAncientsNewBefore, numAncientsNewBefore, nil + } + + log.Info("Ancient Block Migration Started", "process", "ancients", "startBlock", numAncientsNewBefore, "endBlock", numAncientsOld-1, "count", numAncientsOld-numAncientsNewBefore, "step", batchSize) + + g, ctx := errgroup.WithContext(ctx) + readChan := make(chan RLPBlockRange, bufferSize) + transformChan := make(chan RLPBlockRange, bufferSize) + + g.Go(func() error { + return readAncientBlocks(ctx, oldFreezer, numAncientsNewBefore, numAncientsOld, batchSize, readChan) + }) + g.Go(func() error { return transformBlocks(ctx, readChan, transformChan) }) + g.Go(func() error { return writeAncientBlocks(ctx, newFreezer, transformChan, numAncientsOld) }) + + if err = g.Wait(); err != nil { + return 0, 0, fmt.Errorf("failed to migrate ancients: %w", err) + } + + numAncientsNewAfter, err := newFreezer.Ancients() + if err != nil { + return 0, 0, fmt.Errorf("failed to get number of ancients in new freezer: %w", err) + } + + log.Info("Ancient Block Migration Ended", "process", "ancients", "ancientsInOldDB", numAncientsOld, "ancientsInNewDB", numAncientsNewAfter, "migrated", numAncientsNewAfter-numAncientsNewBefore) + return numAncientsNewBefore, numAncientsNewAfter, nil +} + +func readAncientBlocks(ctx context.Context, freezer *rawdb.Freezer, startBlock, endBlock, batchSize uint64, out chan<- RLPBlockRange) error { + defer close(out) + + for i := startBlock; i < endBlock; i += batchSize { + select { + case <-ctx.Done(): + return ctx.Err() + default: + count := min(batchSize, endBlock-i+1) + start := i + + blockRange := RLPBlockRange{ + start: start, + hashes: make([][]byte, count), + headers: make([][]byte, count), + bodies: make([][]byte, count), + receipts: make([][]byte, count), + tds: make([][]byte, count), + } + var err error + + blockRange.hashes, err = freezer.AncientRange(rawdb.ChainFreezerHashTable, start, count, 0) + if err != nil { + return fmt.Errorf("failed to read hashes from old freezer: %w", err) + } + blockRange.headers, err = freezer.AncientRange(rawdb.ChainFreezerHeaderTable, start, count, 0) + if err != nil { + return fmt.Errorf("failed to read headers from old freezer: %w", err) + } + blockRange.bodies, err = freezer.AncientRange(rawdb.ChainFreezerBodiesTable, start, count, 0) + if err != nil { + return fmt.Errorf("failed to read bodies from old freezer: %w", err) + } + blockRange.receipts, err = freezer.AncientRange(rawdb.ChainFreezerReceiptTable, start, count, 0) + if err != nil { + return fmt.Errorf("failed to read receipts from old freezer: %w", err) + } + blockRange.tds, err = freezer.AncientRange(rawdb.ChainFreezerDifficultyTable, start, count, 0) + if err != nil { + return fmt.Errorf("failed to read tds from old freezer: %w", err) + } + + out <- blockRange + } + } + return nil +} + +func transformBlocks(ctx context.Context, in <-chan RLPBlockRange, out chan<- RLPBlockRange) error { + // Transform blocks from the in channel and send them to the out channel + defer close(out) + for blockRange := range in { + select { + case <-ctx.Done(): + return ctx.Err() + default: + for i := range blockRange.hashes { + blockNumber := blockRange.start + uint64(i) + + newHeader, err := transformHeader(blockRange.headers[i]) + if err != nil { + return fmt.Errorf("can't transform header: %w", err) + } + newBody, err := transformBlockBody(blockRange.bodies[i]) + if err != nil { + return fmt.Errorf("can't transform body: %w", err) + } + + if yes, newHash := hasSameHash(newHeader, blockRange.hashes[i]); !yes { + log.Error("Hash mismatch", "block", blockNumber, "oldHash", common.BytesToHash(blockRange.hashes[i]), "newHash", newHash) + return fmt.Errorf("hash mismatch at block %d", blockNumber) + } + + blockRange.headers[i] = newHeader + blockRange.bodies[i] = newBody + } + out <- blockRange + } + } + return nil +} + +func writeAncientBlocks(ctx context.Context, freezer *rawdb.Freezer, in <-chan RLPBlockRange, totalAncientBlocks uint64) error { + // Write blocks from the in channel to the newDb + for blockRange := range in { + select { + case <-ctx.Done(): + return ctx.Err() + default: + _, err := freezer.ModifyAncients(func(aWriter ethdb.AncientWriteOp) error { + for i := range blockRange.hashes { + blockNumber := blockRange.start + uint64(i) + if err := aWriter.AppendRaw(rawdb.ChainFreezerHashTable, blockNumber, blockRange.hashes[i]); err != nil { + return fmt.Errorf("can't write hash to Freezer: %w", err) + } + if err := aWriter.AppendRaw(rawdb.ChainFreezerHeaderTable, blockNumber, blockRange.headers[i]); err != nil { + return fmt.Errorf("can't write header to Freezer: %w", err) + } + if err := aWriter.AppendRaw(rawdb.ChainFreezerBodiesTable, blockNumber, blockRange.bodies[i]); err != nil { + return fmt.Errorf("can't write body to Freezer: %w", err) + } + if err := aWriter.AppendRaw(rawdb.ChainFreezerReceiptTable, blockNumber, blockRange.receipts[i]); err != nil { + return fmt.Errorf("can't write receipts to Freezer: %w", err) + } + if err := aWriter.AppendRaw(rawdb.ChainFreezerDifficultyTable, blockNumber, blockRange.tds[i]); err != nil { + return fmt.Errorf("can't write td to Freezer: %w", err) + } + } + return nil + }) + if err != nil { + return fmt.Errorf("failed to write block range: %w", err) + } + blockRangeEnd := blockRange.start + uint64(len(blockRange.hashes)) - 1 + log.Info("Wrote ancient blocks", "start", blockRange.start, "end", blockRangeEnd, "count", len(blockRange.hashes), "remaining", totalAncientBlocks-blockRangeEnd) + } + } + return nil +} + +// getStrayAncientBlocks returns a list of ancient block numbers / hashes that somehow were not removed from leveldb +func getStrayAncientBlocks(dbPath string) ([]*rawdb.NumberHash, error) { + defer timer("getStrayAncientBlocks")() + + db, err := openDB(dbPath, true) + if err != nil { + return nil, fmt.Errorf("failed to open database: %w", err) + } + defer db.Close() + + numAncients, err := db.Ancients() + if err != nil { + return nil, fmt.Errorf("failed to get number of ancients in database: %w", err) + } + + return rawdb.ReadAllHashesInRange(db, 1, numAncients-1), nil +} diff --git a/op-chain-ops/cmd/celo-migrate/db.go b/op-chain-ops/cmd/celo-migrate/db.go new file mode 100644 index 000000000000..180b9e541dd0 --- /dev/null +++ b/op-chain-ops/cmd/celo-migrate/db.go @@ -0,0 +1,133 @@ +package main + +import ( + "encoding/binary" + "errors" + "fmt" + "os" + "path/filepath" + + "github.com/ethereum/go-ethereum/common" + "github.com/ethereum/go-ethereum/core/rawdb" + "github.com/ethereum/go-ethereum/core/types" + "github.com/ethereum/go-ethereum/ethdb" + "github.com/ethereum/go-ethereum/log" +) + +// Constants for the database +const ( + DBCache = 1024 // size of the cache in MB + DBHandles = 60 // number of handles +) + +var ( + headerPrefix = []byte("h") // headerPrefix + num (uint64 big endian) + hash -> header +) + +// encodeBlockNumber encodes a block number as big endian uint64 +func encodeBlockNumber(number uint64) []byte { + enc := make([]byte, 8) + binary.BigEndian.PutUint64(enc, number) + return enc +} + +// headerKey = headerPrefix + num (uint64 big endian) + hash +func headerKey(number uint64, hash common.Hash) []byte { + return append(append(headerPrefix, encodeBlockNumber(number)...), hash.Bytes()...) +} + +// Opens a database with access to AncientsDb +func openDB(chaindataPath string, readOnly bool) (ethdb.Database, error) { + if _, err := os.Stat(chaindataPath); errors.Is(err, os.ErrNotExist) { + return nil, err + } + + db, err := rawdb.Open(rawdb.OpenOptions{ + Type: "leveldb", + Directory: chaindataPath, + AncientsDirectory: filepath.Join(chaindataPath, "ancient"), + Namespace: "", + Cache: DBCache, + Handles: DBHandles, + ReadOnly: readOnly, + }) + if err != nil { + return nil, err + } + + return db, nil +} + +// Opens a database without access to AncientsDb +func openDBWithoutFreezer(chaindataPath string, readOnly bool) (ethdb.Database, error) { + if _, err := os.Stat(chaindataPath); errors.Is(err, os.ErrNotExist) { + return nil, err + } + + newDB, err := rawdb.NewLevelDBDatabase(chaindataPath, DBCache, DBHandles, "", readOnly) + if err != nil { + return nil, err + } + + return newDB, nil +} + +func createNewDbPathIfNotExists(newDBPath string) error { + if err := os.MkdirAll(newDBPath, 0755); err != nil { + return fmt.Errorf("failed to create new database directory: %w", err) + } + return nil +} + +func removeBlocks(ldb ethdb.Database, numberHashes []*rawdb.NumberHash) error { + defer timer("removeBlocks")() + + if len(numberHashes) == 0 { + return nil + } + + batch := ldb.NewBatch() + + for _, numberHash := range numberHashes { + log.Debug("Removing block", "block", numberHash.Number) + rawdb.DeleteBlockWithoutNumber(batch, numberHash.Hash, numberHash.Number) + rawdb.DeleteCanonicalHash(batch, numberHash.Number) + } + if err := batch.Write(); err != nil { + log.Error("Failed to write batch", "error", err) + } + + return nil +} + +func getHeadHeader(dbpath string) (*types.Header, error) { + db, err := openDBWithoutFreezer(dbpath, true) + if err != nil { + return nil, fmt.Errorf("failed to open database at %q err: %w", dbpath, err) + } + defer db.Close() + + headHeader := rawdb.ReadHeadHeader(db) + if headHeader == nil { + return nil, fmt.Errorf("head header not in database at: %s", dbpath) + } + return headHeader, nil +} + +func cleanupNonAncientDb(dir string) error { + log.Info("Cleaning up non-ancient data in new db") + + files, err := os.ReadDir(dir) + if err != nil { + return fmt.Errorf("failed to read directory: %w", err) + } + for _, file := range files { + if file.Name() != "ancient" { + err := os.RemoveAll(filepath.Join(dir, file.Name())) + if err != nil { + return fmt.Errorf("failed to remove file: %w", err) + } + } + } + return nil +} diff --git a/op-chain-ops/cmd/celo-migrate/genesis.go b/op-chain-ops/cmd/celo-migrate/genesis.go new file mode 100644 index 000000000000..b646b4c73ea5 --- /dev/null +++ b/op-chain-ops/cmd/celo-migrate/genesis.go @@ -0,0 +1,649 @@ +package main + +import ( + "fmt" + + "github.com/ethereum/go-ethereum/core" + "github.com/ethereum/go-ethereum/core/types" + "github.com/ethereum/go-ethereum/params" +) + +// Constants containing the genesis allocation of built-in genesis blocks. +// Copied from https://github.com/celo-org/celo-blockchain/blob/d4ac28c04d5b94855e1e72c9b4436fa3c73e16ad/core/genesis_alloc.go#L19 + +const baklavaAllocJSON = `{ + "fCf982bb4015852e706100B14E21f947a5Bb718E": { + "balance": "200000000000000000000000000" + }, + "0xd71fea6b92d3f21f659152589223385a7329bb11": { + "balance": "1000000000000000000000" + }, + "0x1e477fc9b6a49a561343cd16b2c541930f5da7d2": { + "balance": "1000000000000000000000" + }, + "0x460b3f8d3c203363bb65b1a18d89d4ffb6b0c981": { + "balance": "1000000000000000000000" + }, + "0x3b522230c454ca9720665d66e6335a72327291e8": { + "balance": "1000000000000000000000" + }, + "0x0AFe167600a5542d10912f4A07DFc4EEe0769672": { + "balance": "1000000000000000000000" + }, + "0x412ebe7859e9aa71ff5ce4038596f6878c359c96": { + "balance": "1000000000000000000000" + }, + "0xbbfe73df8b346b3261b19ac91235888aba36d68c": { + "balance": "1000000000000000000000" + }, + "0x02b1d1bea682fcab4448c0820f5db409cce4f702": { + "balance": "1000000000000000000000" + }, + "0xe90f891710f625f18ecbf1e02efb4fd1ab236a10": { + "balance": "1000000000000000000000" + }, + "0x28c52c722df87ed11c5d7665e585e84aa93d7964": { + "balance": "1000000000000000000000" + }, + "0Cc59Ed03B3e763c02d54D695FFE353055f1502D": { + "balance": "103010030000000000000000000" + }, + "3F5084d3D4692cf19b0C98A9b22De614e49e1470": { + "balance": "10011000000000000000000" + }, + "EF0186B8eDA17BE7D1230eeB8389fA85e157E1fb": { + "balance": "10011000000000000000000" + }, + "edDdb60EF5E90Fb09707246DF193a55Df3564c9d": { + "balance": "10011000000000000000000" + }, + "d5e454462b3Fd98b85640977D7a5C783CA162228": { + "balance": "10011000000000000000000" + }, + "a4f1bad7996f346c3E90b90b60a1Ca8B67B51E4B": { + "balance": "10011000000000000000000" + }, + "5B991Cc1Da0b6D54F8befa9De701d8BC85C92324": { + "balance": "10011000000000000000000" + }, + "6dfdAa51D146eCff3B97614EF05629EA83F4997E": { + "balance": "10011000000000000000000" + }, + "D2b16050810600296c9580D947E9D919D0c332ed": { + "balance": "10011000000000000000000" + }, + "Fe144D67068737628efFb701207B3eB30eF93C69": { + "balance": "10011000000000000000000" + }, + "82E64996B355625efeAaD12120710706275b5b9A": { + "balance": "10011000000000000000000" + }, + "241752a3f65890F4AC3eAeC518fF94567954e7b5": { + "balance": "10011000000000000000000" + }, + "1bdDeaF571d5da96ce6a127fEb3CADaDB531f433": { + "balance": "10011000000000000000000" + }, + "F86345e9c9b39aB1cbE82d7aD35854f905B8B835": { + "balance": "10011000000000000000000" + }, + "5c3512b1697302c497B861CBfDA158f8a3c5122C": { + "balance": "10011000000000000000000" + }, + "a02A692d70Fd9A5269397C044aEBDf1085ba090f": { + "balance": "10011000000000000000000" + }, + "aC91f591F12a8B6531Be43E0ccF21cd5fA0E80b0": { + "balance": "10011000000000000000000" + }, + "718A8AC0943a6D3FFa3Ec670086bfB03817ed540": { + "balance": "10011000000000000000000" + }, + "b30980cE21679314E240DE5Cbf437C15ad459EB8": { + "balance": "10011000000000000000000" + }, + "99eCa23623E59C795EceB0edB666eca9eC272339": { + "balance": "10011000000000000000000" + }, + "c030e92d19229c3EfD708cf4B85876543ee1A3F7": { + "balance": "10011000000000000000000" + }, + "5c98A3414Cb6Ff5c24d145F952Cd19F5f1f56643": { + "balance": "10011000000000000000000" + }, + "1979b042Ae2272197f0b74170B3a6F44C3cC5c05": { + "balance": "10011000000000000000000" + }, + "Db871070334b961804A15f3606fBB4fAc7C7f932": { + "balance": "10011000000000000000000" + }, + "C656C97b765D61E0fbCb1197dC1F3a91CC80C2a4": { + "balance": "10011000000000000000000" + }, + "aD95a2f518c197dc9b12eE6381D88bba11F2E0E5": { + "balance": "10011000000000000000000" + }, + "4D4B5bF033E4A7359146C9ddb13B1C821FE1D0d3": { + "balance": "10011000000000000000000" + }, + "9C64dA169d71C57f85B3d7A17DB27C1ce94FBDE4": { + "balance": "10011000000000000000000" + }, + "B5f32e89ccaD3D396f50da32E0a599E43CE87dd7": { + "balance": "10011000000000000000000" + }, + "Ba40Db8ab5325494C9E7e07A4c4720990A39305c": { + "balance": "10011000000000000000000" + }, + "8B7852DA535df3D06D6ADc1906778afd9481588a": { + "balance": "10011000000000000000000" + }, + "a8F41EA062C22dAFFc61e47cF15fc898517b86B1": { + "balance": "10011000000000000000000" + }, + "66a3Fc7E8fd6932568cDB6610F5a67BeD9F5beF8": { + "balance": "10011000000000000000000" + }, + "10301d9389653497F62876f450332467E07eEe1F": { + "balance": "10011000000000000000000" + }, + "6c3ac5fcb13E8DCd908C405Ec6DAcF0EF575D8FC": { + "balance": "10011000000000000000000" + }, + "85226637919D3d47E1A37b3AF989E9aE1a1C4790": { + "balance": "10011000000000000000000" + }, + "43BCa16603c56cb681d1da3636B7a1A225598bfc": { + "balance": "10011000000000000000000" + }, + "E55d8Bc08025BDDF8Da02eEB54882d0586f90700": { + "balance": "10011000000000000000000" + }, + "40E1C73f6228a2c15e10aF2F3e890098b777ED15": { + "balance": "10011000000000000000000" + }, + "DbbF476089a186a406EA13a4c46813f4BccC3660": { + "balance": "10011000000000000000000" + }, + "7baCEA66a75dD974Ad549987768bF8d8908b4917": { + "balance": "10011000000000000000000" + }, + "fbF4C2362a9EB672BAC39A46AFd919B3c12Ce44c": { + "balance": "10011000000000000000000" + }, + "A8dB96136990be5B3d3bfe592e5A5a5223350A7A": { + "balance": "10011000000000000000000" + }, + "1Dd21ED691195EBA816d59B3De7Fab8b3470Ae4B": { + "balance": "10011000000000000000000" + }, + "058A778A6aeEfacc013afba92578A43e38cc012D": { + "balance": "10011000000000000000000" + }, + "13f52Ab66871880DC8F2179d705281a4cf6a15fB": { + "balance": "10011000000000000000000" + }, + "eD1Ed9a71E313d1BCe14aB998E0646F212230a33": { + "balance": "10011000000000000000000" + }, + "c563F264f98e34A409C6a085da7510De8B6FE90B": { + "balance": "10011000000000000000000" + }, + "c6D678fC6Cc1dA9D5eD1c0075cF7c679e7138e02": { + "balance": "10011000000000000000000" + }, + "5179fc80CaB9BB20d5405a50ec0Fb9a36c1B367a": { + "balance": "10011000000000000000000" + }, + "0d473f73AAf1C2bf7EBd2be7196C71dBa6C1724b": { + "balance": "100110000000000000000" + }, + "6958c5b7E3D94B041d0d76Cac2e09378d31201bd": { + "balance": "10011000000000000000000" + }, + "628d4A734d1a2647c67D254209e7B6471a11a5cb": { + "balance": "10011000000000000000000" + }, + "E1601e3172F0ef0100e363B639Bd44420B7E5490": { + "balance": "10011000000000000000000" + }, + "3337F2Cd103976F044b55D3E69aB06d1ebB142Db": { + "balance": "10011000000000000000000" + }, + "8D0D5c57dC232Be15Df4A1a048EF36162C853b94": { + "balance": "10011000000000000000000" + }, + "14800c28F3cF1Dd17AaC55263ef4e173b0e8e3Ef": { + "balance": "10011000000000000000000" + }, + "f3996A0f0f593BfD5E39780059C5430fab7359FD": { + "balance": "10011000000000000000000" + }, + "2217FeBe31Aea6C771AF163dCc453F9f060a4a00": { + "balance": "10011000000000000000000" + }, + "f426CC817400766cd6b44F13Cb63Ca648e323484": { + "balance": "10011000000000000000000" + }, + "B2C4913e257a34445Ec31685E625bb4060FB8e1f": { + "balance": "10011000000000000000000" + }, + "9438dbD05dfC19F049a469185c7599daa82646e8": { + "balance": "10011000000000000000000" + }, + "4BeD66Bf507f3CF524704267908Ea4ee3cDe3053": { + "balance": "10011000000000000000000" + }, + "9a850fe8105e9CCfBD9d1D06D535BB4948f3f6Cf": { + "balance": "10011000000000000000000" + }, + "1277eE554565542A8d0553E1e54006d006db75bd": { + "balance": "10011000000000000000000" + }, + "D7e829bE8E374D3fBbd2F68D9A916cB2f769BA89": { + "balance": "10011000000000000000000" + }, + "3691b847eD14E296afC90Ff3E37D21e518306170": { + "balance": "10011000000000000000000" + }, + "c4C703357B01672cF95bFa0450a5717812Bc7ffb": { + "balance": "10011000000000000000000" + }, + "0c9369077836353A8D92aeD29C72A7DfD300B354": { + "balance": "10011000000000000000000" + }, + "856DF2A3bdBb8086cE406C469dDE94d12C1E3176": { + "balance": "10011000000000000000000" + }, + "E40B3e5c59e2157037b699895329DBe4aA33C039": { + "balance": "10011000000000000000000" + }, + "edb47aF3aC2325735722450D1E7DA082bDDad58c": { + "balance": "10011000000000000000000" + }, + "315D669866E13fA302B76c85481F9181e06304Ce": { + "balance": "10011000000000000000000" + }, + "A5185E3328592428d5989422e0339247dD77e10D": { + "balance": "10011000000000000000000" + }, + "85Fd1d1Cd6655EbB89db7D6cA0a5C9c62F7a4CFf": { + "balance": "10011000000000000000000" + }, + "ACC9E4430EC1011673547395A191C6b152763EA4": { + "balance": "10011000000000000000000" + }, + "3824967C172D52128522dD257FE8f58C9099166B": { + "balance": "10011000000000000000000" + }, + "5542aDEA3092da5541250d70a3Db28Ad9BE7Cfc7": { + "balance": "10011000000000000000000" + }, + "c61Cd4477f0A98BfC97744481181730f7af7c14f": { + "balance": "10011000000000000000000" + }, + "5D7Ffd0fC6DAA67AbF7d48ae69f09dbe53d86983": { + "balance": "10011000000000000000000" + }, + "350914ABD4F095534823C1e8fA1cfD7EF79e7E4c": { + "balance": "10011000000000000000000" + }, + "ECa6f058B718E320c1D45f5D1fb07947367C3D4B": { + "balance": "10011000000000000000000" + }, + "9C577D0795Ed0cA88814d149c2DC61E8Fc48Ad81": { + "balance": "10011000000000000000000" + }, + "72fE8bC8E3Ff1e56543c9c1F9834D6dfC31BEDDC": { + "balance": "10011000000000000000000" + }, + "6Ff2CFa7899073CD029267fd821C9497811b5f7E": { + "balance": "10011000000000000000000" + }, + "4685D123aE928a7912646681ba32035ad6F010a6": { + "balance": "10011000000000000000000" + }, + "4799946c8B21fF5E58A225AeCB6F54ec17a94566": { + "balance": "10011000000000000000000" + }, + "1D7dA5a23a99Fc33e2e94d502E4Fdb564eA0B24C": { + "balance": "10011000000000000000000" + }, + "DFc9719cD9c7982e4A1FFB4B87cC3b861C40E367": { + "balance": "10011000000000000000000" + }, + "0c1F0457ce3e87f5eA8F2C3A007dfe963A6Ff9a7": { + "balance": "10011000000000000000000" + }, + "7dC23b30dFDc326B9a694c6f9723DC889fe16b7d": { + "balance": "10011000000000000000000" + }, + "3F0c4cFDD40D16B7C15878AcCdc91Be9ca4DeE79": { + "balance": "10011000000000000000000" + }, + "B984a83416F560437C7866e26CdDb94bDB821594": { + "balance": "10011000000000000000000" + }, + "138EA4C57F5b3984EFacd944b3b85dfDd5A78Dcc": { + "balance": "10011000000000000000000" + }, + "AD4f16F3435E849505C643714C9E5f40f73c4a5a": { + "balance": "10011000000000000000000" + }, + "6b38E861ec0b65fd288d96d5630711C576362152": { + "balance": "10011000000000000000000" + }, + "AE15D05100CE807d0aC93119f4ada8fa21441Fd2": { + "balance": "10011000000000000000000" + }, + "e0e25c5734bef8b2Add633eAa2518B207DAa0D66": { + "balance": "10011000000000000000000" + }, + "9039Ce107A9cD36Ed116958E50f8BDe090e2406f": { + "balance": "10011000000000000000000" + }, + "089bE2dD42096ebA1d94aad20228b75df2BeeBC7": { + "balance": "10011000000000000000000" + }, + "E3a79AEee437532313015892B52b65f52794F8a2": { + "balance": "10011000000000000000000" + }, + "Cc38EE244819649C9DaB02e268306cED09B20672": { + "balance": "10011000000000000000000" + }, + "eb0357140a1a0A6c1cB9c93Bf9354ef7365C97d9": { + "balance": "10011000000000000000000" + }, + "44370D6b2d010C9eBFa280b6C00010AC99a45660": { + "balance": "10011000000000000000000" + }, + "762438915209d038340C3Af9f8aAb8F93aDc8A9A": { + "balance": "10011000000000000000000" + }, + "9CBa7aD50fa366Ff6fC2CAe468929eC9AD23Ea2B": { + "balance": "10011000000000000000000" + }, + "4f4F159826b2B1eE903A811fCd86E450c9954396": { + "balance": "10011000000000000000000" + }, + "3C132B8465e2D172BB7bab6654D85E398ee7c8AD": { + "balance": "10011000000000000000000" + }, + "0582426C929B7e525c22201Bd4c143E45189C589": { + "balance": "10011000000000000000000" + }, + "fb542740B34dDC0ADE383F2907a1e1E175E0BF5a": { + "balance": "10011000000000000000000" + }, + "184Ca91AfE8F36bC5772b29cE2A76c90fCef34D0": { + "balance": "10011000000000000000000" + }, + "0C6f48B50B166ddcE52CEE051acCAfFB8ecB4976": { + "balance": "10011000000000000000000" + }, + "3aD2bE38fA3DFa7969E79B4053868FD1C368eAb2": { + "balance": "10011000000000000000000" + }, + "a6A690637b088E9A1A89c44c9dC5e14eD4825053": { + "balance": "10011000000000000000000" + }, + "C224B131Ea71e11E7DF38de3774AAAAe7E197BA4": { + "balance": "10011000000000000000000" + }, + "d3C18531f0879B9FB8Ed45830C4ce6b54dC57128": { + "balance": "10011000000000000000000" + }, + "02a272d17E1308beF21E783A93D1658f84F2D414": { + "balance": "10011000000000000000000" + }, + "57A1aC8167d94b899b32C38Ff9D2B2bD0e55C10d": { + "balance": "10011000000000000000000" + }, + "F8fc7D740929E5DD4eBA8fd5a6873Be6a4151087": { + "balance": "10011000000000000000000" + }, + "B2AfC45838b364240dE17D3143AA6096d3340A91": { + "balance": "10011000000000000000000" + }, + "eAf133d1e0Dd325721665B19f67C9b914EE2469F": { + "balance": "10011000000000000000000" + }, + "B7660F1B075e56780e7E026ff66995765f5f1f7F": { + "balance": "10011000000000000000000" + }, + "F25087E27B7a59003bb08d2cAc7A69E7c15a4be8": { + "balance": "10011000000000000000000" + }, + "E65054681206658A845140331459A057C4EB3CA7": { + "balance": "10011000000000000000000" + }, + "e7569A0F93E832a6633d133d23503B5175bEa5Db": { + "balance": "10011000000000000000000" + }, + "a9f6102BCf5351dFdC7fA0CA4Fa0A711e16605c3": { + "balance": "10011000000000000000000" + }, + "1AB9aA0E855DF953CF8d9cC166172799afD12a68": { + "balance": "10011000000000000000000" + }, + "6C04aA35c377E65658EC3600Cab5E8FFa95567D9": { + "balance": "10011000000000000000000" + }, + "6b82AD37e64c91c628305813B2DA82F18f8e2a2B": { + "balance": "10011000000000000000000" + }, + "AD5D1DeD72F0e70a0a5500B26b82B1A2e8A63471": { + "balance": "10011000000000000000000" + }, + "72B3589771Ec8e189a5d9Fe7a214e44085e89054": { + "balance": "10011000000000000000000" + }, + "74F57dA8be3E9AB4463DD70319A06Fb5E3168211": { + "balance": "10011000000000000000000" + }, + "b6f7F57b99DB21027875BEa3b8531d5925c346cE": { + "balance": "10011000000000000000000" + }, + "279d05241d33Dc422d5AEcAc0e089B7f50f879c3": { + "balance": "10011000000000000000000" + }, + "d57FEfe1B634ab451a6815Cd6769182EABA62779": { + "balance": "10011000000000000000000" + }, + "e86C8538Bdfb253E8D6cC29ee24A330905324849": { + "balance": "10011000000000000000000" + }, + "2C58D7f7f9CDF79CF3Cd5F4247761b93428A4E9e": { + "balance": "10011000000000000000000" + }, + "37326cEfAFB1676f7Af1CcDcCD37A846Ec64F19d": { + "balance": "10011000000000000000000" + }, + "f01DCf91d5f74BDB161F520e800c64F686Eb253F": { + "balance": "10011000000000000000000" + }, + "Ba85246bc2A4fdaC1cB2e3C68383Fe79A6466fd9": { + "balance": "10011000000000000000000" + }, + "4A76f81eA26381981a3B740975fb4F605989b585": { + "balance": "10011000000000000000000" + }, + "00ee7168618BaE4F4d2900D5063c62948c6F0566": { + "balance": "10011000000000000000000" + }, + "E1aD0B232B4262E4A279C91070417DAAF202623F": { + "balance": "10011000000000000000000" + }, + "f611173319b22080E0F02eE724781d85f4b39Ae6": { + "balance": "10011000000000000000000" + }, + "158659458dff3a9E5182cA0e8Ba08F53463FA5e7": { + "balance": "10011000000000000000000" + }, + "FEB11610ad367b0c994274A8153E50F4557e473F": { + "balance": "10011000000000000000000" + }, + "e1eB2279f45760Ab9D734782B1a0A8FD3d47D807": { + "balance": "10011000000000000000000" + }, + "8667d005eCF50Eb247890a11FCdCfC321DC1Da9f": { + "balance": "10011000000000000000000" + }, + "5Ce612A664C2f35558Dcab7edb999619e155CD07": { + "balance": "10011000000000000000000" + }, + "aD95f88cCd3aBC12ddd6cD0b9a777B95339b747b": { + "balance": "10011000000000000000000" + }, + "6E5a5A2963F6d0C2EA26682a152fE3ac7CBC1227": { + "balance": "10011000000000000000000" + }, + "000000000000000000000000000000000000ce10": { + "code": "0x60806040526004361061004a5760003560e01c806303386ba3146101e757806342404e0714610280578063bb913f41146102d7578063d29d44ee14610328578063f7e6af8014610379575b6000600160405180807f656970313936372e70726f78792e696d706c656d656e746174696f6e00000000815250601c019050604051809103902060001c0360001b9050600081549050600073ffffffffffffffffffffffffffffffffffffffff168173ffffffffffffffffffffffffffffffffffffffff161415610136576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260158152602001807f4e6f20496d706c656d656e746174696f6e20736574000000000000000000000081525060200191505060405180910390fd5b61013f816103d0565b6101b1576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260188152602001807f496e76616c696420636f6e74726163742061646472657373000000000000000081525060200191505060405180910390fd5b60405136810160405236600082376000803683855af43d604051818101604052816000823e82600081146101e3578282f35b8282fd5b61027e600480360360408110156101fd57600080fd5b81019080803573ffffffffffffffffffffffffffffffffffffffff1690602001909291908035906020019064010000000081111561023a57600080fd5b82018360208201111561024c57600080fd5b8035906020019184600183028401116401000000008311171561026e57600080fd5b909192939192939050505061041b565b005b34801561028c57600080fd5b506102956105c1565b604051808273ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff16815260200191505060405180910390f35b3480156102e357600080fd5b50610326600480360360208110156102fa57600080fd5b81019080803573ffffffffffffffffffffffffffffffffffffffff16906020019092919050505061060d565b005b34801561033457600080fd5b506103776004803603602081101561034b57600080fd5b81019080803573ffffffffffffffffffffffffffffffffffffffff1690602001909291905050506107bd565b005b34801561038557600080fd5b5061038e610871565b604051808273ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff16815260200191505060405180910390f35b60008060007fc5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a47060001b9050833f915080821415801561041257506000801b8214155b92505050919050565b610423610871565b73ffffffffffffffffffffffffffffffffffffffff163373ffffffffffffffffffffffffffffffffffffffff16146104c3576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260148152602001807f73656e64657220776173206e6f74206f776e657200000000000000000000000081525060200191505060405180910390fd5b6104cc8361060d565b600060608473ffffffffffffffffffffffffffffffffffffffff168484604051808383808284378083019250505092505050600060405180830381855af49150503d8060008114610539576040519150601f19603f3d011682016040523d82523d6000602084013e61053e565b606091505b508092508193505050816105ba576040517f08c379a000000000000000000000000000000000000000000000000000000000815260040180806020018281038252601e8152602001807f696e697469616c697a6174696f6e2063616c6c6261636b206661696c6564000081525060200191505060405180910390fd5b5050505050565b600080600160405180807f656970313936372e70726f78792e696d706c656d656e746174696f6e00000000815250601c019050604051809103902060001c0360001b9050805491505090565b610615610871565b73ffffffffffffffffffffffffffffffffffffffff163373ffffffffffffffffffffffffffffffffffffffff16146106b5576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260148152602001807f73656e64657220776173206e6f74206f776e657200000000000000000000000081525060200191505060405180910390fd5b6000600160405180807f656970313936372e70726f78792e696d706c656d656e746174696f6e00000000815250601c019050604051809103902060001c0360001b9050610701826103d0565b610773576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260188152602001807f496e76616c696420636f6e74726163742061646472657373000000000000000081525060200191505060405180910390fd5b8181558173ffffffffffffffffffffffffffffffffffffffff167fab64f92ab780ecbf4f3866f57cee465ff36c89450dcce20237ca7a8d81fb7d1360405160405180910390a25050565b6107c5610871565b73ffffffffffffffffffffffffffffffffffffffff163373ffffffffffffffffffffffffffffffffffffffff1614610865576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260148152602001807f73656e64657220776173206e6f74206f776e657200000000000000000000000081525060200191505060405180910390fd5b61086e816108bd565b50565b600080600160405180807f656970313936372e70726f78792e61646d696e000000000000000000000000008152506013019050604051809103902060001c0360001b9050805491505090565b600073ffffffffffffffffffffffffffffffffffffffff168173ffffffffffffffffffffffffffffffffffffffff161415610960576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260118152602001807f6f776e65722063616e6e6f74206265203000000000000000000000000000000081525060200191505060405180910390fd5b6000600160405180807f656970313936372e70726f78792e61646d696e000000000000000000000000008152506013019050604051809103902060001c0360001b90508181558173ffffffffffffffffffffffffffffffffffffffff167f50146d0e3c60aa1d17a70635b05494f864e86144a2201275021014fbf08bafe260405160405180910390a2505056fea165627a7a72305820959a50d5df76f90bc1825042f47788ee27f1b4725f7ed5d37c5c05c0732ef44f0029", + "storage": { + "0xb53127684a568b3173ae13b9f8a6016e243e63b6e8ee1178d6a717850b5d6103": "0x0Cc59Ed03B3e763c02d54D695FFE353055f1502D" + }, + "balance": "0" + } +}` + +const alfajoresAllocJSON = `{ + "456f41406B32c45D59E539e4BBA3D7898c3584dA": { + "balance": "103010030000000000000000000" + }, + "DD1F519F63423045F526b8c83edC0eB4BA6434a4": { + "balance": "10011000000000000000000" + }, + "050f34537F5b2a00B9B9C752Cb8500a3fcE3DA7d": { + "balance": "10011000000000000000000" + }, + "Cda518F6b5a797C3EC45D37c65b83e0b0748eDca": { + "balance": "10011000000000000000000" + }, + "b4e92c94A2712e98c020A81868264bdE52C188Cb": { + "balance": "10011000000000000000000" + }, + "Ae1ec841923811219b98ACeB1db297AADE2F46F3": { + "balance": "10011000000000000000000" + }, + "621843731fe33418007C06ee48CfD71e0ea828d9": { + "balance": "10011000000000000000000" + }, + "2A43f97f8BF959E31F69A894ebD80A88572C8553": { + "balance": "10011000000000000000000" + }, + "AD682035bE6Ab6f06e478D2BDab0EAb6477B460E": { + "balance": "10011000000000000000000" + }, + "30D060F129817c4DE5fBc1366d53e19f43c8c64f": { + "balance": "10011000000000000000000" + }, + "22579CA45eE22E2E16dDF72D955D6cf4c767B0eF": { + "balance": "10011000000000000000000" + }, + "1173C5A50bf025e8356823a068E396ccF2bE696C": { + "balance": "10011000000000000000000" + }, + "40F71B525A96baa8d14Eaa7Bcd19929782659c64": { + "balance": "10011000000000000000000" + }, + "b923626C6f1d237252793FB2aA12BA21328C51BC": { + "balance": "10011000000000000000000" + }, + "B70f9ABf41F36B3ab60cc9aE1a85Ddda3C88D261": { + "balance": "10011000000000000000000" + }, + "d4369DB59eaDc4Cfa089c0a3c1004ceAb1b318D8": { + "balance": "10011000000000000000000" + }, + "2fd430d3a96eadc38cc1B38b6685C5f52Cf7a083": { + "balance": "10011000000000000000000" + }, + "Fecc71C8f33Ca5952534fd346ADdeDC38DBb9cb7": { + "balance": "10011000000000000000000" + }, + "0de78C89e7BF5060f28dd3f820C15C4A6A81AFB5": { + "balance": "10011000000000000000000" + }, + "75411b92fcE120C1e7fd171b1c2bF802f2E3CF48": { + "balance": "10011000000000000000000" + }, + "563433bD8357b06982Fe001df20B2b43393d21d2": { + "balance": "10011000000000000000000" + }, + "79dfB9d2367E7921d4139D7841d24ED82F48907F": { + "balance": "10011000000000000000000" + }, + "5809369FC5121a071eE67659a975e88ae40fBE3b": { + "balance": "10011000000000000000000" + }, + "7517E54a456bcc6c5c695B5d9f97EBc05d29a824": { + "balance": "10011000000000000000000" + }, + "B0a1A5Ffcb34E6Fa278D2b40613f0AE1042d32f8": { + "balance": "10011000000000000000000" + }, + "EeE9f4DDf49976251E84182AbfD3300Ee58D12aa": { + "balance": "10011000000000000000000" + }, + "Eb5Fd57f87a4e1c7bAa53ec1c0d021bb1710B743": { + "balance": "10011000000000000000000" + }, + "B7Dd51bFb73c5753778e5Af56f1D9669BCe6777F": { + "balance": "10011000000000000000000" + }, + "33C222BB13C63295AF32D6C91278AA34b573e776": { + "balance": "10011000000000000000000" + }, + "83c58603bF72DA067D7f6238E7bF390d91B2f531": { + "balance": "10011000000000000000000" + }, + "6651112198C0da05921355642a2B8dF1fA3Ede93": { + "balance": "10011000000000000000000" + }, + "4EE72A98549eA7CF774C3E2E1b39fF166b4b68BE": { + "balance": "10011000000000000000000" + }, + "840b32F30e1a3b2E8b9E6C0972eBa0148E22B847": { + "balance": "100000000000000000000" + }, + "000000000000000000000000000000000000ce10": { + "code": "0x60806040526004361061004a5760003560e01c806303386ba3146101e757806342404e0714610280578063bb913f41146102d7578063d29d44ee14610328578063f7e6af8014610379575b6000600160405180807f656970313936372e70726f78792e696d706c656d656e746174696f6e00000000815250601c019050604051809103902060001c0360001b9050600081549050600073ffffffffffffffffffffffffffffffffffffffff168173ffffffffffffffffffffffffffffffffffffffff161415610136576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260158152602001807f4e6f20496d706c656d656e746174696f6e20736574000000000000000000000081525060200191505060405180910390fd5b61013f816103d0565b6101b1576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260188152602001807f496e76616c696420636f6e74726163742061646472657373000000000000000081525060200191505060405180910390fd5b60405136810160405236600082376000803683855af43d604051818101604052816000823e82600081146101e3578282f35b8282fd5b61027e600480360360408110156101fd57600080fd5b81019080803573ffffffffffffffffffffffffffffffffffffffff1690602001909291908035906020019064010000000081111561023a57600080fd5b82018360208201111561024c57600080fd5b8035906020019184600183028401116401000000008311171561026e57600080fd5b909192939192939050505061041b565b005b34801561028c57600080fd5b506102956105c1565b604051808273ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff16815260200191505060405180910390f35b3480156102e357600080fd5b50610326600480360360208110156102fa57600080fd5b81019080803573ffffffffffffffffffffffffffffffffffffffff16906020019092919050505061060d565b005b34801561033457600080fd5b506103776004803603602081101561034b57600080fd5b81019080803573ffffffffffffffffffffffffffffffffffffffff1690602001909291905050506107bd565b005b34801561038557600080fd5b5061038e610871565b604051808273ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff16815260200191505060405180910390f35b60008060007fc5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a47060001b9050833f915080821415801561041257506000801b8214155b92505050919050565b610423610871565b73ffffffffffffffffffffffffffffffffffffffff163373ffffffffffffffffffffffffffffffffffffffff16146104c3576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260148152602001807f73656e64657220776173206e6f74206f776e657200000000000000000000000081525060200191505060405180910390fd5b6104cc8361060d565b600060608473ffffffffffffffffffffffffffffffffffffffff168484604051808383808284378083019250505092505050600060405180830381855af49150503d8060008114610539576040519150601f19603f3d011682016040523d82523d6000602084013e61053e565b606091505b508092508193505050816105ba576040517f08c379a000000000000000000000000000000000000000000000000000000000815260040180806020018281038252601e8152602001807f696e697469616c697a6174696f6e2063616c6c6261636b206661696c6564000081525060200191505060405180910390fd5b5050505050565b600080600160405180807f656970313936372e70726f78792e696d706c656d656e746174696f6e00000000815250601c019050604051809103902060001c0360001b9050805491505090565b610615610871565b73ffffffffffffffffffffffffffffffffffffffff163373ffffffffffffffffffffffffffffffffffffffff16146106b5576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260148152602001807f73656e64657220776173206e6f74206f776e657200000000000000000000000081525060200191505060405180910390fd5b6000600160405180807f656970313936372e70726f78792e696d706c656d656e746174696f6e00000000815250601c019050604051809103902060001c0360001b9050610701826103d0565b610773576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260188152602001807f496e76616c696420636f6e74726163742061646472657373000000000000000081525060200191505060405180910390fd5b8181558173ffffffffffffffffffffffffffffffffffffffff167fab64f92ab780ecbf4f3866f57cee465ff36c89450dcce20237ca7a8d81fb7d1360405160405180910390a25050565b6107c5610871565b73ffffffffffffffffffffffffffffffffffffffff163373ffffffffffffffffffffffffffffffffffffffff1614610865576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260148152602001807f73656e64657220776173206e6f74206f776e657200000000000000000000000081525060200191505060405180910390fd5b61086e816108bd565b50565b600080600160405180807f656970313936372e70726f78792e61646d696e000000000000000000000000008152506013019050604051809103902060001c0360001b9050805491505090565b600073ffffffffffffffffffffffffffffffffffffffff168173ffffffffffffffffffffffffffffffffffffffff161415610960576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260118152602001807f6f776e65722063616e6e6f74206265203000000000000000000000000000000081525060200191505060405180910390fd5b6000600160405180807f656970313936372e70726f78792e61646d696e000000000000000000000000008152506013019050604051809103902060001c0360001b90508181558173ffffffffffffffffffffffffffffffffffffffff167f50146d0e3c60aa1d17a70635b05494f864e86144a2201275021014fbf08bafe260405160405180910390a2505056fea165627a7a723058202dbb6037e4381b4ad95015ed99441a23345cc2ae52ef27e2e91d34fb0acd277b0029", + "storage": { + "0xb53127684a568b3173ae13b9f8a6016e243e63b6e8ee1178d6a717850b5d6103": "456f41406B32c45D59E539e4BBA3D7898c3584dA" + }, + "balance": "0" + } +}` + +const mainnetAllocJSON = "{\"0x11901cf7eEae1E2644995FB2E47Ce46bC7F33246\":{\"balance\":\"120000000000000000000000000\"},\"0xC1cDA18694F5B86cFB80c1B4f8Cc046B0d7E6326\":{\"balance\":\"20000000000000000000000000\"},\"0xa5d40D93b01AfBafec84E20018Aff427628F645E\":{\"balance\":\"20000000000000000000000000\"},\"0x8d485780E84E23437f8F6938D96B964645529127\":{\"balance\":\"20000000000000000000000000\"},\"0x5F857c501b73ddFA804234f1f1418D6f75554076\":{\"balance\":\"20000000000000000000000000\"},\"0xaa9064F57F8d7de4b3e08c35561E21Afd6341390\":{\"balance\":\"20000000000000000000000000\"},\"0x7FA26b50b3e9a2eC8AD1850a4c4FBBF94D806E95\":{\"balance\":\"20000000000000000000000000\"},\"0x08960Ce6b58BE32FBc6aC1489d04364B4f7dC216\":{\"balance\":\"20000000000000000000000000\"},\"0x77B68B2e7091D4F242a8Af89F200Af941433C6d8\":{\"balance\":\"20000000000000000000000000\"},\"0x75Bb69C002C43f5a26a2A620518775795Fd45ecf\":{\"balance\":\"20000000000000000000000000\"},\"0x19992AE48914a178Bf138665CffDD8CD79b99513\":{\"balance\":\"20000000000000000000000000\"},\"0xE23a4c6615669526Ab58E9c37088bee4eD2b2dEE\":{\"balance\":\"20000000000000000000000\"},\"0xDe22679dCA843B424FD0BBd70A22D5F5a4B94fe4\":{\"balance\":\"10200014000000000000000000\"},\"0x743D80810fe10c5C3346D2940997cC9647035B13\":{\"balance\":\"20513322000000000000000000\"},\"0x8e1c4355307F1A59E7eD4Ae057c51368b9338C38\":{\"balance\":\"7291740000000000000000000\"},\"0x417fe63186C388812e342c85FF87187Dc584C630\":{\"balance\":\"20000062000000000000000000\"},\"0xF5720c180a6Fa14ECcE82FB1bB060A39E93A263c\":{\"balance\":\"30000061000000000000000000\"},\"0xB80d1e7F9CEbe4b5E1B1Acf037d3a44871105041\":{\"balance\":\"9581366833333333333333335\"},\"0xf8ed78A113cD2a34dF451Ba3D540FFAE66829AA0\":{\"balance\":\"11218686833333333333333333\"},\"0x9033ff75af27222c8f36a148800c7331581933F3\":{\"balance\":\"11218686833333333333333333\"},\"0x8A07541C2eF161F4e3f8de7c7894718dA26626B2\":{\"balance\":\"11218686833333333333333333\"},\"0xB2fe7AFe178335CEc3564d7671EEbD7634C626B0\":{\"balance\":\"11218686833333333333333333\"},\"0xc471776eA02705004C451959129bF09423B56526\":{\"balance\":\"11218686833333333333333333\"},\"0xeF283eca68DE87E051D427b4be152A7403110647\":{\"balance\":\"14375000000000000000000000\"},\"0x7cf091C954ed7E9304452d31fd59999505Ddcb7a\":{\"balance\":\"14375000000000000000000000\"},\"0xa5d2944C32a8D7b284fF0b84c20fDcc46937Cf64\":{\"balance\":\"14375000000000000000000000\"},\"0xFC89C17525f08F2Bc9bA8cb77BcF05055B1F7059\":{\"balance\":\"14375000000000000000000000\"},\"0x3Fa7C646599F3174380BD9a7B6efCde90b5d129d\":{\"balance\":\"14375000000000000000000000\"},\"0x989e1a3B344A43911e02cCC609D469fbc15AB1F1\":{\"balance\":\"14375000000000000000000000\"},\"0xAe1d640648009DbE0Aa4485d3BfBB68C37710924\":{\"balance\":\"20025000000000000000000000\"},\"0x1B6C64779F42BA6B54C853Ab70171aCd81b072F7\":{\"balance\":\"20025000000000000000000000\"},\"000000000000000000000000000000000000ce10\":{\"code\":\"0x60806040526004361061004a5760003560e01c806303386ba3146101e757806342404e0714610280578063bb913f41146102d7578063d29d44ee14610328578063f7e6af8014610379575b6000600160405180807f656970313936372e70726f78792e696d706c656d656e746174696f6e00000000815250601c019050604051809103902060001c0360001b9050600081549050600073ffffffffffffffffffffffffffffffffffffffff168173ffffffffffffffffffffffffffffffffffffffff161415610136576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260158152602001807f4e6f20496d706c656d656e746174696f6e20736574000000000000000000000081525060200191505060405180910390fd5b61013f816103d0565b6101b1576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260188152602001807f496e76616c696420636f6e74726163742061646472657373000000000000000081525060200191505060405180910390fd5b60405136810160405236600082376000803683855af43d604051818101604052816000823e82600081146101e3578282f35b8282fd5b61027e600480360360408110156101fd57600080fd5b81019080803573ffffffffffffffffffffffffffffffffffffffff1690602001909291908035906020019064010000000081111561023a57600080fd5b82018360208201111561024c57600080fd5b8035906020019184600183028401116401000000008311171561026e57600080fd5b909192939192939050505061041b565b005b34801561028c57600080fd5b506102956105c1565b604051808273ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff16815260200191505060405180910390f35b3480156102e357600080fd5b50610326600480360360208110156102fa57600080fd5b81019080803573ffffffffffffffffffffffffffffffffffffffff16906020019092919050505061060d565b005b34801561033457600080fd5b506103776004803603602081101561034b57600080fd5b81019080803573ffffffffffffffffffffffffffffffffffffffff1690602001909291905050506107bd565b005b34801561038557600080fd5b5061038e610871565b604051808273ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff16815260200191505060405180910390f35b60008060007fc5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a47060001b9050833f915080821415801561041257506000801b8214155b92505050919050565b610423610871565b73ffffffffffffffffffffffffffffffffffffffff163373ffffffffffffffffffffffffffffffffffffffff16146104c3576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260148152602001807f73656e64657220776173206e6f74206f776e657200000000000000000000000081525060200191505060405180910390fd5b6104cc8361060d565b600060608473ffffffffffffffffffffffffffffffffffffffff168484604051808383808284378083019250505092505050600060405180830381855af49150503d8060008114610539576040519150601f19603f3d011682016040523d82523d6000602084013e61053e565b606091505b508092508193505050816105ba576040517f08c379a000000000000000000000000000000000000000000000000000000000815260040180806020018281038252601e8152602001807f696e697469616c697a6174696f6e2063616c6c6261636b206661696c6564000081525060200191505060405180910390fd5b5050505050565b600080600160405180807f656970313936372e70726f78792e696d706c656d656e746174696f6e00000000815250601c019050604051809103902060001c0360001b9050805491505090565b610615610871565b73ffffffffffffffffffffffffffffffffffffffff163373ffffffffffffffffffffffffffffffffffffffff16146106b5576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260148152602001807f73656e64657220776173206e6f74206f776e657200000000000000000000000081525060200191505060405180910390fd5b6000600160405180807f656970313936372e70726f78792e696d706c656d656e746174696f6e00000000815250601c019050604051809103902060001c0360001b9050610701826103d0565b610773576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260188152602001807f496e76616c696420636f6e74726163742061646472657373000000000000000081525060200191505060405180910390fd5b8181558173ffffffffffffffffffffffffffffffffffffffff167fab64f92ab780ecbf4f3866f57cee465ff36c89450dcce20237ca7a8d81fb7d1360405160405180910390a25050565b6107c5610871565b73ffffffffffffffffffffffffffffffffffffffff163373ffffffffffffffffffffffffffffffffffffffff1614610865576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260148152602001807f73656e64657220776173206e6f74206f776e657200000000000000000000000081525060200191505060405180910390fd5b61086e816108bd565b50565b600080600160405180807f656970313936372e70726f78792e61646d696e000000000000000000000000008152506013019050604051809103902060001c0360001b9050805491505090565b600073ffffffffffffffffffffffffffffffffffffffff168173ffffffffffffffffffffffffffffffffffffffff161415610960576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260118152602001807f6f776e65722063616e6e6f74206265203000000000000000000000000000000081525060200191505060405180910390fd5b6000600160405180807f656970313936372e70726f78792e61646d696e000000000000000000000000008152506013019050604051809103902060001c0360001b90508181558173ffffffffffffffffffffffffffffffffffffffff167f50146d0e3c60aa1d17a70635b05494f864e86144a2201275021014fbf08bafe260405160405180910390a2505056fea165627a7a723058206808dd43e7d765afca53fe439122bc5eac16d708ce7d463451be5042426f101f0029\",\"storage\":{\"0xb53127684a568b3173ae13b9f8a6016e243e63b6e8ee1178d6a717850b5d6103\":\"0xE23a4c6615669526Ab58E9c37088bee4eD2b2dEE\"},\"balance\":\"0\"}}" + +var celoL1GenesisAllocJSON = map[uint64]string{ + MainnetNetworkID: mainnetAllocJSON, + AlfajoresNetworkID: alfajoresAllocJSON, + BaklavaNetworkID: baklavaAllocJSON, +} + +// GetCeloL1GenesisAlloc returns the legacy Celo L1 genesis allocation JSON for the given network ID. +func GetCeloL1GenesisAlloc(config *params.ChainConfig) ([]byte, error) { + chainID := config.ChainID.Uint64() + allocJSON, ok := celoL1GenesisAllocJSON[chainID] + if !ok { + return nil, fmt.Errorf("no genesis allocation JSON found for network ID %d", chainID) + } + return []byte(allocJSON), nil +} + +// BuildGenesis creates a genesis block from the given parameters. +func BuildGenesis(config *params.ChainConfig, allocs, extraData []byte, timestamp uint64) (*core.Genesis, error) { + genesisAlloc := &types.GenesisAlloc{} + if err := genesisAlloc.UnmarshalJSON(allocs); err != nil { + return nil, err + } + return &core.Genesis{ + Config: config, + Timestamp: timestamp, + ExtraData: extraData, + Alloc: *genesisAlloc, + }, nil +} diff --git a/op-chain-ops/cmd/celo-migrate/main.go b/op-chain-ops/cmd/celo-migrate/main.go new file mode 100644 index 000000000000..c75732a9e9ed --- /dev/null +++ b/op-chain-ops/cmd/celo-migrate/main.go @@ -0,0 +1,454 @@ +package main + +import ( + "context" + "errors" + "fmt" + "math/big" + "os" + "os/exec" + "runtime/debug" + "time" + + "log/slog" + + "github.com/ethereum-optimism/optimism/op-chain-ops/foundry" + "github.com/ethereum-optimism/optimism/op-chain-ops/genesis" + "github.com/ethereum-optimism/optimism/op-service/ioutil" + "github.com/ethereum-optimism/optimism/op-service/jsonutil" + oplog "github.com/ethereum-optimism/optimism/op-service/log" + "github.com/mattn/go-isatty" + + "github.com/urfave/cli/v2" + + "github.com/ethereum/go-ethereum/core/rawdb" + "github.com/ethereum/go-ethereum/core/types" + "github.com/ethereum/go-ethereum/ethclient" + "github.com/ethereum/go-ethereum/log" + "github.com/ethereum/go-ethereum/rpc" + + "golang.org/x/sync/errgroup" +) + +var ( + deployConfigFlag = &cli.PathFlag{ + Name: "deploy-config", + Usage: "Path to the JSON file that was used for the bedrock contracts deployment. A test example can be found here 'op-chain-ops/genesis/testdata/test-deploy-config-full.json' and documentation for the fields is at https://docs.optimism.io/builders/chain-operators/management/configuration", + Required: true, + } + l1DeploymentsFlag = &cli.PathFlag{ + Name: "l1-deployments", + Usage: "Path to L1 deployments JSON file, the output of running the bedrock contracts deployment for the given 'deploy-config'", + Required: true, + } + l1RPCFlag = &cli.StringFlag{ + Name: "l1-rpc", + Usage: "RPC URL for a node of the L1 defined in the 'deploy-config'", + Required: true, + } + l2AllocsFlag = &cli.PathFlag{ + Name: "l2-allocs", + Usage: "Path to L2 genesis allocs file. You can find instructions on how to generate this file in the README", + Required: true, + } + outfileRollupConfigFlag = &cli.PathFlag{ + Name: "outfile.rollup-config", + Usage: "Path to write the rollup config JSON file, to be provided to op-node with the 'rollup.config' flag", + Required: true, + } + outfileGenesisFlag = &cli.PathFlag{ + Name: "outfile.genesis", + Usage: "Path to write the genesis JSON file, to be used to sync new nodes", + Required: true, + } + migrationBlockNumberFlag = &cli.Uint64Flag{ + Name: "migration-block-number", + Usage: "Specifies the migration block number. If the source db is not synced exactly to the block immediately before this number (i.e. migration-block-number - 1), the migration will fail.", + Required: true, + } + migrationBlockTimeFlag = &cli.Uint64Flag{ + Name: "migration-block-time", + Usage: "Specifies a unix timestamp to use for the migration block. If not provided, the current time will be used.", + } + oldDBPathFlag = &cli.PathFlag{ + Name: "old-db", + Usage: "Path to the old Celo chaindata dir, can be found at '/celo/chaindata'", + Required: true, + } + newDBPathFlag = &cli.PathFlag{ + Name: "new-db", + Usage: "Path to write migrated Celo chaindata, note the new node implementation expects to find this chaindata at the following path '/geth/chaindata", + Required: true, + } + batchSizeFlag = &cli.Uint64Flag{ + Name: "batch-size", + Usage: "Batch size to use for block migration, larger batch sizes can speed up migration but require more memory. If increasing the batch size consider also increasing the memory-limit", + Value: 50000, // TODO(Alec) optimize default parameters + } + bufferSizeFlag = &cli.Uint64Flag{ + Name: "buffer-size", + Usage: "Buffer size to use for ancient block migration channels. Defaults to 0. Included to facilitate testing for performance improvements.", + Value: 0, + } + memoryLimitFlag = &cli.Int64Flag{ + Name: "memory-limit", + Usage: "Memory limit in MiB, should be set lower than the available amount of memory in your system to prevent out of memory errors", + Value: 7500, + } + reset = &cli.BoolFlag{ + Name: "reset", + Usage: "Delete everything in the destination directory aside from /ancients. This is useful if you need to re-run the full migration but do not want to repeat the lengthy ancients migration. If you'd like to reset the entire destination directory, you can delete it manually.", + Value: false, + } + + preMigrationFlags = []cli.Flag{ + oldDBPathFlag, + newDBPathFlag, + batchSizeFlag, + bufferSizeFlag, + memoryLimitFlag, + reset, + } + fullMigrationFlags = append( + preMigrationFlags, + deployConfigFlag, + l1DeploymentsFlag, + l1RPCFlag, + l2AllocsFlag, + outfileRollupConfigFlag, + outfileGenesisFlag, + migrationBlockTimeFlag, + migrationBlockNumberFlag, + ) +) + +type preMigrationOptions struct { + oldDBPath string + newDBPath string + batchSize uint64 + bufferSize uint64 + memoryLimit int64 + resetNonAncients bool +} + +type stateMigrationOptions struct { + deployConfig string + l1Deployments string + l1RPC string + l2AllocsPath string + outfileRollupConfig string + outfileGenesis string + migrationBlockTime uint64 +} + +type fullMigrationOptions struct { + preMigrationOptions + stateMigrationOptions + migrationBlockNumber uint64 +} + +func parsePreMigrationOptions(ctx *cli.Context) preMigrationOptions { + return preMigrationOptions{ + oldDBPath: ctx.String(oldDBPathFlag.Name), + newDBPath: ctx.String(newDBPathFlag.Name), + batchSize: ctx.Uint64(batchSizeFlag.Name), + bufferSize: ctx.Uint64(bufferSizeFlag.Name), + memoryLimit: ctx.Int64(memoryLimitFlag.Name), + resetNonAncients: ctx.Bool(reset.Name), + } +} + +func parseStateMigrationOptions(ctx *cli.Context) stateMigrationOptions { + return stateMigrationOptions{ + deployConfig: ctx.Path(deployConfigFlag.Name), + l1Deployments: ctx.Path(l1DeploymentsFlag.Name), + l1RPC: ctx.String(l1RPCFlag.Name), + l2AllocsPath: ctx.Path(l2AllocsFlag.Name), + outfileRollupConfig: ctx.Path(outfileRollupConfigFlag.Name), + outfileGenesis: ctx.Path(outfileGenesisFlag.Name), + migrationBlockTime: ctx.Uint64(migrationBlockTimeFlag.Name), + } +} + +func parseFullMigrationOptions(ctx *cli.Context) fullMigrationOptions { + return fullMigrationOptions{ + preMigrationOptions: parsePreMigrationOptions(ctx), + stateMigrationOptions: parseStateMigrationOptions(ctx), + migrationBlockNumber: ctx.Uint64(migrationBlockNumberFlag.Name), + } +} + +func main() { + + color := isatty.IsTerminal(os.Stderr.Fd()) + handler := log.NewTerminalHandlerWithLevel(os.Stderr, slog.LevelInfo, color) + oplog.SetGlobalLogHandler(handler) + + app := &cli.App{ + Name: "celo-migrate", + Usage: "Migrate Celo block and state data to a CeL2 DB", + Commands: []*cli.Command{ + { + Name: "pre", + Usage: "Perform a pre-migration of ancient blocks and copy over all other data without transforming it. This should be run a day before the full migration command is run to minimize downtime.", + Flags: preMigrationFlags, + Action: func(ctx *cli.Context) error { + if _, _, err := runPreMigration(parsePreMigrationOptions(ctx)); err != nil { + return fmt.Errorf("failed to run pre-migration: %w", err) + } + log.Info("Finished pre migration successfully!") + return nil + }, + }, + { + Name: "full", + Usage: "Perform a full migration of both block and state data to a CeL2 DB", + Flags: fullMigrationFlags, + Action: func(ctx *cli.Context) error { + if err := runFullMigration(parseFullMigrationOptions(ctx)); err != nil { + return fmt.Errorf("failed to run full migration: %w", err) + } + log.Info("Finished full migration successfully!") + return nil + }, + }, + }, + OnUsageError: func(ctx *cli.Context, err error, isSubcommand bool) error { + if isSubcommand { + return err + } + _ = cli.ShowAppHelp(ctx) + return fmt.Errorf("please provide a valid command") + }, + } + + if err := app.Run(os.Args); err != nil { + log.Crit("error in migration", "err", err) + } +} + +func runFullMigration(opts fullMigrationOptions) error { + defer timer("full migration")() + + log.Info("Full Migration Started", "oldDBPath", opts.oldDBPath, "newDBPath", opts.newDBPath) + + head, err := getHeadHeader(opts.oldDBPath) + if err != nil { + return fmt.Errorf("failed to get head header: %w", err) + } + if head.Number.Uint64() != opts.migrationBlockNumber-1 { + return fmt.Errorf("old-db head block number not synced to the block immediately before the migration block number: %d != %d", head.Number.Uint64(), opts.migrationBlockNumber-1) + } + + log.Info("Source db is synced to correct height", "head", head.Number.Uint64(), "migrationBlock", opts.migrationBlockNumber) + + var numAncients uint64 + var strayAncientBlocks []*rawdb.NumberHash + + if strayAncientBlocks, numAncients, err = runPreMigration(opts.preMigrationOptions); err != nil { + return fmt.Errorf("failed to run pre-migration: %w", err) + } + + if err = runNonAncientMigration(opts.newDBPath, strayAncientBlocks, opts.batchSize, numAncients); err != nil { + return fmt.Errorf("failed to run non-ancient migration: %w", err) + } + if err = runStateMigration(opts.newDBPath, opts.stateMigrationOptions); err != nil { + return fmt.Errorf("failed to run state migration: %w", err) + } + + log.Info("Full Migration Finished", "oldDBPath", opts.oldDBPath, "newDBPath", opts.newDBPath) + + return nil +} + +func runPreMigration(opts preMigrationOptions) ([]*rawdb.NumberHash, uint64, error) { + defer timer("pre-migration")() + + log.Info("Pre-Migration Started", "oldDBPath", opts.oldDBPath, "newDBPath", opts.newDBPath, "batchSize", opts.batchSize, "memoryLimit", opts.memoryLimit) + + // Check that `rsync` command is available. We use this to copy the db excluding ancients, which we will copy separately + if _, err := exec.LookPath("rsync"); err != nil { + return nil, 0, fmt.Errorf("please install `rsync` to run block migration") + } + + debug.SetMemoryLimit(opts.memoryLimit * 1 << 20) // Set memory limit, converting from MiB to bytes + + var err error + + if err = createNewDbPathIfNotExists(opts.newDBPath); err != nil { + return nil, 0, fmt.Errorf("failed to create new db path: %w", err) + } + + if opts.resetNonAncients { + if err = cleanupNonAncientDb(opts.newDBPath); err != nil { + return nil, 0, fmt.Errorf("failed to cleanup non-ancient db: %w", err) + } + } + + var numAncientsNewBefore uint64 + var numAncientsNewAfter uint64 + var strayAncientBlocks []*rawdb.NumberHash + g, ctx := errgroup.WithContext(context.Background()) + g.Go(func() error { + if numAncientsNewBefore, numAncientsNewAfter, err = migrateAncientsDb(ctx, opts.oldDBPath, opts.newDBPath, opts.batchSize, opts.bufferSize); err != nil { + return fmt.Errorf("failed to migrate ancients database: %w", err) + } + // Scanning for stray ancient blocks is slow, so we do it as soon as we can after the lock on oldDB is released by migrateAncientsDb + // Doing this in parallel with copyDbExceptAncients still saves time if ancients have already been pre-migrated + if strayAncientBlocks, err = getStrayAncientBlocks(opts.oldDBPath); err != nil { + return fmt.Errorf("failed to get stray ancient blocks: %w", err) + } + return nil + }) + g.Go(func() error { + // By doing this once during the premigration, we get a speedup when we run it again in a full migration. + return copyDbExceptAncients(opts.oldDBPath, opts.newDBPath) + }) + + if err = g.Wait(); err != nil { + return nil, 0, fmt.Errorf("failed to migrate blocks: %w", err) + } + + log.Info("Pre-Migration Finished", "oldDBPath", opts.oldDBPath, "newDBPath", opts.newDBPath, "migratedAncients", numAncientsNewAfter-numAncientsNewBefore, "strayAncientBlocks", len(strayAncientBlocks)) + + return strayAncientBlocks, numAncientsNewAfter, nil +} + +func runNonAncientMigration(newDBPath string, strayAncientBlocks []*rawdb.NumberHash, batchSize, numAncients uint64) error { + defer timer("non-ancient migration")() + + newDB, err := openDBWithoutFreezer(newDBPath, false) + if err != nil { + return fmt.Errorf("failed to open new database: %w", err) + } + defer newDB.Close() + + // get the last block number + hash := rawdb.ReadHeadHeaderHash(newDB) + lastBlock := *rawdb.ReadHeaderNumber(newDB, hash) + lastAncient := numAncients - 1 + + log.Info("Non-Ancient Block Migration Started", "process", "non-ancients", "newDBPath", newDBPath, "batchSize", batchSize, "startBlock", numAncients, "endBlock", lastBlock, "count", lastBlock-lastAncient, "lastAncientBlock", lastAncient) + + var numNonAncients uint64 + if numNonAncients, err = migrateNonAncientsDb(newDB, lastBlock, numAncients, batchSize); err != nil { + return fmt.Errorf("failed to migrate non-ancients database: %w", err) + } + + err = removeBlocks(newDB, strayAncientBlocks) + if err != nil { + return fmt.Errorf("failed to remove stray ancient blocks: %w", err) + } + log.Info("Removed stray ancient blocks still in leveldb", "process", "non-ancients", "removedBlocks", len(strayAncientBlocks)) + + log.Info("Non-Ancient Block Migration Completed", "process", "non-ancients", "migratedNonAncients", numNonAncients) + + return nil +} + +func runStateMigration(newDBPath string, opts stateMigrationOptions) error { + defer timer("state migration")() + + log.Info("State Migration Started", "newDBPath", newDBPath, "deployConfig", opts.deployConfig, "l1Deployments", opts.l1Deployments, "l1RPC", opts.l1RPC, "l2AllocsPath", opts.l2AllocsPath, "outfileRollupConfig", opts.outfileRollupConfig) + + // Read deployment configuration + config, err := genesis.NewDeployConfig(opts.deployConfig) + if err != nil { + return err + } + + if config.DeployCeloContracts { + return errors.New("DeployCeloContracts is not supported in migration") + } + if config.FundDevAccounts { + return errors.New("FundDevAccounts is not supported in migration") + } + + // Try reading the L1 deployment information + deployments, err := genesis.NewL1Deployments(opts.l1Deployments) + if err != nil { + return fmt.Errorf("cannot read L1 deployments at %s: %w", opts.l1Deployments, err) + } + config.SetDeployments(deployments) + + // Get latest block information from L1 + var l1StartBlock *types.Block + client, err := ethclient.Dial(opts.l1RPC) + if err != nil { + return fmt.Errorf("cannot dial %s: %w", opts.l1RPC, err) + } + + if config.L1StartingBlockTag == nil { + l1StartBlock, err = client.BlockByNumber(context.Background(), nil) + if err != nil { + return fmt.Errorf("cannot fetch latest block: %w", err) + } + tag := rpc.BlockNumberOrHashWithHash(l1StartBlock.Hash(), true) + config.L1StartingBlockTag = (*genesis.MarshalableRPCBlockNumberOrHash)(&tag) + } else if config.L1StartingBlockTag.BlockHash != nil { + l1StartBlock, err = client.BlockByHash(context.Background(), *config.L1StartingBlockTag.BlockHash) + if err != nil { + return fmt.Errorf("cannot fetch block by hash: %w", err) + } + } else if config.L1StartingBlockTag.BlockNumber != nil { + l1StartBlock, err = client.BlockByNumber(context.Background(), big.NewInt(config.L1StartingBlockTag.BlockNumber.Int64())) + if err != nil { + return fmt.Errorf("cannot fetch block by number: %w", err) + } + } + + // Ensure that there is a starting L1 block + if l1StartBlock == nil { + return fmt.Errorf("no starting L1 block") + } + + // Sanity check the config. Do this after filling in the L1StartingBlockTag + // if it is not defined. + if err := config.Check(log.New()); err != nil { + return err + } + + log.Info("Using L1 Start Block", "number", l1StartBlock.Number(), "hash", l1StartBlock.Hash().Hex()) + + // Build the L2 genesis block + l2Allocs, err := foundry.LoadForgeAllocs(opts.l2AllocsPath) + if err != nil { + return err + } + + l2Genesis, err := genesis.BuildL2Genesis(config, l2Allocs, l1StartBlock.Header()) + if err != nil { + return fmt.Errorf("error creating l2 genesis: %w", err) + } + + // Write changes to state to actual state database + cel2Header, err := applyStateMigrationChanges(config, l2Genesis.Alloc, newDBPath, opts.outfileGenesis, opts.migrationBlockTime, l1StartBlock) + if err != nil { + return err + } + log.Info("Updated Cel2 state") + + rollupConfig, err := config.RollupConfig(l1StartBlock.Header(), cel2Header.Hash(), cel2Header.Number.Uint64()) + if err != nil { + return err + } + if err := rollupConfig.Check(); err != nil { + return fmt.Errorf("generated rollup config does not pass validation: %w", err) + } + + log.Info("Writing rollup config", "file", opts.outfileRollupConfig) + if err := jsonutil.WriteJSON(rollupConfig, ioutil.ToStdOutOrFileOrNoop(opts.outfileRollupConfig, OutFilePerm)); err != nil { + return err + } + + log.Info("State Migration Completed") + + return nil +} + +func timer(name string) func() { + start := time.Now() + return func() { + log.Info("TIMER", "process", name, "duration", time.Since(start)) + } +} diff --git a/op-chain-ops/cmd/celo-migrate/non-ancients.go b/op-chain-ops/cmd/celo-migrate/non-ancients.go new file mode 100644 index 000000000000..5ad184e2dd24 --- /dev/null +++ b/op-chain-ops/cmd/celo-migrate/non-ancients.go @@ -0,0 +1,118 @@ +package main + +import ( + "fmt" + "os" + "os/exec" + "strings" + + "github.com/ethereum/go-ethereum/common" + "github.com/ethereum/go-ethereum/core/rawdb" + "github.com/ethereum/go-ethereum/ethdb" + "github.com/ethereum/go-ethereum/log" +) + +func copyDbExceptAncients(oldDbPath, newDbPath string) error { + defer timer("copyDbExceptAncients")() + + log.Info("Copying files from old database (excluding ancients)", "process", "non-ancients") + + // Get rsync help output + cmdHelp := exec.Command("rsync", "--help") + output, _ := cmdHelp.CombinedOutput() + + // Convert output to string + outputStr := string(output) + + opts := []string{"-v", "-a", "--exclude=ancient", "--checksum", "--delete"} + + // Check for supported options + // Prefer --info=progress2 over --progress + if strings.Contains(outputStr, "--info") { + opts = append(opts, "--info=progress2") + } else if strings.Contains(outputStr, "--progress") { + opts = append(opts, "--progress") + } + + cmd := exec.Command("rsync", append(opts, oldDbPath+"/", newDbPath)...) + + // rsync copies any file with a different timestamp or size. + // + // '--exclude=ancient' excludes the ancient directory from the copy + // + // '--delete' Tells rsync to delete extraneous files from the receiving side (ones that aren’t on the sending side) + // + // '-a' archive mode; equals -rlptgoD. It is a quick way of saying you want recursion and want to preserve almost everything, including timestamps, ownerships, permissions, etc. + // Timestamps are important here because they are used to determine which files are newer and should be copied over. + // + // '--whole-file' This is the default when both the source and destination are specified as local paths, which they are here (oldDbPath and newDbPath). + // This option disables rsync’s delta-transfer algorithm, which causes all transferred files to be sent whole. The delta-transfer algorithm is normally used when the destination is a remote system. + // + // '--checksum' This forces rsync to compare the checksums of all files to determine if they are the same. This is slows down the transfer but ensures that source and destination directories end up with the same contents (excluding /ancients). + + log.Info("Running rsync command", "command", cmd.String()) + cmd.Stdout = os.Stdout + cmd.Stderr = os.Stderr + if err := cmd.Run(); err != nil { + return fmt.Errorf("failed to copy old database to new database: %w", err) + } + return nil +} + +func migrateNonAncientsDb(newDB ethdb.Database, lastBlock, numAncients, batchSize uint64) (uint64, error) { + defer timer("migrateNonAncientsDb")() + + // The genesis block is also migrated in the ancient db migration as it is stored in both places. + // The genesis block is the only block that should remain stored in the non-ancient db even after it is frozen. + if numAncients > 0 { + log.Info("Migrating genesis block in non-ancient db", "process", "non-ancients") + if err := migrateNonAncientBlock(0, rawdb.ReadCanonicalHash(newDB, 0), newDB); err != nil { + return 0, err + } + } + + for i := numAncients; i <= lastBlock; i += batchSize { + numbersHash := rawdb.ReadAllHashesInRange(newDB, i, i+batchSize-1) + + log.Info("Processing Block Range", "process", "non-ancients", "from", i, "to(inclusve)", i+batchSize-1, "count", len(numbersHash)) + for _, numberHash := range numbersHash { + if err := migrateNonAncientBlock(numberHash.Number, numberHash.Hash, newDB); err != nil { + return 0, err + } + } + } + + migratedCount := lastBlock - numAncients + 1 + return migratedCount, nil +} + +func migrateNonAncientBlock(number uint64, hash common.Hash, newDB ethdb.Database) error { + // read header and body + header := rawdb.ReadHeaderRLP(newDB, hash, number) + body := rawdb.ReadBodyRLP(newDB, hash, number) + + // transform header and body + newHeader, err := transformHeader(header) + if err != nil { + return fmt.Errorf("failed to transform header: block %d - %x: %w", number, hash, err) + } + newBody, err := transformBlockBody(body) + if err != nil { + return fmt.Errorf("failed to transform body: block %d - %x: %w", number, hash, err) + } + + if yes, newHash := hasSameHash(newHeader, hash[:]); !yes { + log.Error("Hash mismatch", "block", number, "oldHash", hash, "newHash", newHash) + return fmt.Errorf("hash mismatch at block %d - %x", number, hash) + } + + // write header and body + batch := newDB.NewBatch() + rawdb.WriteBodyRLP(batch, hash, number, newBody) + _ = batch.Put(headerKey(number, hash), newHeader) + if err := batch.Write(); err != nil { + return fmt.Errorf("failed to write header and body: block %d - %x: %w", number, hash, err) + } + + return nil +} diff --git a/op-chain-ops/cmd/celo-migrate/state.go b/op-chain-ops/cmd/celo-migrate/state.go new file mode 100644 index 000000000000..960d0a771814 --- /dev/null +++ b/op-chain-ops/cmd/celo-migrate/state.go @@ -0,0 +1,451 @@ +package main + +import ( + "encoding/json" + "errors" + "fmt" + "math/big" + "os" + "time" + + "github.com/ethereum-optimism/optimism/op-chain-ops/genesis" + "github.com/ethereum-optimism/optimism/op-service/ioutil" + "github.com/ethereum-optimism/optimism/op-service/jsonutil" + "github.com/ethereum-optimism/optimism/op-service/predeploys" + "github.com/ethereum/go-ethereum/common" + "github.com/ethereum/go-ethereum/contracts/addresses" + "github.com/ethereum/go-ethereum/core/rawdb" + "github.com/ethereum/go-ethereum/core/state" + "github.com/ethereum/go-ethereum/core/tracing" + "github.com/ethereum/go-ethereum/core/types" + "github.com/ethereum/go-ethereum/core/vm" + "github.com/ethereum/go-ethereum/ethdb" + "github.com/ethereum/go-ethereum/log" + "github.com/ethereum/go-ethereum/params" + "github.com/ethereum/go-ethereum/trie" + + "github.com/holiman/uint256" +) + +const ( + MainnetNetworkID = uint64(42220) + BaklavaNetworkID = uint64(62320) + AlfajoresNetworkID = uint64(44787) + + OutFilePerm = os.FileMode(0o440) +) + +var ( + Big10 = uint256.NewInt(10) + Big9 = uint256.NewInt(9) + Big18 = uint256.NewInt(18) + + // Allowlist of accounts that are allowed to be overwritten + // If the value for an account is set to true, the nonce and storage will be overwritten + // This must be checked for each account, as this might create issues with contracts + // calling `CREATE` or `CREATE2` + accountOverwriteAllowlist = map[uint64]map[common.Address]bool{ + // Add any addresses that should be allowed to overwrite existing accounts here. + AlfajoresNetworkID: { + // Create2Deployer + // OP uses a version without an owner who can pause the contract, + // so we overwrite the existing contract during migration + common.HexToAddress("0x13b0D85CcB8bf860b6b79AF3029fCA081AE9beF2"): true, + + // Same code as in allocs file + // EntryPoint_v070 + common.HexToAddress("0x0000000071727De22E5E9d8BAf0edAc6f37da032"): false, + // Permit2 + common.HexToAddress("0x000000000022D473030F116dDEE9F6B43aC78BA3"): false, + // EntryPoint_v060 + common.HexToAddress("0x5FF137D4b0FDCD49DcA30c7CF57E578a026d2789"): false, + // DeterministicDeploymentProxy + common.HexToAddress("0x4e59b44847b379578588920cA78FbF26c0B4956C"): false, + // SafeL2_v130 + common.HexToAddress("0xfb1bffC9d739B8D520DaF37dF666da4C687191EA"): false, + // MultiSend_v130 + common.HexToAddress("0x998739BFdAAdde7C933B942a68053933098f9EDa"): false, + // SenderCreator_v070 + common.HexToAddress("0xEFC2c1444eBCC4Db75e7613d20C6a62fF67A167C"): false, + // SenderCreator_v060 + common.HexToAddress("0x7fc98430eAEdbb6070B35B39D798725049088348"): false, + // MultiCall3 + common.HexToAddress("0xcA11bde05977b3631167028862bE2a173976CA11"): false, + // Safe_v130 + common.HexToAddress("0x69f4D1788e39c87893C980c06EdF4b7f686e2938"): false, + // MultiSendCallOnly_v130 + common.HexToAddress("0xA1dabEF33b3B82c7814B6D82A79e50F4AC44102B"): false, + // SafeSingletonFactory + common.HexToAddress("0x914d7Fec6aaC8cd542e72Bca78B30650d45643d7"): false, + // CreateX + common.HexToAddress("0xba5Ed099633D3B313e4D5F7bdc1305d3c28ba5Ed"): false, + }, + } + unreleasedTreasuryAddressMap = map[uint64]common.Address{ + AlfajoresNetworkID: common.HexToAddress("0x07bf0b2461A0cb608D5CF9a82ba97dAbA850F79F"), + } + celoTokenAddressMap = map[uint64]common.Address{ + AlfajoresNetworkID: addresses.CeloTokenAlfajoresAddress, + MainnetNetworkID: addresses.CeloTokenAddress, + } +) + +func applyStateMigrationChanges(config *genesis.DeployConfig, l2Allocs types.GenesisAlloc, dbPath, genesisOutPath string, migrationBlockTime uint64, l1StartBlock *types.Block) (*types.Header, error) { + log.Info("Opening Celo database", "dbPath", dbPath) + + ldb, err := openDBWithoutFreezer(dbPath, false) + if err != nil { + return nil, fmt.Errorf("cannot open DB: %w", err) + } + log.Info("Loaded Celo L1 DB", "db", ldb) + + // Grab the hash of the tip of the legacy chain. + hash := rawdb.ReadHeadHeaderHash(ldb) + log.Info("Reading chain tip from database", "hash", hash) + + // Grab the header number. + num := rawdb.ReadHeaderNumber(ldb, hash) + if num == nil { + return nil, fmt.Errorf("cannot find header number for %s", hash) + } + log.Info("Reading chain tip num from database", "number", *num) + + // Grab the full header. + header := rawdb.ReadHeader(ldb, hash, *num) + log.Info("Read header from database", "header", header) + + // We need to update the chain config to set the correct hardforks. + genesisHash := rawdb.ReadCanonicalHash(ldb, 0) + cfg := rawdb.ReadChainConfig(ldb, genesisHash) + if cfg == nil { + log.Crit("chain config not found") + } + log.Info("Read chain config from database", "config", cfg) + + // Set up the backing store. + underlyingDB := state.NewDatabase(ldb) + + // Open up the state database. + db, err := state.New(header.Root, underlyingDB, nil) + if err != nil { + return nil, fmt.Errorf("cannot open StateDB: %w", err) + } + + // Apply the changes to the state DB. + err = applyAllocsToState(db, l2Allocs, accountOverwriteAllowlist[cfg.ChainID.Uint64()]) + if err != nil { + return nil, fmt.Errorf("cannot apply allocations to state: %w", err) + } + + // Initialize the unreleased treasury contract + // This uses the original config which won't enable recent hardforks (and things like the PUSH0 opcode) + // This is fine, as the token uses solc 0.5.x and therefore compatible bytecode + err = setupUnreleasedTreasury(db, cfg) + if err != nil { + // An error here shouldn't stop the migration, just log it + log.Warn("Error setting up unreleased treasury", "error", err) + } + + migrationBlock := new(big.Int).Add(header.Number, common.Big1) + + // We're done messing around with the database, so we can now commit the changes to the DB. + // Note that this doesn't actually write the changes to disk. + log.Info("Committing state DB") + newRoot, err := db.Commit(migrationBlock.Uint64(), true) + if err != nil { + return nil, err + } + + baseFee := new(big.Int).SetUint64(params.InitialBaseFee) + if header.BaseFee != nil { + baseFee = header.BaseFee + } + + if migrationBlockTime == 0 { + migrationBlockTime = uint64(time.Now().Unix()) + } + + // If gas limit was zero at the transition point use a default of 30M. + // Note that in op-geth we use gasLimit==0 to indicate a pre-gingerbread + // block and adjust encoding appropriately, so we must make sure that + // gasLimit is non-zero, bacause L2 blocks are all post gingerbread. + gasLimit := header.GasLimit + if gasLimit == 0 { + gasLimit = 30e6 + } + // Create the header for the Cel2 transition block. + cel2Header := &types.Header{ + ParentHash: header.Hash(), + UncleHash: types.EmptyUncleHash, + Coinbase: predeploys.SequencerFeeVaultAddr, + Root: newRoot, + TxHash: types.EmptyTxsHash, + ReceiptHash: types.EmptyReceiptsHash, + Bloom: types.Bloom{}, + Difficulty: new(big.Int).Set(common.Big0), + Number: migrationBlock, + GasLimit: gasLimit, + GasUsed: 0, + Time: migrationBlockTime, + Extra: []byte("Celo L2 migration"), + MixDigest: common.Hash{}, + Nonce: types.BlockNonce{}, + BaseFee: baseFee, + // Added during Shanghai hardfork + // As there're no withdrawals in L2, we set it to the empty hash + WithdrawalsHash: &types.EmptyWithdrawalsHash, + // Blobs are disabled in L2 + BlobGasUsed: new(uint64), + ExcessBlobGas: new(uint64), + // This is set to the ParentBeaconRoot of the L1 origin (see `PreparePayloadAttributes`) + // Use the L1 start block's ParentBeaconRoot + ParentBeaconRoot: l1StartBlock.Header().ParentBeaconRoot, + } + log.Info("Build Cel2 migration header", "header", cel2Header) + + // We need to set empty withdrawals in the body, otherwise types.NewBlock will nullify the withdrawals hash in the given header. + b := &types.Body{ + Withdrawals: []*types.Withdrawal{}, + } + // Create the Cel2 transition block from the header. Note that there are no transactions, + // uncle blocks, or receipts in the Cel2 transition block. + cel2Block := types.NewBlock(cel2Header, b, nil, trie.NewStackTrie(nil)) + + // We did it! + log.Info( + "Built Cel2 migration block", + "hash", cel2Block.Hash(), + "root", cel2Block.Root(), + "number", cel2Block.NumberU64(), + ) + + log.Info("Committing trie DB") + if err := db.Database().TrieDB().Commit(newRoot, true); err != nil { + return nil, err + } + + // Next we write the Cel2 migration block to the database. + rawdb.WriteTd(ldb, cel2Block.Hash(), cel2Block.NumberU64(), cel2Block.Difficulty()) + rawdb.WriteBlock(ldb, cel2Block) + rawdb.WriteReceipts(ldb, cel2Block.Hash(), cel2Block.NumberU64(), nil) + rawdb.WriteCanonicalHash(ldb, cel2Block.Hash(), cel2Block.NumberU64()) + rawdb.WriteHeadBlockHash(ldb, cel2Block.Hash()) + rawdb.WriteHeadFastBlockHash(ldb, cel2Block.Hash()) + rawdb.WriteHeadHeaderHash(ldb, cel2Block.Hash()) + + // Mark the first CeL2 block as finalized + rawdb.WriteFinalizedBlockHash(ldb, cel2Block.Hash()) + + // Set the standard options. + cfg.LondonBlock = cel2Block.Number() + cfg.BerlinBlock = cel2Block.Number() + cfg.ArrowGlacierBlock = cel2Block.Number() + cfg.GrayGlacierBlock = cel2Block.Number() + cfg.MergeNetsplitBlock = cel2Block.Number() + cfg.TerminalTotalDifficulty = big.NewInt(0) + cfg.TerminalTotalDifficultyPassed = true + cfg.ShanghaiTime = &cel2Header.Time + cfg.CancunTime = &cel2Header.Time + + // Set the Optimism options. + cfg.Optimism = ¶ms.OptimismConfig{ + EIP1559Denominator: config.EIP1559Denominator, + EIP1559DenominatorCanyon: &config.EIP1559DenominatorCanyon, + EIP1559Elasticity: config.EIP1559Elasticity, + } + // Set Optimism hardforks + cfg.BedrockBlock = cel2Block.Number() + cfg.RegolithTime = &cel2Header.Time + cfg.CanyonTime = &cel2Header.Time + cfg.EcotoneTime = &cel2Header.Time + cfg.FjordTime = &cel2Header.Time + cfg.GraniteTime = &cel2Header.Time + cfg.Cel2Time = &cel2Header.Time + + // Write the chain config to disk. + rawdb.WriteChainConfig(ldb, genesisHash, cfg) + marshalledConfig, err := json.Marshal(cfg) + if err != nil { + return nil, fmt.Errorf("failed to marshal chain config to JSON: %w", err) + } + log.Info("Wrote updated chain config", "config", string(marshalledConfig)) + + // Write genesis JSON to outfile and store genesis state spec in the database. + if err = writeGenesis(cfg, ldb, genesisOutPath, genesisHash); err != nil { + return nil, err + } + + // We're done! + log.Info( + "Wrote CeL2 migration block", + "height", cel2Header.Number, + "root", cel2Header.Root.String(), + "hash", cel2Header.Hash().String(), + "timestamp", cel2Header.Time, + ) + + // Close the database handle + if err := ldb.Close(); err != nil { + return nil, err + } + + return cel2Header, nil +} + +// applyAllocsToState applies the account allocations from the allocation file to the state database. +// It creates new accounts, sets their nonce, balance, code, and storage values. +// If an account already exists, it adds the balance of the new account to the existing balance. +// If the code of an existing account is different from the code in the genesis block, it logs a warning. +// This changes the state root, so `Commit` needs to be called after this function. +func applyAllocsToState(db vm.StateDB, allocs types.GenesisAlloc, allowlist map[common.Address]bool) error { + log.Info("Starting to migrate OP contracts into state DB") + + copyCounter := 0 + overwriteCounter := 0 + + for k, v := range allocs { + // Check that the balance of the account to written is zero, + // as we must not create new CELO tokens + if v.Balance != nil && v.Balance.Cmp(big.NewInt(0)) != 0 { + return fmt.Errorf("account balance is not zero, would change celo supply: %s", k.Hex()) + } + + if db.Exist(k) { + writeNonceAndStorage := false + writeCode, allowed := allowlist[k] + + // If the account is not allowed and has a non zero nonce or code size, bail out we will need to manually investigate how to handle this. + if !allowed && (db.GetCodeSize(k) > 0 || db.GetNonce(k) > 0) { + return fmt.Errorf("account exists and is not allowed, account: %s, nonce: %d, code: %d", k.Hex(), db.GetNonce(k), db.GetCode(k)) + } + + // This means that the account just has balance, in that case we wan to copy over the account + if db.GetCodeSize(k) == 0 && db.GetNonce(k) == 0 { + writeCode = true + writeNonceAndStorage = true + } + + if writeCode { + overwriteCounter++ + + db.SetCode(k, v.Code) + + if writeNonceAndStorage { + db.SetNonce(k, v.Nonce) + for key, value := range v.Storage { + db.SetState(k, key, value) + } + } + log.Info("Overwrote account", "address", k.Hex(), "writeNonceAndStorage", writeNonceAndStorage) + } + continue + } + + // Account does not exist, create it + db.CreateAccount(k) + db.SetCode(k, v.Code) + db.SetNonce(k, v.Nonce) + for key, value := range v.Storage { + db.SetState(k, key, value) + } + + copyCounter++ + log.Info("Copied account", "address", k.Hex()) + } + + log.Info("Migrated OP contracts into state DB", "totalAllocs", len(allocs), "copiedAccounts", copyCounter, "overwrittenAccounts", overwriteCounter) + return nil +} + +// setupUnreleasedTreasury sets up the unreleased treasury contract with the correct balance +// The balance is set to the difference between the ceiling and the total supply of the token +func setupUnreleasedTreasury(db *state.StateDB, config *params.ChainConfig) error { + log.Info("Setting up CeloUnreleasedTreasury balance") + + celoUnreleasedTreasuryAddress, exists := unreleasedTreasuryAddressMap[config.ChainID.Uint64()] + if !exists { + return errors.New("CeloUnreleasedTreasury address not configured for this chain, skipping migration step") + } + + if !db.Exist(celoUnreleasedTreasuryAddress) { + return errors.New("CeloUnreleasedTreasury account does not exist, skipping migration step") + } + + tokenAddress, exists := celoTokenAddressMap[config.ChainID.Uint64()] + if !exists { + return errors.New("celo token address not configured for this chain, skipping migration step") + } + log.Info("Read contract addresses", "tokenAddress", tokenAddress, "celoUnreleasedTreasuryAddress", celoUnreleasedTreasuryAddress) + + // totalSupply is stored in the third slot + totalSupply := db.GetState(tokenAddress, common.HexToHash("0x02")).Big() + + // Get total supply of celo token + billion := new(uint256.Int).Exp(Big10, Big9) + ethInWei := new(uint256.Int).Exp(Big10, Big18) + + ceiling := new(uint256.Int).Mul(billion, ethInWei) + + supplyU256 := uint256.MustFromBig(totalSupply) + if supplyU256.Cmp(ceiling) > 0 { + return fmt.Errorf("supply %s is greater than ceiling %s", totalSupply, ceiling) + } + + balance := new(uint256.Int).Sub(ceiling, supplyU256) + // Don't discard existing balance of the account + balance = new(uint256.Int).Add(balance, db.GetBalance(celoUnreleasedTreasuryAddress)) + db.SetBalance(celoUnreleasedTreasuryAddress, balance, tracing.BalanceChangeUnspecified) + + log.Info("Set up CeloUnreleasedTreasury balance", "celoUnreleasedTreasuryAddress", celoUnreleasedTreasuryAddress, "balance", balance, "total_supply", supplyU256, "ceiling", ceiling) + return nil +} + +// writeGenesis writes the genesis json to --outfile.genesis and stores the genesis state spec (alloc) in the database. +// Note that this is different than the cel2Block / migration block. Rather, this is the migrated genesis block of Celo from before the L2 transition. +// Nodes will need the genesis json file in order to snap sync on the L2 chain. +func writeGenesis(config *params.ChainConfig, db ethdb.Database, genesisOutPath string, genesisHash common.Hash) error { + // Derive the genesis object using hardcoded legacy alloc and the transformed extra data stored in the new db. + legacyGenesisAlloc, err := GetCeloL1GenesisAlloc(config) + if err != nil { + return err + } + genesisHeader := rawdb.ReadHeader(db, genesisHash, 0) + genesis, err := BuildGenesis(config, legacyGenesisAlloc, genesisHeader.Extra, genesisHeader.Time) + if err != nil { + return err + } + + // Convert genesis to JSON byte slice + genesisBytes, err := json.Marshal(genesis) + if err != nil { + return fmt.Errorf("failed to marshal genesis to JSON: %w", err) + } + + // Unmarshal JSON byte slice to map + var genesisMap map[string]interface{} + if err := json.Unmarshal(genesisBytes, &genesisMap); err != nil { + return fmt.Errorf("failed to unmarshal genesis JSON to map: %w", err) + } + + // Delete fields that are not in Celo Legacy Genesis, otherwise genesis hashes won't match when syncing + delete(genesisMap, "difficulty") + delete(genesisMap, "gasLimit") + delete(genesisMap, "excessBlobGas") + delete(genesisMap, "blobGasUsed") + delete(genesisMap, "baseFeePerGas") + delete(genesisMap, "mixHash") + delete(genesisMap, "nonce") + + // Write the modified JSON to the file + if err := jsonutil.WriteJSON(genesisMap, ioutil.ToStdOutOrFileOrNoop(genesisOutPath, OutFilePerm)); err != nil { + return fmt.Errorf("failed to write genesis JSON to file: %w", err) + } + log.Info("Wrote genesis file for syncing new nodes", "path", genesisOutPath) + + // Legacy Celo did not store the genesis state spec (alloc) in the database. + // Write it now for forward compatibility. + rawdb.WriteGenesisStateSpec(db, genesisHash, legacyGenesisAlloc) + log.Info("Wrote genesis state spec (alloc) to database") + + return nil +} diff --git a/op-chain-ops/cmd/celo-migrate/state_test.go b/op-chain-ops/cmd/celo-migrate/state_test.go new file mode 100644 index 000000000000..dbe6ce90ef4e --- /dev/null +++ b/op-chain-ops/cmd/celo-migrate/state_test.go @@ -0,0 +1,147 @@ +package main + +import ( + "bytes" + "math/big" + "testing" + + "github.com/ethereum/go-ethereum/common" + "github.com/ethereum/go-ethereum/core/rawdb" + "github.com/ethereum/go-ethereum/core/state" + "github.com/ethereum/go-ethereum/core/tracing" + "github.com/ethereum/go-ethereum/core/types" + "github.com/holiman/uint256" + "github.com/stretchr/testify/assert" +) + +var ( + contractCode = []byte{0x01, 0x02} + defaultBalance int64 = 123 + address = common.HexToAddress("a") +) + +func TestApplyAllocsToState(t *testing.T) { + tests := []struct { + name string + existingAccount *types.Account + newAccount types.Account + allowlist map[common.Address]bool + wantErr bool + }{ + { + name: "Write to empty account", + newAccount: types.Account{ + Code: contractCode, + Nonce: 1, + }, + wantErr: false, + }, + { + name: "Copy account with non-zero balance fails", + existingAccount: &types.Account{ + Balance: big.NewInt(defaultBalance), + }, + newAccount: types.Account{ + Balance: big.NewInt(1), + }, + wantErr: true, + }, + { + name: "Write to account with only balance should overwrite and keep balance", + existingAccount: &types.Account{ + Balance: big.NewInt(defaultBalance), + }, + newAccount: types.Account{ + Code: contractCode, + Nonce: 5, + }, + wantErr: false, + }, + { + name: "Write to account with existing nonce fails", + existingAccount: &types.Account{ + Balance: big.NewInt(defaultBalance), + Nonce: 5, + }, + newAccount: types.Account{ + Code: contractCode, + Nonce: 5, + }, + wantErr: true, + }, + { + name: "Write to account with contract code fails", + existingAccount: &types.Account{ + Balance: big.NewInt(defaultBalance), + Code: bytes.Repeat([]byte{0x01}, 10), + }, + newAccount: types.Account{ + Code: contractCode, + Nonce: 5, + }, + wantErr: true, + }, + { + name: "Write account with allowlist overwrite, keeps nonce", + existingAccount: &types.Account{ + Balance: big.NewInt(defaultBalance), + Nonce: 4, + Code: bytes.Repeat([]byte{0x01}, 10), + }, + newAccount: types.Account{ + Code: contractCode, + Nonce: 5, + }, + allowlist: map[common.Address]bool{address: true}, + wantErr: false, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + db := rawdb.NewMemoryDatabase() + tdb := state.NewDatabase(db) + sdb, _ := state.New(types.EmptyRootHash, tdb, nil) + + if tt.existingAccount != nil { + sdb.CreateAccount(address) + + if tt.existingAccount.Balance != nil { + sdb.SetBalance(address, uint256.MustFromBig(tt.existingAccount.Balance), tracing.BalanceChangeUnspecified) + } + if tt.existingAccount.Nonce != 0 { + sdb.SetNonce(address, tt.existingAccount.Nonce) + } + if tt.existingAccount.Code != nil { + sdb.SetCode(address, tt.existingAccount.Code) + } + } + + if err := applyAllocsToState(sdb, types.GenesisAlloc{address: tt.newAccount}, tt.allowlist); (err != nil) != tt.wantErr { + t.Errorf("applyAllocsToState() error = %v, wantErr %v", err, tt.wantErr) + } + + // Don't check account state if an error was thrown + if tt.wantErr { + return + } + + if !sdb.Exist(address) { + t.Errorf("account does not exists as expected: %v", address.Hex()) + } + + assert.Equal(t, tt.newAccount.Code, sdb.GetCode(address)) + + if tt.existingAccount != nil && tt.existingAccount.Nonce != 0 { + assert.Equal(t, tt.existingAccount.Nonce, sdb.GetNonce(address)) + } else { + assert.Equal(t, tt.newAccount.Nonce, sdb.GetNonce(address)) + } + + if tt.existingAccount != nil { + assert.True(t, big.NewInt(defaultBalance).Cmp(sdb.GetBalance(address).ToBig()) == 0) + } else { + assert.True(t, big.NewInt(0).Cmp(sdb.GetBalance(address).ToBig()) == 0) + } + }) + } +} diff --git a/op-chain-ops/cmd/celo-migrate/testdata/deploy-config-dango.json b/op-chain-ops/cmd/celo-migrate/testdata/deploy-config-dango.json new file mode 100644 index 000000000000..dd69e13ab4f9 --- /dev/null +++ b/op-chain-ops/cmd/celo-migrate/testdata/deploy-config-dango.json @@ -0,0 +1,91 @@ + { + "l1StartingBlockTag": "0xe18e94c26beea64e318e25c32303c9a1ee2bfcee4492337bb5ac14181e99bd0c", + + "l1ChainID": 17000, + "l2ChainID": 44787, + "l2BlockTime": 2, + "l1BlockTime": 12, + + "maxSequencerDrift": 600, + "sequencerWindowSize": 3600, + "channelTimeout": 300, + + "p2pSequencerAddress": "0x644C82d76A43Fe9c76eda0EEd0f0DC17235c3005", + "batchInboxAddress": "0xff00000000000000000000000000000000044787", + "batchSenderAddress": "0x1660B1F70De0f32490b50f976e8983213dCF7FD5", + + "l2OutputOracleSubmissionInterval": 120, + "l2OutputOracleStartingBlockNumber": 26216760, + "l2OutputOracleStartingTimestamp": 1726651200, + + "l2OutputOracleProposer": "0x1BA11Ec6581FC8C3e35D6E345aEC977796Ffe89b", + "l2OutputOracleChallenger": "0xc07C5A1fBF6c7BC6b4f321E7dd031c0E1E98d32d", + + "finalizationPeriodSeconds": 12, + + "proxyAdminOwner": "0xc07C5A1fBF6c7BC6b4f321E7dd031c0E1E98d32d", + "baseFeeVaultRecipient": "0xc07C5A1fBF6c7BC6b4f321E7dd031c0E1E98d32d", + "l1FeeVaultRecipient": "0xc07C5A1fBF6c7BC6b4f321E7dd031c0E1E98d32d", + "sequencerFeeVaultRecipient": "0xc07C5A1fBF6c7BC6b4f321E7dd031c0E1E98d32d", + "finalSystemOwner": "0xc07C5A1fBF6c7BC6b4f321E7dd031c0E1E98d32d", + "superchainConfigGuardian": "0xc07C5A1fBF6c7BC6b4f321E7dd031c0E1E98d32d", + + "baseFeeVaultMinimumWithdrawalAmount": "0x8ac7230489e80000", + "l1FeeVaultMinimumWithdrawalAmount": "0x8ac7230489e80000", + "sequencerFeeVaultMinimumWithdrawalAmount": "0x8ac7230489e80000", + "baseFeeVaultWithdrawalNetwork": 0, + "l1FeeVaultWithdrawalNetwork": 0, + "sequencerFeeVaultWithdrawalNetwork": 0, + + "gasPriceOracleOverhead": 0, + "gasPriceOracleScalar": 1000000, + + "deployCeloContracts": false, + + "enableGovernance": false, + "governanceTokenSymbol": "OP", + "governanceTokenName": "Optimism", + "governanceTokenOwner": "0xc07C5A1fBF6c7BC6b4f321E7dd031c0E1E98d32d", + + "l2GenesisBlockGasLimit": "0x1c9c380", + "l2GenesisBlockBaseFeePerGas": "0x3b9aca00", + + "eip1559Denominator": 50, + "eip1559DenominatorCanyon": 250, + "eip1559Elasticity": 6, + "l2GenesisFjordTimeOffset": "0x0", + "l2GenesisRegolithTimeOffset": "0x0", + "l2GenesisEcotoneTimeOffset": "0x0", + "l2GenesisDeltaTimeOffset": "0x0", + "l2GenesisCanyonTimeOffset": "0x0", + "systemConfigStartBlock": 0, + + "requiredProtocolVersion": "0x0000000000000000000000000000000000000000000000000000000000000000", + "recommendedProtocolVersion": "0x0000000000000000000000000000000000000000000000000000000000000000", + + "faultGameAbsolutePrestate": "0x03c7ae758795765c6664a5d39bf63841c71ff191e9189522bad8ebff5d4eca98", + "faultGameMaxDepth": 44, + "faultGameClockExtension": 0, + "faultGameMaxClockDuration": 1200, + "faultGameGenesisBlock": 0, + "faultGameGenesisOutputRoot": "0x0000000000000000000000000000000000000000000000000000000000000000", + "faultGameSplitDepth": 14, + "faultGameWithdrawalDelay": 600, + + "preimageOracleMinProposalSize": 1800000, + "preimageOracleChallengePeriod": 300, + + "fundDevAccounts": false, + "useFaultProofs": false, + "proofMaturityDelaySeconds": 604800, + "disputeGameFinalityDelaySeconds": 302400, + "respectedGameType": 0, + + "useAltDA": true, + "daCommitmentType": "GenericCommitment", + "daChallengeWindow": 1, + "daResolveWindow": 1, + + "useCustomGasToken": true, + "customGasTokenAddress": "0x0000000000000000000000000000000000000000" +} diff --git a/op-chain-ops/cmd/celo-migrate/testdata/deploy-config-holesky-alfajores.json b/op-chain-ops/cmd/celo-migrate/testdata/deploy-config-holesky-alfajores.json new file mode 100644 index 000000000000..6b9dbe97e068 --- /dev/null +++ b/op-chain-ops/cmd/celo-migrate/testdata/deploy-config-holesky-alfajores.json @@ -0,0 +1,89 @@ +{ + "l1StartingBlockTag": "0xbbed3612407993e67f8ca7a423b181837ae164a531941e78f5ee48e766d39cad", + + "l1ChainID": 17000, + "l2ChainID": 44787, + "l2BlockTime": 2, + "l1BlockTime": 12, + + "maxSequencerDrift": 600, + "sequencerWindowSize": 3600, + "channelTimeout": 300, + + "p2pSequencerAddress": "0x644C82d76A43Fe9c76eda0EEd0f0DC17235c3005", + "batchInboxAddress": "0xff00000000000000000000000000000000044787", + "batchSenderAddress": "0x1660B1F70De0f32490b50f976e8983213dCF7FD5", + + "l2OutputOracleSubmissionInterval": 120, + "l2OutputOracleStartingBlockNumber": 0, + "l2OutputOracleStartingTimestamp": 1718312256, + + "l2OutputOracleProposer": "0x1BA11Ec6581FC8C3e35D6E345aEC977796Ffe89b", + "l2OutputOracleChallenger": "0xc07C5A1fBF6c7BC6b4f321E7dd031c0E1E98d32d", + + "finalizationPeriodSeconds": 12, + + "proxyAdminOwner": "0xc07C5A1fBF6c7BC6b4f321E7dd031c0E1E98d32d", + "baseFeeVaultRecipient": "0xc07C5A1fBF6c7BC6b4f321E7dd031c0E1E98d32d", + "l1FeeVaultRecipient": "0xc07C5A1fBF6c7BC6b4f321E7dd031c0E1E98d32d", + "sequencerFeeVaultRecipient": "0xc07C5A1fBF6c7BC6b4f321E7dd031c0E1E98d32d", + "finalSystemOwner": "0xc07C5A1fBF6c7BC6b4f321E7dd031c0E1E98d32d", + "superchainConfigGuardian": "0xc07C5A1fBF6c7BC6b4f321E7dd031c0E1E98d32d", + + "baseFeeVaultMinimumWithdrawalAmount": "0x8ac7230489e80000", + "l1FeeVaultMinimumWithdrawalAmount": "0x8ac7230489e80000", + "sequencerFeeVaultMinimumWithdrawalAmount": "0x8ac7230489e80000", + "baseFeeVaultWithdrawalNetwork": 0, + "l1FeeVaultWithdrawalNetwork": 0, + "sequencerFeeVaultWithdrawalNetwork": 0, + + "gasPriceOracleOverhead": 0, + "gasPriceOracleScalar": 1000000, + + "enableGovernance": false, + "governanceTokenSymbol": "OP", + "governanceTokenName": "Optimism", + "governanceTokenOwner": "0xc07C5A1fBF6c7BC6b4f321E7dd031c0E1E98d32d", + + "l2GenesisBlockGasLimit": "0x1c9c380", + "l2GenesisBlockBaseFeePerGas": "0x3b9aca00", + "l2GenesisRegolithTimeOffset": "0x0", + + "eip1559Denominator": 50, + "eip1559DenominatorCanyon": 250, + "eip1559Elasticity": 6, + + "l2GenesisEcotoneTimeOffset": "0x0", + "l2GenesisDeltaTimeOffset": "0x0", + "l2GenesisCanyonTimeOffset": "0x0", + + "systemConfigStartBlock": 0, + + "requiredProtocolVersion": "0x0000000000000000000000000000000000000000000000000000000000000000", + "recommendedProtocolVersion": "0x0000000000000000000000000000000000000000000000000000000000000000", + + "faultGameAbsolutePrestate": "0x03c7ae758795765c6664a5d39bf63841c71ff191e9189522bad8ebff5d4eca98", + "faultGameMaxDepth": 44, + "faultGameClockExtension": 0, + "faultGameMaxClockDuration": 600, + "faultGameGenesisBlock": 0, + "faultGameGenesisOutputRoot": "0x0000000000000000000000000000000000000000000000000000000000000000", + "faultGameSplitDepth": 14, + "faultGameWithdrawalDelay": 604800, + + "preimageOracleMinProposalSize": 1800000, + "preimageOracleChallengePeriod": 86400, + + "fundDevAccounts": false, + "useFaultProofs": false, + "proofMaturityDelaySeconds": 604800, + "disputeGameFinalityDelaySeconds": 302400, + "respectedGameType": 0, + + "usePlasma": false, + "daCommitmentType": "KeccakCommitment", + "daChallengeWindow": 160, + "daResolveWindow": 160, + "daBondSize": 1000000, + "daResolverRefundPercentage": 0 +} diff --git a/op-chain-ops/cmd/celo-migrate/testdata/deployment-l1-dango.json b/op-chain-ops/cmd/celo-migrate/testdata/deployment-l1-dango.json new file mode 100644 index 000000000000..58c1f74772ed --- /dev/null +++ b/op-chain-ops/cmd/celo-migrate/testdata/deployment-l1-dango.json @@ -0,0 +1,37 @@ +{ + "AddressManager": "0x5Ab3E4093B06a69954885caDAF29b280613c674f", + "AnchorStateRegistry": "0x5b530E1DD25513F0Da1CC39333f6fa91143884aB", + "AnchorStateRegistryProxy": "0xCCb982Cc00F9Cc970f71AF753Ff834e9AadADad6", + "CustomGasToken": "0xf7a83872810aBea270FAFF54206A522AB53f3df1", + "CustomGasTokenProxy": "0x3c300204B89F8A333350d9FfE72352D990DDA36f", + "DelayedWETH": "0x9eC20144141099E0c0581ABe0C507aCd2B07Da68", + "DelayedWETHProxy": "0xb0b158f73CF23c2edc220E586Bd043CB1b69c5Ad", + "DisputeGameFactory": "0x88ca8D6b24b691F617C9A02Ce8179F23C13e13Ae", + "DisputeGameFactoryProxy": "0x3889C8BA734148dec7A31472B4A370BC7fc3E50c", + "L1CrossDomainMessenger": "0xDb787d88Fa092d7144053138023137b01b2FB1af", + "L1CrossDomainMessengerProxy": "0x4401780183661b211a9ED386D56b400E430b0995", + "L1ERC721Bridge": "0x442Cc0A770Cd776930a14C0d3363a2931aBE273b", + "L1ERC721BridgeProxy": "0xC262D512A52D4F90D7Bc221979A584857Ab90dd9", + "L1StandardBridge": "0x3D4712d8cA71b8f5ae918213421e098761415898", + "L1StandardBridgeProxy": "0x59f2BDd9674c8C7c5750b12a004783da9a992e5A", + "L2OutputOracle": "0x1dd308d7e3aC77ea70f3c7069b29350665ab57F4", + "L2OutputOracleProxy": "0xD31bf46c090Ea08191fd11BFbf0758fAbeE468cC", + "Mips": "0x30f82aD995f412FBC3386022270AB071e616CaEA", + "OptimismMintableERC20Factory": "0x4626Da66AEE29bC145c7B04614437Aa6540D82F2", + "OptimismMintableERC20FactoryProxy": "0xabcd2bd6a5bc474287Fc460aD99F70d42711AA33", + "OptimismPortal": "0xD10c0833b38b47662200A026d60f0c891ea6C960", + "OptimismPortal2": "0xAAcC881F37e45CAb4C57A6757DE0334FC5b4DE7F", + "OptimismPortalProxy": "0xf550A0aEc6cD8fDFA904242d6e07F71E38BE5240", + "PermissionedDelayedWETHProxy": "0x85E318CE71dE55c19290Fa22149cC773526C8987", + "PreimageOracle": "0x2c010f9f6A63234d03baB647788633Fd5000410A", + "ProtocolVersions": "0x9233fCE30bfb57de432B1a8cf0Cad87A12011eFC", + "ProtocolVersionsProxy": "0x4CE6aDa5fef5BffD2f13257aAF5a40efd412C8E7", + "ProxyAdmin": "0x9f03734bE814De3652C757B3a5702BfE99BA0098", + "SafeProxyFactory": "0xa6B71E26C5e0845f74c812102Ca7114b6a896AB2", + "SafeSingleton": "0xd9Db270c1B5E3Bd161E8c8503c55cEABeE709552", + "SuperchainConfig": "0x05B71600Fe59197339837509F702da31F9fa5cf3", + "SuperchainConfigProxy": "0xDDfcA11fD8553C6Fc1185aD5f492230c3A0091E3", + "SystemConfig": "0x5b10806e8068B517eaBd8DeD5B872034f31878F9", + "SystemConfigProxy": "0xFbe6510A1E209d9E2148d54F134f4E5634bE7241", + "SystemOwnerSafe": "0xD751beb42dBcde9BDe9E2866c0490DccA6A11Ba9" +} \ No newline at end of file diff --git a/op-chain-ops/cmd/celo-migrate/testdata/deployment-l1-holesky.json b/op-chain-ops/cmd/celo-migrate/testdata/deployment-l1-holesky.json new file mode 100644 index 000000000000..b37b79f4d4c8 --- /dev/null +++ b/op-chain-ops/cmd/celo-migrate/testdata/deployment-l1-holesky.json @@ -0,0 +1,34 @@ +{ + "AddressManager": "0x2d256f3b82f673Ee377C393fBF2Cf3DcA5D1D901", + "AnchorStateRegistry": "0x036fDE501893043825356Ce49dfd554809F07597", + "AnchorStateRegistryProxy": "0xe5077701c64782954d27384da76D95ABf320460f", + "DelayedWETH": "0x408Ad04Dd953958B080226025E17d6Ba12987EB7", + "DelayedWETHProxy": "0x27f7Ade64F031A39553Be8104bF8B0b410735845", + "DisputeGameFactory": "0xd7771F9687804Bba1D360B08AD9e4d8CB4523738", + "DisputeGameFactoryProxy": "0x193FdDF22D31c227f1Af1286cf2B051d701FF86E", + "L1CrossDomainMessenger": "0x1e3513a619AA4f2550CDD95709B92C1FE0397184", + "L1CrossDomainMessengerProxy": "0x35841aC1f5FdC5b812562adB17F6A0B9A178F643", + "L1ERC721Bridge": "0x695b01393f0539ec64AC316d4998E4130309efB0", + "L1ERC721BridgeProxy": "0x2b9C1e5b9a0D01256388cc4A0F8F290E839F2d82", + "L1StandardBridge": "0x2d1A818544b657Bc5d1E8c8B80F953bd0CA1C9B2", + "L1StandardBridgeProxy": "0xD10A531CB9b80BD507501F34D87Ad4083E9b7F98", + "L2OutputOracle": "0x04CD14625ff0Da62d6E0820a816b4dD3eCd0FF27", + "L2OutputOracleProxy": "0x5636f9D582DB69EAf1Eb9f05B0738225C91fBC1E", + "Mips": "0x60E1b8b535626Fc9fFCdf6147B45879634645771", + "OptimismMintableERC20Factory": "0x3fcd69a03857aA6e79AE9408fc7c887EE70FC145", + "OptimismMintableERC20FactoryProxy": "0x23c80F2503b93a58EC620D20b6b9B6AB8cCa2a12", + "OptimismPortal": "0xdF803FAC1d84a31Ff5aee841f11659f9a3787CE5", + "OptimismPortal2": "0x60bc423dDf0B24fa5104EcacAC5000674Ac3EBfB", + "OptimismPortalProxy": "0xa292B051eA58e2558243f4A9f74262B1796c9648", + "PreimageOracle": "0xEC19353B7364Fb85b9b0A57EaEEC6aCeBbFb6a53", + "ProtocolVersions": "0x077d61D4fb3378025950Bb60AD69179B38921107", + "ProtocolVersionsProxy": "0x791D5101840A547F1EE91148d34E061412A57ECD", + "ProxyAdmin": "0x4ddC758DA1697Ad58D86D03150872c042390dCa2", + "SafeProxyFactory": "0xa6B71E26C5e0845f74c812102Ca7114b6a896AB2", + "SafeSingleton": "0xd9Db270c1B5E3Bd161E8c8503c55cEABeE709552", + "SuperchainConfig": "0xA4f7dB67A6e098613B107be3F8441475Ec30FCC2", + "SuperchainConfigProxy": "0xB21214DA32a85A0d43372310D62095cf91d67765", + "SystemConfig": "0xeFA98Ba3ada6c6AC4bB84074820685E1F01C835d", + "SystemConfigProxy": "0x733043Aa78d25F6759d9e6Ce2B2897bE6d630E08", + "SystemOwnerSafe": "0xD2a6B91aB77691D6F8688eAFA7a5f188bc5baA3a" +} diff --git a/op-chain-ops/cmd/celo-migrate/testdata/rollup-config-dango.json b/op-chain-ops/cmd/celo-migrate/testdata/rollup-config-dango.json new file mode 100644 index 000000000000..83de52fa1ace --- /dev/null +++ b/op-chain-ops/cmd/celo-migrate/testdata/rollup-config-dango.json @@ -0,0 +1,42 @@ +{ + "genesis": { + "l1": { + "hash": "0xe18e94c26beea64e318e25c32303c9a1ee2bfcee4492337bb5ac14181e99bd0c", + "number": 2358856 + }, + "l2": { + "hash": "0x0e1287f3eba321cb620a032d4d26978424e15b0533e1d19e870baeb1f0078e10", + "number": 25275061 + }, + "l2_time": 1726651200, + "system_config": { + "batcherAddr": "0x1660b1f70de0f32490b50f976e8983213dcf7fd5", + "overhead": "0x0000000000000000000000000000000000000000000000000000000000000000", + "scalar": "0x00000000000000000000000000000000000000000000000000000000000f4240", + "gasLimit": 30000000 + } + }, + "block_time": 2, + "max_sequencer_drift": 600, + "seq_window_size": 3600, + "channel_timeout": 300, + "l1_chain_id": 17000, + "l2_chain_id": 44787, + "regolith_time": 0, + "cel2_time": 0, + "canyon_time": 0, + "delta_time": 0, + "ecotone_time": 0, + "fjord_time": 0, + "batch_inbox_address": "0xff00000000000000000000000000000000044787", + "deposit_contract_address": "0xf550a0aec6cd8fdfa904242d6e07f71e38be5240", + "l1_system_config_address": "0xfbe6510a1e209d9e2148d54f134f4e5634be7241", + "protocol_versions_address": "0x0000000000000000000000000000000000000000", + "alt_da": { + "da_challenge_contract_address": "0x0000000000000000000000000000000000000000", + "da_commitment_type": "GenericCommitment", + "da_challenge_window": 1, + "da_resolve_window": 1 + } +} + diff --git a/op-chain-ops/cmd/celo-migrate/testdata/rollup-config.json b/op-chain-ops/cmd/celo-migrate/testdata/rollup-config.json new file mode 100644 index 000000000000..8dfd1f25e28d --- /dev/null +++ b/op-chain-ops/cmd/celo-migrate/testdata/rollup-config.json @@ -0,0 +1,36 @@ +{ + "genesis": { + "l1": { + "hash": "0xbbed3612407993e67f8ca7a423b181837ae164a531941e78f5ee48e766d39cad", + "number": 1729797 + }, + "l2": { + "hash": "0x2664d0a1f45dc9a010e553e815a25f33c6d949cbb0d38e179c6209fc0486aa41", + "number": 23912613 + }, + "l2_time": 1718312256, + "system_config": { + "batcherAddr": "0x1660b1f70de0f32490b50f976e8983213dcf7fd5", + "overhead": "0x0000000000000000000000000000000000000000000000000000000000000000", + "scalar": "0x00000000000000000000000000000000000000000000000000000000000f4240", + "gasLimit": 30000000 + } + }, + "block_time": 2, + "max_sequencer_drift": 600, + "seq_window_size": 3600, + "channel_timeout": 300, + "l1_chain_id": 17000, + "l2_chain_id": 44787, + "regolith_time": 0, + "cel2_time": 0, + "canyon_time": 0, + "delta_time": 0, + "ecotone_time": 0, + "batch_inbox_address": "0xff00000000000000000000000000000000044787", + "deposit_contract_address": "0xa292b051ea58e2558243f4a9f74262b1796c9648", + "l1_system_config_address": "0x733043aa78d25f6759d9e6ce2b2897be6d630e08", + "protocol_versions_address": "0x0000000000000000000000000000000000000000", + "da_challenge_contract_address": "0x0000000000000000000000000000000000000000" +} + diff --git a/op-chain-ops/cmd/celo-migrate/transform.go b/op-chain-ops/cmd/celo-migrate/transform.go new file mode 100644 index 000000000000..5a80e8a51566 --- /dev/null +++ b/op-chain-ops/cmd/celo-migrate/transform.go @@ -0,0 +1,109 @@ +package main + +import ( + "bytes" + "errors" + "fmt" + "math/big" + + "github.com/ethereum/go-ethereum/common" + "github.com/ethereum/go-ethereum/core/types" + "github.com/ethereum/go-ethereum/crypto" + "github.com/ethereum/go-ethereum/rlp" +) + +var ( + IstanbulExtraVanity = 32 // Fixed number of extra-data bytes reserved for validator vanity +) + +// IstanbulAggregatedSeal is the aggregated seal for Istanbul blocks +type IstanbulAggregatedSeal struct { + // Bitmap is a bitmap having an active bit for each validator that signed this block + Bitmap *big.Int + // Signature is an aggregated BLS signature resulting from signatures by each validator that signed this block + Signature []byte + // Round is the round in which the signature was created. + Round *big.Int +} + +// IstanbulExtra is the extra-data for Istanbul blocks +type IstanbulExtra struct { + // AddedValidators are the validators that have been added in the block + AddedValidators []common.Address + // AddedValidatorsPublicKeys are the BLS public keys for the validators added in the block + AddedValidatorsPublicKeys [][96]byte + // RemovedValidators is a bitmap having an active bit for each removed validator in the block + RemovedValidators *big.Int + // Seal is an ECDSA signature by the proposer + Seal []byte + // AggregatedSeal contains the aggregated BLS signature created via IBFT consensus. + AggregatedSeal IstanbulAggregatedSeal + // ParentAggregatedSeal contains and aggregated BLS signature for the previous block. + ParentAggregatedSeal IstanbulAggregatedSeal +} + +// transformHeader removes the aggregated seal from the header +func transformHeader(header []byte) ([]byte, error) { + newHeader := new(types.Header) + err := rlp.DecodeBytes(header, &newHeader) + if err != nil { + return nil, err + } + + if len(newHeader.Extra) < IstanbulExtraVanity { + return nil, errors.New("invalid istanbul header extra-data") + } + + istanbulExtra := IstanbulExtra{} + err = rlp.DecodeBytes(newHeader.Extra[IstanbulExtraVanity:], &istanbulExtra) + if err != nil { + return nil, err + } + + istanbulExtra.AggregatedSeal = IstanbulAggregatedSeal{} + + payload, err := rlp.EncodeToBytes(&istanbulExtra) + if err != nil { + return nil, err + } + + newHeader.Extra = append(newHeader.Extra[:IstanbulExtraVanity], payload...) + + return rlp.EncodeToBytes(newHeader) +} + +func hasSameHash(newHeader, oldHash []byte) (bool, common.Hash) { + newHash := crypto.Keccak256Hash(newHeader) + return bytes.Equal(oldHash, newHash.Bytes()), newHash +} + +// transformBlockBody migrates the block body from the old format to the new format (works with []byte input output) +func transformBlockBody(oldBodyData []byte) ([]byte, error) { + // decode body into celo-blockchain Body structure + // remove epochSnarkData and randomness data + var celoBody struct { + Transactions types.Transactions + Randomness rlp.RawValue + EpochSnarkData rlp.RawValue + } + if err := rlp.DecodeBytes(oldBodyData, &celoBody); err != nil { + // body may have already been transformed in a previous migration + body := types.Body{} + if err := rlp.DecodeBytes(oldBodyData, &body); err == nil { + return oldBodyData, nil + } + return nil, fmt.Errorf("failed to RLP decode body: %w", err) + } + + // transform into op-geth types.Body structure + newBody := types.Body{ + Transactions: celoBody.Transactions, + Uncles: []*types.Header{}, + } + newBodyData, err := rlp.EncodeToBytes(newBody) + if err != nil { + return nil, fmt.Errorf("failed to RLP encode body: %w", err) + } + + return newBodyData, nil +} diff --git a/op-chain-ops/justfile b/op-chain-ops/justfile new file mode 100644 index 000000000000..b4b6886b4e64 --- /dev/null +++ b/op-chain-ops/justfile @@ -0,0 +1,16 @@ +abis := '../packages/contracts-bedrock/snapshots/abi' + +bindings-celo-migrate: + #!/usr/bin/env bash + set -euxo pipefail + + build_abi() { + local lowercase=$(echo "$2" | awk '{print tolower($0)}') + abigen \ + --abi "{{abis}}/$1.json" \ + --pkg bindings \ + --out "cmd/celo-migrate/bindings/$lowercase.go" \ + --type $2 + } + + build_abi GoldToken CeloToken diff --git a/packages/contracts-bedrock/foundry.toml b/packages/contracts-bedrock/foundry.toml index cef9f85bbaeb..4445f08446d0 100644 --- a/packages/contracts-bedrock/foundry.toml +++ b/packages/contracts-bedrock/foundry.toml @@ -51,6 +51,7 @@ fs_permissions = [ { access='read-write', path='./.testdata/' }, { access='read', path='./kout-deployment' }, { access='read', path='./test/fixtures' }, + { access='read-write', path='../../op-chain-ops/cmd/celo-migrate/testdata/' }, ] libs = ["node_modules", "lib"]