Archive Node Online Migration #1863
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| return fmt.Errorf("failed to create iterator: %w", err) | ||
| } | ||
| defer itr.Close() | ||
|
|
||
| startTimeBatch := time.Now() // Measure time for every 10,000 iterations |
Check warning
Code scanning / CodeQL
Calling the system time Warning
cmd/seid/cmd/root.go
Outdated
| go func() { | ||
| homeDir := cast.ToString(appOpts.Get(flags.FlagHome)) | ||
| stateStore := app.GetStateStore() | ||
| latestVersion := rootmulti.GetLatestVersion(db) |
There was a problem hiding this comment.
This latest version would keep changing as we restart right? Is there any concern ?
There was a problem hiding this comment.
Yea this will change to the migrate-height flag
Kbhat1
force-pushed
the
ArchiveNodeMigrationOnline
branch
from
7ba74d6
to
27a1676
Compare
* Add QMS for online migration * Fix lint --------- Co-authored-by: kbhat1 <[email protected]>
| go func() { | ||
| homeDir := cast.ToString(appOpts.Get(flags.FlagHome)) | ||
| stateStore := app.GetStateStore() | ||
| migrationHeight := cast.ToInt64(appOpts.Get("migrate-height")) | ||
| migrator := ss.NewMigrator(homeDir, db, stateStore) | ||
| if err := migrator.Migrate(migrationHeight, homeDir); err != nil { | ||
| panic(err) | ||
| } | ||
| }() |
Check notice
Code scanning / CodeQL
Spawning a Go routine Note
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1863 +/- ##
==========================================
- Coverage 61.33% 61.29% -0.05%
==========================================
Files 261 263 +2
Lines 23152 23334 +182
==========================================
+ Hits 14201 14302 +101
- Misses 7949 8022 +73
- Partials 1002 1010 +8
Flags with carried forward coverage won't be shown. Click here to find out more.
|
| fmt.Println("Scanning database and exporting leaf nodes...") | ||
| fmt.Println("SeiDB Archive Migration: Scanning database and exporting leaf nodes...") | ||
|
|
||
| startTimeTotal := time.Now() // Start measuring total time |
Check warning
Code scanning / CodeQL
Calling the system time Warning
| if module != startModule && startModule != "" { | ||
| continue | ||
| } | ||
| startTimeModule := time.Now() // Measure time for each module |
Check warning
Code scanning / CodeQL
Calling the system time Warning
| }) | ||
|
|
||
| batchLeafNodeCount = 0 | ||
| startTimeBatch = time.Now() |
Check warning
Code scanning / CodeQL
Calling the system time Warning
| The overall process will work as follows: | ||
|
|
||
| 1. Stop archive node and note down its height, call it MIGRATION_HEIGHT | ||
| 2. Update config to enable SeiDB (state committment + state store) |
There was a problem hiding this comment.
This should probably be the last step?
|
|
||
| The overall process will work as follows: | ||
|
|
||
| 1. Stop archive node and note down its height, call it MIGRATION_HEIGHT |
There was a problem hiding this comment.
Actually we don't need this step, we can get the height when we run SC migration
|
|
||
| 1. Stop archive node and note down its height, call it MIGRATION_HEIGHT | ||
| 2. Update config to enable SeiDB (state committment + state store) | ||
| 3. Run sc migration at the MIGRATION_HEIGHT |
There was a problem hiding this comment.
Default it will run at latest height, which we should note it down
| 3. Run sc migration at the MIGRATION_HEIGHT | ||
| 4. Re start seid with `--migrate-iavl` enabled (migrating state store in background) | ||
| 5. Verify migration at various sampled heights once state store is complete | ||
| 6. Stop seid, clear out iavl and restart seid normally, now only using SeiDB fully |
There was a problem hiding this comment.
We should not delete IAVL data here but launch with SeiDB first, and only delete the IAVL data after we successfully run the node for a while
Describe your changes and provide context
Testing performed to validate your change