Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

eth: start_block recovery after crashing during sync #22

Open
gregfromstl opened this issue Feb 26, 2024 · 3 comments
Open

eth: start_block recovery after crashing during sync #22

gregfromstl opened this issue Feb 26, 2024 · 3 comments
Assignees

Comments

@gregfromstl
Copy link
Contributor

Right now, we're fetching the last indexed block by finding the max block number in chain_events. Because we insert log types sequentially, there's a chance the program crashes after inserting some log types for a stretch of blocks but not all, leaving the other log types absent when the sync restarts and fetches the latest block number.

The easiest way to solve this is probably by getting the max block number in chain_events for each event type. However, this will make the log collection messier (we'd be fetching logs for different ranges for each event type, since we don't want any duplicate events), so I'm not 100% convinced this is the best approach.

@haardikk21
Copy link
Contributor

what if we:

  1. change all INSERT queries to be upsert (INSERT ... ON CONFLICT ...) instead
  2. and, fetch start_block as the minimum value of latest block of all event types

@gregfromstl
Copy link
Contributor Author

I think that would work, we would just have to make the transaction_hash column unique in chain_events to prevent duplicates there

@haardikk21
Copy link
Contributor

i can take this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants