Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Background node fetching for storage writes #9138

Closed
2 tasks
Longarithm opened this issue Jun 1, 2023 · 2 comments
Closed
2 tasks

Background node fetching for storage writes #9138

Longarithm opened this issue Jun 1, 2023 · 2 comments
Assignees
Labels
A-storage Area: storage and databases C-tracking-issue Category: a tracking issue

Comments

@Longarithm
Copy link
Member

Longarithm commented Jun 1, 2023

With flat storage released, we don't necessarily have to read nodes to get an old value during storage_write execution. We can reuse prefetcher functionality to schedule background node reads, so that contract won't be blocked on IO latency, and in the end of chunk processing, when it's time to create new nodes out of old ones, most of old nodes are already fetched.

At first, we implement simple prototype to check feasibility. Perhaps we need to have a separate PrefetchApi for now.

Idea originates from here.

Dedicated GDrive with docs and progress: https://drive.google.com/drive/folders/1hysZYMZfsfGdfk1WUJgUj7R2MpLASGfC

2023 July milestones

  • Implementation done by 30% (TODO - define what it means)
  • Align on cost model
@Longarithm Longarithm added the A-storage Area: storage and databases label Jun 1, 2023
@Longarithm Longarithm self-assigned this Jun 1, 2023
@Longarithm Longarithm changed the title Prototype for background trie reads during contract calls Background node fetching for storage writes Jul 10, 2023
@Longarithm Longarithm added the C-tracking-issue Category: a tracking issue label Jul 10, 2023
@Longarithm
Copy link
Member Author

Longarithm commented Jul 26, 2023

July results:

  • Implementation - wasn't started;
  • Align on cost model - our hypothesis on existence of suitable gas model failed.

We spent 4 weeks in July on it. Most of the time was spent on carefully aggregating receipts data and understanding the impact of gas model change (doc). We had one conversation about how we can communicate receipt failures/cost increases.

However, later I realized that we don't have enough evidence for this gas model proposal. We could propose higher costs, but it would dramatically increase number of failing receipts. After thet, we tried to understand how we could proceed with background fetching without significant cost changes and formulated a new proposal (another doc).

Ideas for August:

  • Align on implementation without gas cost changes
  • Prove with benchmarking that it solves cause of July mainnet incident

@pugachAG pugachAG assigned pugachAG and Longarithm and unassigned Longarithm and pugachAG Aug 2, 2023
@Longarithm
Copy link
Member Author

Won't needed because of near/near-one-project-tracking#5

@Longarithm Longarithm closed this as not planned Won't fix, can't repro, duplicate, stale Feb 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-storage Area: storage and databases C-tracking-issue Category: a tracking issue
Projects
None yet
Development

No branches or pull requests

2 participants