Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Describe congestion fundamentals #10873

Merged
merged 6 commits into from
Mar 28, 2024
Merged

Conversation

jakmeier
Copy link
Contributor

This document describes a few fundamental congestion control problems and ideas to solve them.

The added page serves as secondary document to NEP to summarise the thought process behind the most important design decisions. But it is generally applicable to congestion in Near Protocol's receipt execution system as it works today. It can even serve as documentation for how congestion can occur today.

The document includes 8 graphs generated using graphviz.
To regenerate after modifying the *.dot files, install the graphviz toolbox (on systems with apt: sudo apt install graphviz) and then run dot -Tsvg img_name.dot > img_name.svg.

Copy link

codecov bot commented Mar 25, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.52%. Comparing base (589ff5f) to head (690059d).

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #10873      +/-   ##
==========================================
- Coverage   71.53%   71.52%   -0.01%     
==========================================
  Files         759      759              
  Lines      151627   151627              
  Branches   151627   151627              
==========================================
- Hits       108467   108456      -11     
- Misses      38671    38678       +7     
- Partials     4489     4493       +4     
Flag Coverage Δ
backward-compatibility 0.24% <ø> (ø)
db-migration 0.24% <ø> (ø)
genesis-check 1.43% <ø> (ø)
integration-tests 37.14% <ø> (-0.04%) ⬇️
linux 70.01% <ø> (-0.01%) ⬇️
linux-nightly 71.03% <ø> (+0.03%) ⬆️
macos 54.38% <ø> (-0.06%) ⬇️
pytests 1.66% <ø> (ø)
sanity-checks 1.44% <ø> (ø)
unittests 67.16% <ø> (+<0.01%) ⬆️
upgradability 0.29% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@wacban wacban left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic explanation and I love the idea to present it as flow graphs!

docs/architecture/how/receipt-congestion.md Outdated Show resolved Hide resolved
But this is not ideal Just consider this example where everybody tries to access
a contract on the same shard.

![graph](../../images/congestion/receipt_flow_example_1.svg)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depending on how it's interpreted the graph may also need a 999 Tgas arrow from shard 3 to itself and then the arrow to sink should be 2997 Tgas.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for the graph below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree for the arrow shard3 -> shard3. But I think it should still be 2998 Tgas. Otherwise the sum of flows from the source is not equal to the sum to the sinks. It would be 2997 Tgas for the receipts and 1 Tgas for conversion, both should point to sink, showing that the gas has been burnt.

I'm adding the arrows to the two graphs as best as I can without it getting too unreadable... Aligning things isn't always easy in graphviz graphs^^

Comment on lines 77 to 79
The sad thing is, there is no way around it in the most general case. A
congestion control strategy that does apply this limit to this workload will
always have infinitely sized queues.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's the other way around, a strategy that accepts more than this limit will have unbounded queues.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right^^ I think a "not" went missing somewhere, I'll add one to correct it.

docs/architecture/how/receipt-congestion.md Outdated Show resolved Hide resolved
docs/architecture/how/receipt-congestion.md Outdated Show resolved Hide resolved
docs/architecture/how/receipt-congestion.md Outdated Show resolved Hide resolved
@jakmeier jakmeier enabled auto-merge March 28, 2024 15:06
@jakmeier jakmeier added this pull request to the merge queue Mar 28, 2024
Merged via the queue into near:master with commit 44c3921 Mar 28, 2024
33 checks passed
@jakmeier jakmeier deleted the congestion-docs branch March 28, 2024 15:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants