Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(rfc): operation cache warmer #1115

Closed
wants to merge 19 commits into from

Conversation

StarpTech
Copy link
Contributor

Motivation and Context

TODO

The User can push individual operations to the distributed operation cache by using the CLI:

```bash
wgc router cache add -g mygraph operations.json
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be under the namespace "router"?
At which level do we apply cache warming?
Org? Namespace? Graph? Router?

As an alternative, you could consider creating a "cache-warming-bucket", which you can then use to add operations, and which can be used in the router config, e.g.:

version: "1"

cache_warmup:
  enabled: true
  interval: 5m
  bucket: myBucket

I'm not sure though if this is the right direction.
The cache warming bucket is somewhat closely related to a Graph because the containing operations need to be compatible with the client schema of the graph.

]
```

The cli command is idempotent and always updates the cache with the latest operations. This doesn't trigger the computation of the Top-N operations which is done periodically by the Cosmo Platform.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another aspect to think about is pre-compiling operation plans.
We could make plans serializable, generate and serialize them at composition time.
Once the router wants to "load" them, it doesn't have to plan everything but can load the plan much faster.

@StarpTech StarpTech changed the title chore(rfc): distributed operation cache chore(rfc): operation cache warmer Aug 25, 2024
Copy link

@vasylruban-nw vasylruban-nw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much for drafting this proposal, it looks great, can't wait to see it live!

Left a few questions I had during reading.

rfc/operation-cache-warmer.md Show resolved Hide resolved
rfc/operation-cache-warmer.md Show resolved Hide resolved
rfc/operation-cache-warmer.md Show resolved Hide resolved
Copy link

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added the Stale label Sep 25, 2024
Copy link

github-actions bot commented Oct 9, 2024

Closed as inactive. Feel free to reopen if this PR is still being worked on.

@github-actions github-actions bot closed this Oct 9, 2024
@StarpTech StarpTech removed the Stale label Oct 9, 2024
@StarpTech StarpTech reopened this Oct 9, 2024
Copy link

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added the Stale label Oct 24, 2024
@StarpTech StarpTech removed the Stale label Oct 24, 2024
@github-actions github-actions bot added the Stale label Nov 8, 2024
@StarpTech StarpTech removed the Stale label Nov 8, 2024
@github-actions github-actions bot added the Stale label Nov 23, 2024
@StarpTech StarpTech removed the Stale label Nov 23, 2024
@github-actions github-actions bot added the Stale label Dec 8, 2024
@StarpTech StarpTech removed the Stale label Dec 8, 2024
@github-actions github-actions bot added the Stale label Dec 23, 2024
@StarpTech StarpTech removed the Stale label Dec 23, 2024
@wundergraph wundergraph deleted a comment from github-actions bot Dec 23, 2024
@wundergraph wundergraph deleted a comment from github-actions bot Dec 23, 2024
@wundergraph wundergraph deleted a comment from github-actions bot Dec 23, 2024
@wundergraph wundergraph deleted a comment from github-actions bot Dec 27, 2024
Copy link

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added the Stale label Jan 11, 2025
Copy link

Closed as inactive. Feel free to reopen if this PR is still being worked on.

@github-actions github-actions bot closed this Jan 26, 2025
@StarpTech
Copy link
Contributor Author

StarpTech commented Jan 26, 2025

The feature has been implemented and released.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants