Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use database of partial paths to speed up bindings resolution #1198

Closed

Conversation

ggiraldez
Copy link
Contributor

@ggiraldez ggiraldez commented Dec 19, 2024

Builds on top of #1195

  • Resolve all references at once using a database of minimal partial paths. This speeds up resolution considerably (since it avoids a lot of duplicated work) at the expense of higher memory consumption.
  • Change Definition and Reference to hold a Rc<> to the BindingGraph as opposed to a normal reference. This should allow easier integration with WASM since there are already wrappers for ref counted objects.
  • Split the implementation of BindingGraph and splint off a BindingGraphBuilder in which to add user files and built-ins and then call resolve() which will consume the builder and return a leaner BindingGraph with all bindings resolved.
  • The changes here require using our fork of stack-graphs which adds the ability to rewind the arena allocator used for partial paths after resolving each reference, but still allows using the default database to hold the set of minimal partial paths.

This results in some references being resolved to many ambiguous definitions,
some of which we were able to resolve via ranking.
Most remaining assertion tests were redundant as there were already snapshots
that cover those cases.
This also removes the ranking algorithm for resolution results, since it's no
longer needed.
…attributes

This *should* make it easier to construct a partial paths databases in which
these nodes are endpoints.
@ggiraldez ggiraldez requested review from a team as code owners December 19, 2024 21:36
Copy link

changeset-bot bot commented Dec 19, 2024

⚠️ No Changeset found

Latest commit: 1b36243

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@ggiraldez ggiraldez force-pushed the hooks-database-stitching branch 2 times, most recently from 5d3a2e6 to 156722b Compare December 19, 2024 23:10
scripts/_common.sh Outdated Show resolved Hide resolved
Cargo.toml Outdated Show resolved Hide resolved
Cargo.toml Show resolved Hide resolved
Cargo.toml Show resolved Hide resolved
This uses the added `save_checkpoint`/`restore_checkpoint` which rewind the
allocation pointer in the `PartialPaths` arenas. For this to work properly, we
also first `ensure_both_directions` in the database so that after that it
doesn't need further mutation.
The database resolver will resolve all references at once by using a database of
minimal partial paths.
This makes it impossible to try to access definitions/references before
resolving, and allows dropping the entire stack graph and database of partial
paths used for resolution after they are no longer necessary.
@ggiraldez ggiraldez force-pushed the hooks-database-stitching branch from 15f35da to 8c55602 Compare December 23, 2024 21:16
@ggiraldez
Copy link
Contributor Author

After #1195 is merged, I'll rebase this PR and the conflicts should be resolved.

…records

The `definitions` test name is now a bit misleading since no definitions are
retrieved there, but it's still where user source files are ingested.
@OmarTawfik
Copy link
Contributor

Replaced by #1204

@OmarTawfik OmarTawfik closed this Dec 27, 2024
github-merge-queue bot pushed a commit that referenced this pull request Jan 2, 2025
Rebasing #1198 on latest after #1195 was merged.

---------

Co-authored-by: Gustavo Giráldez <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants