Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft implementation for cross-db query plan execution. #551

Draft
wants to merge 11 commits into
base: main
Choose a base branch
from

Conversation

obi1kenobi
Copy link
Contributor

Not ready to merge. Please look at it for big-picture design issues and not small nits.

There's a bunch of debugging code currently in there as well, since it has no unit tests and we have no cross-db integration tests, and I've been testing it by hand instead.


full_query_args = dict(query_args)

plan_and_depth = _get_plan_and_depth_in_dfs_order(query_plan_descriptor.root_sub_query_plan)
Copy link
Collaborator

@bojanserafimov bojanserafimov Sep 10, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the code below depend on the order being a dfs order? In some very trivial cases we will have to start execution from the leafs. Just making sure we don't lock ourselves out from that execution plan with the code structure.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing we should document better than we currently have is the distinction between the output of split_query() and the output of make_query_plan() -- I think that would clarify the situation here. split_query() tells us where we cross schemas, but does not specify an execution order. make_query_plan() specifies an order between the subqueries, and is free to specify any valid order -- and may choose an optimized order if it has access to statistics etc.

Given that, once a query plan is made, it's always executed from the root onward. We may in the future allow different styles of executors (e.g. DFS, BFS, async + parallel across children, etc.), but this executor function is just a simple sync DFS.

@obi1kenobi obi1kenobi changed the base branch from master to main June 24, 2020 14:01
@codecov
Copy link

codecov bot commented Oct 26, 2020

Codecov Report

Merging #551 (3551b46) into main (33cc5e8) will decrease coverage by 0.82%.
The diff coverage is 20.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #551      +/-   ##
==========================================
- Coverage   94.36%   93.54%   -0.83%     
==========================================
  Files         113      113              
  Lines        9065     9168     +103     
==========================================
+ Hits         8554     8576      +22     
- Misses        511      592      +81     
Impacted Files Coverage Δ
...aphql_compiler/schema_transformation/query_plan.py 51.42% <17.50%> (ø)
graphql_compiler/schema_transformation/utils.py 88.09% <28.57%> (-7.17%) ⬇️
...phql_compiler/schema_transformation/split_query.py 97.15% <100.00%> (ø)
graphql_compiler/deserialization.py 95.00% <0.00%> (-2.30%) ⬇️
graphql_compiler/schema/__init__.py 97.75% <0.00%> (-2.25%) ⬇️
...hql_compiler/query_formatting/cypher_formatting.py 37.28% <0.00%> (-1.05%) ⬇️
graphql_compiler/query_formatting/common.py 97.84% <0.00%> (-0.03%) ⬇️
...ql_compiler/schema_transformation/rename_schema.py 99.64% <0.00%> (-0.01%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 33cc5e8...3551b46. Read the comment docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants