Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restructure graph and ingestor #101

Merged
merged 5 commits into from
Mar 20, 2024

Conversation

ctron
Copy link
Contributor

@ctron ctron commented Mar 19, 2024

This restructures the graph and ingestor modules to follow the idea of modules/importer. Basically this means:

  • all functionality of the each module is confined to that module, including: their endpoints, model objects, services, server-loops
  • those modules have APIs which can be used by other modules (or components)
  • the current server crate becomes an empty wrapper, mounting endpoints from other modules and starting all server loops required.

The next step could be to create dedicated sub-commands in the trustd binary to only start certain modules (vs the one super-server right now).

I am sure there are other ways to do this. This is just one way that seems reasonable for the goal of preventing components which seem to belong together to be ripped apart into different locations of the project. And yes, sometimes the lines are a bit blurry.

@ctron
Copy link
Contributor Author

ctron commented Mar 19, 2024

And it looks like this needs to be rebased. But I'll only do this if we believe this is the correct layout.

@bobmcwhirter
Copy link
Contributor

Depending on your friskiness when rebasing, sorting out ingestors and importer to a single one or the other would be useful. Or fine in a further PR.

@ctron
Copy link
Contributor Author

ctron commented Mar 20, 2024

Depending on your friskiness when rebasing, sorting out ingestors and importer to a single one or the other would be useful. Or fine in a further PR.

If the general direction is ok, we can do that in a follow up PR.

I would expect that ingestors would be more aligned with the graph stuff. importers is basically the management and execution of the import process. ingestors (to my understanding) is the process of adding new stuff to the graph. Then again, I am not too deep into that part. So I am only assuming.

Assuming (even more) that we need some "store to disk" feature for the actual content (which the graph stuff would not do I presume). Maybe it makes sense to keep the ingestor stuff, add some storage and keep it as dedicated module. graph + storage => ingestor?

@ctron ctron added this pull request to the merge queue Mar 20, 2024
Merged via the queue into trustification:main with commit 81e6117 Mar 20, 2024
1 check passed
@ctron ctron deleted the feature/restructure_1 branch March 20, 2024 12:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants