Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store information on used compute resources per module #1099

Open
3 tasks
makortel opened this issue Nov 27, 2024 · 0 comments
Open
3 tasks

Store information on used compute resources per module #1099

makortel opened this issue Nov 27, 2024 · 0 comments

Comments

@makortel
Copy link
Collaborator

makortel commented Nov 27, 2024

Spinning off from #17

Use cases (so far)

  • Alpaka modules for which the backend is overridden from the process-wide default
    • Presently each Alpaka producer produces an integer into the Event that indicates what backend was used
  • SONIC modules that perform the actual computations on a remote node

Things brainstormed June 29, 2023

  • Check how the existence of Transformer looks in the event-level provenance
  • How does ProductProvenance map to data products? Is it per data product or per module?
    • Seems like it is saved per BranchID. So we could make Transformer to add information there.
  • One alternative way would be to add "not identical but equivalent" concept for PSets. Keep the "identity" PSet in the PSetRegistry for Runs and Lumis, and add "equivalence" PSet for per-event information (replace or add additional ProcessHistoryID to EventAuxiliary). Would have to add a concept between "tracked" and "untracked" parameters that would be stored only in the "equivalence PSet" (but do not expose this functionality publicly)

Also the comment #17 (comment) is relevant

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 📋 Backlog
Development

No branches or pull requests

1 participant