feat: EDM4HEPSchema and Newstyle FCCSChema #1245

prayagyadav · 2025-01-15T09:35:20Z

Here is a clean draft for the EDM4HEPSchema (edm4hep1) and FCCSchema based on the same. I have not yet managed to add many comments and descriptions, but I plan to add them eventually.

Workings:

The EDM4HEPSchema reads the edm4hep.yaml file from the assets directory. I felt this was necessary to add maximum functionality to the schema. Reading the specifications of all the 'components' and 'datatypes' from the yaml file helps to identify the 'members' (example, energy is a member of edm4hep::ReconstructedParticle datatype) and which members correspond to the various types of cross-branch relations in EDM4HEP: vector members, OneToOneRelations, OneToManyRelations and Links.
The Schema fetches the comments in the edm4hep.yaml file and assigns them as docstrings to the relevant branches.
The EDM4HEPSchema supports all these relations (With Links needing some manual boilerplate code from the user).
The version of the edm4hep.yaml file used is here. Please note that the way Links are represented in EDM4HEP has changed in the latest commit. @tmadlener can comment more on this. In any case, it seems necessary to find a way to track the changes from edm4hep.yaml, so that the COFFEA EDM4HEPSchema does not become obsolete after a few version changes.

Link to example Notebooks:

Tests:

The test sample for EDM4HEP was obtained from here (which, I believe, was pointed out by @tmadlener).
The test sample for the Newstyle (edm4hep1) FCC was provided by @davidlange6
Both the schema pass the basic tests (the tests are similar to the ones that I wrote for the oldstyle (pre-edm4hep1) FCCSchema)

Other comments:

I understand that this commit includes a lot of code, and its purpose may not be immediately clear, but I can provide a detailed explanation for specific snippets if needed.
While I have tried to add most of the functionality to the schema, I cannot figure out how to add the ExtraCode sections mentioned in edm4hep.yaml. They appear to be declarations for C++ methods specific to certain collections.

tests/samples/p8_ee_WW_ecm240_edm4hep.root

lgray · 2025-01-15T15:11:01Z

tests/test_nanoevents_edm4hep.py

+    assert field in delayed_fields
+
+
+def test_MC_daughters(delayed_events):


these tests should also check correctness!

lgray · 2025-01-15T15:11:25Z

tests/test_nanoevents_fcc_edm4hep1.py

+    assert field in delayed_fields
+
+
+def test_MC_daughters(delayed_events):


these tests should also check correctness!

lgray · 2025-01-15T15:12:07Z

tests/test_nanoevents_fcc_edm4hep1.py

+#
+
+
+def test_KaonParent_to_PionDaughters_Loop(eager_events):


ah ok, I guess correctness is checked here in the end.

Perhaps some more basic checks in the prior tests would still be useful?

Yeah, test_KaonParent_to_PionDaughters_Loop attempts to check the correctness of Parents and Daughters relations. Sure, I can add more basic tests

lgray · 2025-01-15T16:36:21Z

can you rebase all the commits into 1 so that that 12MB file isn't into the repo history at all? Thanks!

prayagyadav · 2025-01-15T16:46:44Z

can you rebase all the commits into 1 so that that 12MB file isn't into the repo history at all? Thanks!

Done

src/coffea/nanoevents/transforms.py

for more information, see https://pre-commit.ci

prayagyadav changed the title ~~EDM4HEPSchema and Newstyle FCCSChema~~ feat: EDM4HEPSchema and Newstyle FCCSChema Jan 15, 2025