Integrate RNTuple

[makortel]

Successor of #103.

Miscellaneous

• Don't throw on ROOT::Experimental RNtuple warning #116

NanoAOD output module

• Backfill support through RNTuple late model extension mechanism #612
• Update ROOT master version of NanoAOD output module for the improved framework-facing RNTuple API (after the ROOT-side update)
• Complete support for NanoAOD auxiliary trees
• Implementation of parallel RNTuple writing based on event clusters
• Unit tests and relval workflow
• Documentation

Deprecation of edm::OwnVector

• Open an issue in CMSSW GitHub #633
• Mark edm::OwnVector as CMS_DEPRECATED #634
  • Experiment a bit what kind of placement for the custom attribute makes most sense (whole class, constructor, something else?)
  • Ask Shahzad to revive CMSDEPRECATED IB? Maybe consider enabling some -Wdeprecated* there as well?
• Collect all uses of edm::OwnVector in the issue #599
• Remove unnecessary uses of OwnVector #600
• Remove or replace OwnVector<SiStripMatchedRecHit2D> #827
• Migrate OwnVector<BaseTrackerRecHit> to vector<unique_ptr<BaseTrackerRecHit>> #665
• Migrate OwnVector<TrackingRegion> to vector<unique_ptr<TrackingRegion>> #835
• Migrate many cases of edm::OwnVector to std::vector #666
  • This will result in data format backwards incompatibilities. Need to understand when we can deploy. E.g. do we need to merge first in DEVEL IB, and when 14_0_X opens we merge all those from DEVEL to master?
• Develop a new version of OwnVector that internally uses std::vector<std::variant<...>>
  • Need to figure out a good name
• Migrate all genuine uses of edm::OwnVector to the variant-based new `OwnVector
  • List in edm::OwnVector is deprecated cmssw#42734 (comment)
  • Also here need to think of the deployment strategy
  • Need ROOT to support vector<variant<...>> in TTree first

Deprecation of edm::RefToBase

• Collect uses of edm::RefToBase and open an issue #667
• Mark edm::RefToBase as CMS_DEPRECATED
• Study schema evolution options for RefToBase -> Ptr #674
• Remove unused definitions of edm::RefToBase
• Remove unused class template instantiations
• Make 3 class templates to produce PtrVector in addition of RefToBaseVector
• Migrate edm::RefToBasereco::CaloCluster to edm::Ptr
• Migrate edm::RefToBasereco::Muon to edm::Ptr
• Migrate edm::RefToBasereco::Track to edm::Ptr
• Migrate edm::RefToBasereco::Jet to edm::Ptr
• Migrate edm::RefToBasereco::Candidate to edm::Ptr
• Migrate edm::RefToBase to edm::Ptr
• Remove production of RefToBaseVector from 4 class templates
• Remove the uses of RefToBase of reco::Photon, reco::Electron, and reco::GsfElectron

Data model planning for RNTuple

• Craft slides on the problem for core software meeting 2023-09-12 #669
• Craft slides of the data model evolution for XPOG meeting 2023-10-18 #705
• Craft slides of the data model evolution for Core Software meeting 2023-10-24 #706
• Craft slides of the data model evolution for O&C weekly meeting 2023-11-01 #707
• Craft slides for CMS feedback for RNTuple workshop 2023-11-06..07 #708

Data model exploration for RNTuple

• Attempt evolution of pat::Electron with one change deep in the inheritance hierarchy #723
• Test evolution of std::multimap to std::vector<std::pair<...>> #724
• Changed to constexpr in reco::Vertex #798

General output module

• Deal with the difference in the branch name (https://github.com/cms-sw/cmssw/blob/6636ecebfbf89c6ddff93c31f6c394b125d0c0ae/DataFormats/Provenance/src/BranchDescription.cc#L141-L148)
  • TTree needs a trailing dot ([Tutorial] How to create ROOT Dictionary for a custom class is a stand-alone C++ project root-project/root#13205 (comment))
  • RNTuple forbids (but gives an equivalent behavior out of the box)

First OutputModule and Source prototypes

• Add OutputModule test for TestProcessor #948
• Add TestSourceProcessor helper #950
• Tell RNTuple not to split some classes #958
• Update classes_def.xml files to new RNTuple version #1076