testTauEmbeddingProducers had ERRORS

[silviodonato]

We are getting an error in the unit test TauAnalysis/MCEmbeddingTools in CMSSW_11_3_X_2020-12-02-2300
https://cmssdt.cern.ch/SDT/cgi-bin/logreader/slc7_amd64_gcc900/CMSSW_11_3_X_2020-12-02-2300/unitTestLogs/TauAnalysis/MCEmbeddingTools#/2001-2001

----- Begin Fatal Exception 03-Dec-2020 04:18:24 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 274199 lumi: 125 event: 245779397 stream: 0
   [1] Running path 'MINIAODSIMoutput_step'
   [2] Prefetching for module PoolOutputModule/'MINIAODSIMoutput'
   [3] Prefetching for module PATElectronSlimmer/'slimmedLowPtElectrons'
   [4] Prefetching for module PATElectronSelector/'selectedPatLowPtElectrons'
   [5] Prefetching for module PATElectronProducer/'patLowPtElectrons'
   [6] Prefetching for module GsfElectronProducer/'lowPtGsfElectrons'
   [7] Prefetching for module LowPtGsfElectronCoreProducer/'lowPtGsfElectronCores'
   [8] Prefetching for module LowPtGsfElectronSCProducer/'lowPtGsfElectronSuperClusters'
   [9] Prefetching for module PFElecTkProducer/'lowPtGsfElePfGsfTracks'
   [10] Prefetching for module GsfTrackProducer/'lowPtGsfEleGsfTracks'
   [11] Prefetching for module CkfTrackCandidateMaker/'lowPtGsfEleCkfTrackCandidates'
   [12] Calling method for module LowPtGsfElectronSeedProducer/'lowPtGsfElectronSeeds'
Exception Message:
RefCore: A request to resolve a reference to a product of type 'TrajectorySeed' with ProductID '5:571'
can not be satisfied because the product cannot be found.
Probably the branch containing the product is not stored in the input file.
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

It seems related to #31220

[silviodonato]

assign reconstruction, xpog

[cmsbuild]

New categories assigned: xpog,reconstruction

@slava77,@fgolf,@mariadalfonso,@perrotta,@jpata,@gouskos you have been requested to review this Pull request/Issue and eventually sign? Thanks

[cmsbuild]

A new Issue was created by @silviodonato Silvio Donato.

@Dr15Jones, @dpiparo, @silviodonato, @smuzaffar, @makortel, @qliphy can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[perrotta]

It seems related to #31220

@bainbrid

[silviodonato]

The error is originated from the last step of https://github.com/cms-sw/cmssw/blob/master/TauAnalysis/MCEmbeddingTools/test/runtests.sh#L39

cmsDriver.py MERGE -s PAT --filein file:simulated_and_cleaned.root  --fileout file:merged.root --era Run2_2016_HIPM --data --scenario pp --conditions auto:run2_data --eventcontent  MINIAODSIM --datatier USER --customise TauAnalysis/MCEmbeddingTools/customisers.customiseMerging --customise_commands "process.patTrigger.processName = cms.string('SIMembedding')" -n -1  || die 'Failure during merging step' $?

[bainbrid]

Hi all,

I took a look at this.

I am able to reproduce the Exception with local tests.

When inspecting the command here, I realised through edmConfigDump that this PAT step actually also includes large parts of the RECO (somehow), including the full lowPtElectron RECO sequence and the lowPtElectronSeeds module that gives the exception.

With @slava77 's help, I've tried to decipher the commands and the --customise TauAnalysis/MCEmbeddingTools/customisers.customiseMerging argument, which seems to be responsible for the scheduling of RECO modules.

I've attempted to suppress the execution of lowPtElectron modules by adding here the following two lines:

process.merge_step.remove(process.lowPtGsfElectronTask)
process.merge_step.remove(process.gsfTracksOpenConversions)

I think this was successful. However, I then saw errors regarding PAT-related lowPtElectron steps, such as the lowPtGsfLinks module. I added further .remove lines such as:

process.merge_step.remove(process.lowPtGsfLinks)
process.merge_step.remove(process.slimmedLowPtElectrons)

but this was unsuccessful, I suspect because I see several copies of RECO modules in the various intermediate files and it seems that RECO (or parts of it) are run more than once.

The relevant PAT modules are
https://github.com/cms-sw/cmssw/blob/master/PhysicsTools/PatAlgos/python/producersLayer1/lowPtElectronProducer_cff.py
https://github.com/cms-sw/cmssw/blob/master/PhysicsTools/PatAlgos/python/selectionLayer1/lowPtElectronSelector_cfi.py
https://github.com/cms-sw/cmssw/blob/master/PhysicsTools/PatAlgos/python/slimming/slimmedLowPtElectrons_cff.py

Apologies, but I cannot decipher the details of what the script is trying to achieve.

[silviodonato]

see #32400

[bainbrid]

Ah yes, I missed this line. Thanks.

[perrotta]

+1

• Fixed by Fix in TauAnalysis/MCEmbeddingTools/python/customisers.py (lowPtGsfElectronTask) #32400

[mariadalfonso]

+xpog

[cmsbuild]

This issue is fully signed and ready to be closed.