-
Notifications
You must be signed in to change notification settings - Fork 4.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MaxMemoryPreload monitoring shows noticeable increase when large number of packages are rebuilt #46359
Comments
assign core |
New categories assigned: core @Dr15Jones,@makortel,@smuzaffar you have been requested to review this Pull request/Issue and eventually sign? Thanks |
cms-bot internal usage |
A new Issue was created by @makortel. @Dr15Jones, @antoniovilela, @makortel, @mandrenguyen, @rappoccio, @sextonkennedy, @smuzaffar can you please review it and eventually sign/assign? Thanks. cms-bot commands are listed here |
Here is an example from #46204 (comment) |
With @Dr15Jones we hypothesized maybe the |
Another reason could be the PluginManager cache would also include reading from the workspace's lib directory and therefore increase its working memory. |
Another example from #46388 (comment) |
Interestingly #46241 (comment), that was a test with |
I was able to reproduce locally a "rebuild of most of the universe in a developer area" leading to ~30 MB higher |
I ran IgProf for both cases, below are the MEM_LIVE after 10 events
and the cause is in cmssw/FWCore/PluginManager/src/CacheParser.cc Lines 120 to 122 in b96fd02
|
So the cost is that the developer area's In my "rebuild" developer area the |
I wonder if there would be any easy way to exclude the contribution from |
Instrumenting some of the PluginManager code, I see (in my rebuild area) that the job reads
So with nearly all of CMSSW build, most of the plugins have 3 entries in the |
#46542 proposes to exclude the PluginManager cache from the MaxMemoryPreload monitoring as a workaround. |
Observed in #46200 (comment).
The text was updated successfully, but these errors were encountered: