Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Address memory use in analyzer #6

Open
hschellman opened this issue Dec 13, 2021 · 2 comments
Open

Address memory use in analyzer #6

hschellman opened this issue Dec 13, 2021 · 2 comments
Assignees

Comments

@hschellman
Copy link
Collaborator

Lots of samples and variables leads to a huge memory footprint.

Suggest read the file to map the contents. That just requires the keys. Store the histogram names in a map.

Then get them as needed in the extraction, sample by sample and variable by variable. Drop when no longer used.

ie
std::cout << " just before 2D loop" << std::endl;
for (auto samples:hists2D){
std::string sample=samples.first;
for (auto variables:hists2D[sample]){ // only do this for a subset to save output time.
std::string variable = variables.first;
std::string basename = "h2D_"+sample+"_"+variable;
hists2D[sample][variable] = LoadMe(sample, variable);
int exit = GetCrossSection(sample,variable,basename,hists2D[sample][variable],response2D[sample][variable],config,canvas2D,norm,POTScale,h_flux_dewidthed,unfold,num_iter,DEBUG);
DropMe(sample, variable);
if (DEBUG) std::cout << exit << std::endl;
}
}

Where LoadMe and DropMe do the Get and then Delete on the relevant histograms

@nhvaughan
Copy link
Contributor

Maybe move plotting functions outside of the actual analysis operations. This would start by removing the PlotCVAndError and PlotErrorSummary functions from GetXSec.h. Since we write the hists to file at each step of the analysis, they could be retrieved in a different part of the C++ code (either GetXSec.h or analyze), or accessed separately using a python script.

@hschellman
Copy link
Collaborator Author

addressed partially with analyze_v8 which takes a sample name as a 3rd argument

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants