Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Batches and multiprocess #6

Open
wants to merge 21 commits into
base: main
Choose a base branch
from
Open

Batches and multiprocess #6

wants to merge 21 commits into from

Conversation

waredjeb
Copy link

This is a first draft to run the optimizer in batches and use multiprocess to process each batch separately. This setup was needed for start looking at how we can do the same thing but using HTCondor instead of the local machine.

I am opening the PR mostly for suggestion, I leave it as draft
Little changes are needed in the user interface, below an example
The reco_and_validate function would need an additional parameter to write the results on different files for each process.

def reco_and_validate(params, worker_id):
    if not os.path.exists('temp'):
        os.mkdir('temp')
    write_csv('temp/parameters.csv', params)
    validation_result = 'temp/simple_validation_'+str(worker_id)+".root"
    subprocess.run(['cmsRun', config, 'nEvents=' + str(args.num_events),
                     'parametersFile=temp/parameters.csv', 'outputFile=' + validation_result])
    num_particles = len(params)

    with uproot.open(validation_result) as uproot_file:
        population_fitness = np.array(
            [get_metrics(uproot_file, i) for i in range(num_particles)], dtype = float)
    return population_fitness

MOPSO initialization: additional parameter to set the number of batches

    pso = MOPSO(objective_functions=[reco_and_validate],lower_bounds=lb, upper_bounds=ub,
                num_objectives=2, num_particles=args.num_particles, num_iterations=args.num_iterations,
                inertia_weight=0.5, cognitive_coefficient=1., social_coefficient=1., num_batch = args.num_batches,
                max_iter_no_improv=10, optimization_mode='global')

@rsreds
Copy link
Collaborator

rsreds commented Nov 3, 2023

Big ask, but can you try and rebase it on https://github.com/cms-patatrack/The-Optimizer/tree/split-objective?

@waredjeb
Copy link
Author

waredjeb commented Nov 6, 2023

Big ask, but can you try and rebase it on https://github.com/cms-patatrack/The-Optimizer/tree/split-objective?

@rsreds hopefully it's done!

@rsreds rsreds marked this pull request as ready for review November 7, 2023 08:49
@rsreds rsreds marked this pull request as draft November 7, 2023 08:50
@rsreds
Copy link
Collaborator

rsreds commented Nov 7, 2023

Rebase on main, I pulled the changes in the split-objective branch.
tests/schaffer.py must be slightly adjusted.

If you feel that the "default" objective should behave as a batch=1 then BatchObjective is not needed.
However I'd consider separating the two. So as to make the worker_id argument in the objective function required only if using BatchObjective. What do you think?

@waredjeb
Copy link
Author

waredjeb commented Nov 7, 2023

@rsreds Ok, I'll rebase on main, and I agree, we can keep the two separated for now!

@rsreds rsreds mentioned this pull request Nov 7, 2023
Merged
@waredjeb
Copy link
Author

waredjeb commented Nov 7, 2023

@rsreds ok I think I rebased it. I just have to fix BatchObjective and the corresponding tests. But so far, all the tests (adapted) are working. I just need some more testing

@rsreds rsreds linked an issue Nov 14, 2023 that may be closed by this pull request
Batch objectives #10
Open
waredjeb and others added 21 commits December 6, 2023 09:52
@waredjeb
first draft of batches and multiprocessing
8570361
@waredjeb
fix indentation and syntax
364da0f
@waredjeb
working - multiprocess and batch splitting
3f4fbf3
@waredjeb
clean-up
a54dc9c
@waredjeb
buf fix: update particle in batch
4c051a1
@rsreds @waredjeb
Save to file (cms-patatrack#7)
8c0b1ee 
* First implementation of FileManager

* Fix tests for File Manager

* Fix examples

* Changed module name to snake_case

* Format and lint

* Format and lint

* Changed FileManager to static class

* Delegate all IO to FileManager

* Update example to previous commit
@rsreds @waredjeb
Set incremental pareto as an option
bba7358
@rsreds @waredjeb
First implementation of split
c93af38
@waredjeb
first draft of batches and multiprocessing
94326f3
@waredjeb
add worker id to Objective
8dc49e9
@waredjeb
fix save state
09c4e91
@waredjeb
first draft of batches and multiprocessing
3ddceaa
@waredjeb
working - multiprocess and batch splitting
0cdd70b
@waredjeb
clean-up
615d950
@rsreds @waredjeb
Save to file (cms-patatrack#7)
3cfd458 
* First implementation of FileManager

* Fix tests for File Manager

* Fix examples

* Changed module name to snake_case

* Format and lint

* Format and lint

* Changed FileManager to static class

* Delegate all IO to FileManager

* Update example to previous commit
@rsreds @waredjeb
Remove diversity coefficient.
5bc8ae5
@rsreds @waredjeb
Set incremental pareto as an option
a400dbf
@rsreds @waredjeb
First implementation of split
aaf5c3d
@rsreds @waredjeb
Add evaluation function to objectives
c54ecc7
@waredjeb
first draft of batches and multiprocessing
b25cc94
@waredjeb
fix process batch and restore batches
dbfd03a
@waredjeb waredjeb marked this pull request as ready for review December 6, 2023 09:11
@waredjeb
Copy link
Author

waredjeb commented Dec 6, 2023

@rsreds Almost done, I just rebased, going to do some tests to check if everything works fine after the rebase

@rsreds
Copy link
Collaborator

rsreds commented Feb 28, 2024

@waredjeb did you test it? Can we fix the conflicts and merge?

@rsreds rsreds added this to the v0.2 milestone Feb 28, 2024
@rsreds rsreds force-pushed the main branch 2 times, most recently from 698ca99 to 44acf9d Compare April 4, 2024 08:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v0.2
Development

Successfully merging this pull request may close these issues.

Batch objectives
2 participants
@waredjeb @rsreds