Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: initial multi-objective support #180

Merged
merged 1 commit into from
Jan 24, 2025
Merged

feat: initial multi-objective support #180

merged 1 commit into from
Jan 24, 2025

Conversation

eddiebergman
Copy link
Contributor

@eddiebergman eddiebergman commented Jan 24, 2025

Adds multi-objective support to the NePS backend, meaning that users can now return multiple objectives and the runtime can handle it properly. NOTE No optimizers other than random or grid-search can actually handle multi-objective at the moment, this is only such that the infrastructure of NePS can handle it.

To return multiple objectives:

def evaluate_pipeline(...):
    return [y1, y2]

def evaluate_pipeline(...):
    return { "objective_to_minimize": [y1, y2], ...}

Since we never concretely define what a user can return, there is now a very dedicated parsing function which is pretty exhaustive about it and comes with a lot of tests. UserResult.parse(whatever_the_user_returns_from_evaluate_pipeline)


I also had to go through quite a bit of the summary related things to account for this. I also discovered CLI and plotting stuff was broken from before, and consequently, is also still broken during this change. This is a further thing that needs to be done.

We should consider if we really want to provide plotting code. It's usually quite custom per use-case, what's better is to provide examples on how to do it, but also just a way to export dataframes easily which people can interpret and plot as they like. This is now done by full_frame, short_summary = neps.status(root_directory), where full_frame contains all the information relating to trials, while short summary is more for printing purposes and is just a pd.Series.

@eddiebergman eddiebergman merged commit ff08feb into master Jan 24, 2025
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

1 participant