Trainer.hyperparameter_search kwargs parameter has an inexact definition if using Optuna

[GuillemGSubies]

According to the docs, those kwargs will only be used in the create_study method in Optuna.

However that is not exactly true, as we can see in run_hp_search_optuna. Some parameters from those kwargs will end up being used in the optimize method (which is great because it is what I wanted to do but the docs said I could not do it).

transformers/src/transformers/integrations/integration_utils.py

Line 256 in b54109c

n_jobs = kwargs.pop("n_jobs", 1)

transformers/src/transformers/integrations/integration_utils.py

Line 261 in b54109c

study.optimize(_objective, n_trials=n_trials, timeout=timeout, n_jobs=n_jobs, gc_after_trial=gc_after_trial)

I didn't check ray or sigopt but I guess that something similar can be happening.

I don't really know what would be a good solution given that not every parameter in optimize is used and creating a kwargs_optimize would not make sense given that you support 2 other backends. Maybe a simple list of accepted paramters in the docstring?

If you tell me what solution would be the best, I could send a PR.

@stevhliu

[stevhliu]

Hey, thanks for pointing out the discrepancy! I think your proposed solution of a list of accepted parameters in the docstring would be the simplest fix 🙂