Improving robustness of fit_gpytorch_mll #2367
-
|
I've had some inconsistent and strange results for hyperparameter fitting on the same datasets, not particularly large ones but with a custom kernel. Are there any convenience methods for multi starting this, or is there a more robust optimizer I can be using? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
Hi @George-Lu-1997. I am adding an option for fitting the model multiple times and picking the best in #2373. This may help a bit, at least in the sense of getting model with a larger MLL. In general, since the MLL is non-convex, there may be multiple near optimal model fits that look quite different from each other. With a single attempt, we're more likely to converge to a local optimum, so multiple attempts should increase consistency. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you! |
Beta Was this translation helpful? Give feedback.
Hi @George-Lu-1997. I am adding an option for fitting the model multiple times and picking the best in #2373. This may help a bit, at least in the sense of getting model with a larger MLL. In general, since the MLL is non-convex, there may be multiple near optimal model fits that look quite different from each other. With a single attempt, we're more likely to converge to a local optimum, so multiple attempts should increase consistency.