Need help for using EHVI acquisition function

[An-xu]

Hi, may I ask how to use the EHVI acquisition function? I am really confused by the documents and the code I wrote has some errors.
After I fit the model, the code I wrote is:
ref_point = torch.tensor([10, 10])
EHVI = ExpectedHypervolumeImprovement(model=model, ref_point=ref_point, partitioning=NondominatedPartitioning(ref_point=ref_point) )

The error is: pareto_Y has not been initialized
ExpectedHypervolumeImprovement.init(self, model, ref_point, partitioning, objective)
125 if len(ref_point) != partitioning.num_outcomes:
126 raise ValueError(
127 "The length of the reference point must match the number of outcomes. "
128 f"Got ref_point with {len(ref_point)} elements, but expected "
129 f"{partitioning.num_outcomes}."
130 )
131 ref_point = torch.tensor(
132 ref_point,
--> 133 dtype=partitioning.pareto_Y.dtype,
134 device=partitioning.pareto_Y.device,
135 )
136 better_than_ref = (partitioning.pareto_Y > ref_point).all(dim=1)
137 if not better_than_ref.any() and partitioning.pareto_Y.shape[0] > 0:

Is there has any example that can show me how to use this acquisition function and how to fix this error?

Thanks a lot for helping me!!

[saitcakmak]

I've responded in the other thread. Let's continue here if you have any follow up questions.

Hi @An-xu. You should pass in the Y observed so far into NondominatedPartitioning. The qEHVI parts of this tutorial are probably the closest reference we have for EHVI.

[An-xu]

Thank you for your reply! I also encounter a problem:
for each iteration, I did this:
train_x = torch.cat([train_x, candidates])
with torch.no_grad():
new = model.posterior(candidates).mean
uncertainty = model.posterior(candidates).variance
train_y = torch.cat([train_y, new],dim=0)
print(f"New pred: {new}, uncertainty: {uncertainty}")
for hvs_list, train_obj in zip(
( hvs_ehvi,),
(
train_y,
),
):
# compute hypervolume
bd = DominatedPartitioning(ref_point=ref_point, Y=train_y)
volume = bd.compute_hypervolume().item()
hvs_list.append(volume)
print(f'Batch {i}: Hypervolume (EHVI)=( {hvs_ehvi[-1]:>4.2f})')

But the hypervolumes are all zeros, and I don't know how to fix it

[saitcakmak]

But the hypervolumes are all zeros

This tells me that none of train_y dominates the ref_point. BoTorch assumes maximization of objectives throughout (with the option to use minimization in a few places), so dominating here would mean that for some value in train_y, all outcomes must be greater than the corresponding ref_point value.

The new = model.posterior(candidates).mean part also looks a bit odd to me. Why are you using the model posterior mean as the new observation and not the objective function that you're trying to optimize?

[An-xu]

Hi,
For this question "The new = model.posterior(candidates).mean part also looks a bit odd to me. Why are you using the model posterior mean as the new observation and not the objective function that you're trying to optimize?"

That's because I want to maximize the output which is train_y here. And for me, there is no objective function to define because we have experiment data points as the observation points and I want to generate new inputs that potentially can achieve a better result. But I the Hypervolume improvements are all zero here.

[saitcakmak]

The hypervolume is computed with respect to the reference point, so the reference point plays a crucial role here. Is your initial hypervolume also zero? If so, you may want to relax the reference point to ensure that the acquisition function has some non-zero hypervolumes to optimize over. To find a suitable reference point based on the nadir point, you can use the infer_reference_point helper.

It also sounds like you're trying to generate multiple candidates here, and using the posterior mean as the observations to facilitate this. A better approach might be to use qExpectedHypervolumeImprovement or qNoisyExpectedHypervolumeImprovement and call optimize_acqf with q as the number of candidates you want to generate and sequential=True. This will condition on the possible values of the previously generated candidates while generating the next candidate, without restricting the possible values to the posterior mean.

[An-xu]

Hi!

Can this compute_hypervolume() give the corresponding Hypervolume of the point or it is given an optimized hypervolume?
I try to look the code of compute_hypervolume() but I didn't find it.
bd = DominatedPartitioning(ref_point=ref_point, Y=train_y)
volume = bd.compute_hypervolume().item()

[saitcakmak]

compute_hypervolume calculates the hypervolume defined by the Y & the ref_point provided to the corresponding box decomposition / partitioning class. There's no optimization that happens under the hood, it just calculates the hypervolume from the provided inputs.

The main method is defined in the parent class https://github.com/pytorch/botorch/blob/main/botorch/utils/multi_objective/box_decompositions/box_decomposition.py#L231 and it uses the private methods from the particular class to do the computations, e.g., https://github.com/pytorch/botorch/blob/main/botorch/utils/multi_objective/box_decompositions/dominated.py#L51. This part of the code is a bit complicated as there's a lot of code sharing with the parent classes.

[An-xu]

Hi,
I would like to ask one question about EHVI. The recommended answer of EHVI is not stable (not 1 answer even the input is the same). Is it because of the num_restart and raw_samples in the optimize_acqf function?
Thank you so much for your help!

[Balandat]

Yes, the optimization surface for these problems can be mine quite complicated, and as this is a non-convex problem we generally don't have a guarantee to find the global optimum. So yes, the randomness here is likely due to the randomness in the initial stating points from which we do gradient based optimization.

You should be able to set a random seed though to make the behavior deterministic if you need that.

[An-xu]

Thank you so much for your help!

[An-xu]

I have another question, I would like to add the standard deviation of the input to the model, can I still use SingleTaskGP? And can I tune the hyperparameter of the SingleTaskGP model?

[Balandat]

I would like to add the standard deviation of the input to the model

Do you mean the variance of the observations? SingleTaskGP infers that noise level, but you can use FixedNoiseGP to pass the variances.

And can I tune the hyperparameter of the SingleTaskGP model?

In the context of passing in the variance? Or more generally? What hyperparameters would you like to tune?

[An-xu]

I would like to tune the lengthscale, but I didn't see that in documents

[saitcakmak]

When you use fit_gpytorch_mll to fit a model, it trains the model hyper-parameters, including the length-scales. The model fitting part should be included in most tutorials on the website. Unless you want to change the priors on the length-scales, you do not need to make any changes to the models. If you want to change the priors, you can pass in a custom covar_module while initializing the model (this would require some understanding of GPyTorch).

[An-xu]

Thank you for your reply, I understand it can train the model hyper-parameters including length-scales. But I would like to know if there is any way for me to tune the length-scales or make a bound for length-scales.