Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question on intuition behind choice of -21 for min_losses #10

Open
imanfs opened this issue Sep 5, 2024 · 0 comments
Open

Question on intuition behind choice of -21 for min_losses #10

imanfs opened this issue Sep 5, 2024 · 0 comments

Comments

@imanfs
Copy link

imanfs commented Sep 5, 2024

Hello,

In the toy example here you have set self.min_losses = torch.Tensor([-21 * scale, -21]) where scale = 1e-1 by default. If I understand correctly, this is meant to replicate a scenario where the task losses have a scale imbalance during training, by always multiplying the first task's loss by the scale parameter.

However I was wondering why the scale for the first loss-pair was chosen as [-21 * scale, -21]--why -21 as opposed to a positive number or just 1? Is it because of the Toy problem's loss function, which from glancing at the code

 f1_sq = ((-x1 + 7).pow(2) + 0.1 * (-x2 - 8).pow(2)) / 10 - 20

potentially calculates a negative loss value, or is there a deeper reason for it?

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant