Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

labels != -100的作用是什么 #38

Open
LSX-Sneakerprogrammer opened this issue Jul 19, 2023 · 3 comments
Open

labels != -100的作用是什么 #38

LSX-Sneakerprogrammer opened this issue Jul 19, 2023 · 3 comments

Comments

@LSX-Sneakerprogrammer
Copy link

您好，我想请问一下在代码中labels != -100的作用是什么。根据论文中的理解，mask的作用应该是遮盖query的以计算response的长度，但是按照代码中的写法，似乎是固定的max_length长度。希望您能够帮助解答，感谢！
@GanjinZero
Copy link
Owner

计算response长度；如果实现不符合预期就是bug

@LSX-Sneakerprogrammer
Copy link
Author

LSX-Sneakerprogrammer commented Jul 19, 2023 •
edited
Loading

计算response长度；如果实现不符合预期就是bug

感谢您的解答！我想再问一下，对response计算rrhf_loss和ft_loss时，是需要把query和padding部分都mask掉吗，还是只mask掉query部分呢

@GanjinZero
Copy link
Owner

都mask

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@GanjinZero @LSX-Sneakerprogrammer