New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

关于ppl的方差 #54

Open

skepsun opened this issue Feb 1, 2024 · 1 comment

skepsun commented Feb 1, 2024

我在hh数据集上训练模型，并统计ppl，只基于回答计算ppl，发现ppl的方差特别大，一些特别短的回复，比如"270 votes"，ppl可以达到35000多，我看RRHF论文中没有提及ppl的方差，想问一下是否也有类似的情况

Owner

GanjinZero commented Feb 1, 2024

基于问题和回答计算就没有那么大的方差了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment