New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

vocab #19

Open

pureblacker opened this issue Mar 4, 2022 · 1 comment

pureblacker commented Mar 4, 2022

请问一下不是说tokennizer里面会包含有这个模型所用的词表嘛？还是说做中文的文本生成，都要自己添加词表

Owner

liucongg commented Mar 11, 2022

中文的文本生成，都要自己添加词表

tokenizer加载词表有什么问题吗？

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment