关于大海捞针测试的问题 #919
关于大海捞针测试的问题
#919
-
|
看config中cdme数据集配置时有一个'tokenizer_model': 'gpt-4'参数，想请问一下这里的这个参数是什么作用呢，如果要测试自己的模型也要用gpt-4嘛 还是需要修改成自己模型tokenizer的路径呢？感谢回答 |
Beta Was this translation helpful? Give feedback.
Answered by
tonysy
Feb 26, 2024
Replies: 1 comment 3 replies
-
|
We use it for length calculation(token). |
Beta Was this translation helpful? Give feedback.
3 replies
-
|
那请问一下如果想测试自己的模型的时候需要修改嘛 |
Beta Was this translation helpful? Give feedback.
-
|
No |
Beta Was this translation helpful? Give feedback.
-
|
好的好的感谢 |
Beta Was this translation helpful? Give feedback.
Answer selected by
tonysy
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We use it for length calculation(token).