微调阶段会占用多少GPU? #146
Unanswered
lychee-2724540853
asked this question in
Q&A
微调阶段会占用多少GPU?
#146
Replies: 1 comment 1 reply
-
|
按照当前的训练设置(微调max_seq_length=512，预训练block_size=512)，P40使用显存约22.5G。 |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
是去掉了去除embed_tokens层和lm_head的训练是吗 |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
我现在在单卡24G的GPU上，去除embed_tokens层和lm_head的训练，句子长度64，，GPU基本拉满。。。有其他设置能降低占用吗？
另外，预训练阶段的block_size参数有什么作用？微调阶段没有这个参数
Beta Was this translation helpful? Give feedback.
All reactions