Skip to content

Issues: zyushun/Adam-mini

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

16 Open 13 Closed
16 Open 13 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

FSDP and tensor-paralllel compatibility
#36 opened Dec 16, 2024 by asahni04
10
Default process group has not been initialized
#35 opened Dec 11, 2024 by yi249
2
No module named 'torch.distributed._tensor'
#33 opened Nov 29, 2024 by SK-HC
3
No Query or Key Found when using nn.TransformerEncoderLayer
#30 opened Sep 25, 2024 by USBskycrafts
24
Hessian Calculation
#29 opened Sep 25, 2024 by lucasliunju
1
RuntimeError: No backend type associated with device type cpu
#28 opened Sep 18, 2024 by minienglish1
3
does not support bf16-only training
#26 opened Aug 18, 2024 by bghira
1
Qwen2-0.5B cannot be Adam-mini-optimized in 4 shards (Deepspeed Zero-3)
#25 opened Aug 16, 2024 by xiningnlp
3
Megatron support, please
#23 opened Aug 15, 2024 by zhuango
2
ValueError: algorithm not supported when running gpt2 example
#20 opened Jul 25, 2024 by Benjamin-eecs
2
集成到transformer的trainer后会会爆显存，用adamW不会
#15 opened Jul 16, 2024 by Panda-eat-meat
1
使用Adam-mini，全参量微调7B模型需要多少显存？
#13 opened Jul 14, 2024 by NTDXYG
4
Memory saving only in checkpoint size, not during training
#8 opened Jul 6, 2024 by aditya2331
7
works well on smaller models, updates for torchtitan and 8B size
#7 opened Jul 4, 2024 by lessw2020
23
Adam mini can't save when using with FSDP in Huggingface Trainer
#5 opened Jun 27, 2024 by hahuyhoang411
19
Is there any plant to implement Quantized Adam-mini?
#4 opened Jun 27, 2024 by Kyeongpil
4
ProTip! Type g p on any issue or pull request to go back to the pull request listing page.