Llama 3.1 liger example is not working

[Stealthwriter]

Please check that this issue hasn't been reported before.

• I searched previous Bug Reports didn't find any similar reports.

Expected Behavior

it should train

Current behaviour

examples/llama-3/fft-8b-liger-fsdp.yaml

this example is not working
optimizer: paged_adamw_8bit
is not compatible with fsdp
I tried changing it I still get this error: Value error, FSDP Offload not compatible with adamw_bnb_8bit

I commented out the fsdp settings and used deep speed it worked

Steps to reproduce

run example as is

Config yaml

No response

Possible solution

No response

Which Operating Systems are you using?

• Linux
• macOS
• Windows

Python Version

3.10

axolotl branch-commit

latest

Acknowledgements

• My issue title is concise, descriptive, and in title casing.
• I have searched the existing issues to make sure this bug has not been reported yet.
• I am using the latest version of axolotl.
• I have provided enough information for the maintainers to reproduce and diagnose the issue.

[ganler]

Seems not working for Mistral too.

Got the same error in linkedin/Liger-Kernel#100 even if upgraded Liger