[Fixing] More finetuning support

[danielhanchen]

• Support sequence classification
• Flex Attention for Gemma and others
• Variable sequence length and auto unpadding / padding
• Tool Calling
• Refactor and merge xformers, SDPA, flash-attn, flex-attention

[Zzhiter]

Need help? I can help with 1

[Zzhiter]

@danielhanchen Hi, in fact, I have implemented sequence classification and supported LoRA. Through my experiments, there is no loss. May I contribute my code to the community?

[shimmyshimmer]

@danielhanchen Hi, in fact, I have implemented sequence classification and supported LoRA. Through my experiments, there is no loss. May I contribute my code to the community?

Need help? I can help with 1

Hi there @Zzhiter much much apologies for the late reply! We missed your comment. absolutely we would LOVE to have you collab with us. Do you happen to have Discord or Slack or would you just like to communicate here? :)

[Zzhiter]

@danielhanchen Hi, in fact, I have implemented sequence classification and supported LoRA. Through my experiments, there is no loss. May I contribute my code to the community?

Need help? I can help with 1

Hi there @Zzhiter much much apologies for the late reply! We missed your comment. absolutely we would LOVE to have you collab with us. Do you happen to have Discord or Slack or would you just like to communicate here? :)

I've joined the Unsloth community's Discord server. However, I haven't used Discord frequently before. Let's communicate here directly! Shall I briefly introduce my implementation ideas first?

[Dillion]

i'm interested in joining this discussion, @Zzhiter can you share?