Finetuning scripts

[trufae]

Extending base models with custom information like r2 source code, the book, disassembly data, decompilation code and more is interesting, so r2ai should provide the basic infrastructure to finetune models without the hassle to write code. This guide is quite comprensible and easy to follow.

https://medium.com/@mohammed97ashraf/your-ultimate-guide-to-instinct-fine-tuning-and-optimizing-googles-gemma-2b-using-lora-51ac81467ad2

[dnakov]

Yeah it'd be nice to facilitate that process but I think it'd be best if we provided a data set and something like an example for how to run a LORA fine tune, or even just a link to one. There are many other projects that are better suited to handle the fine tuning process.
Fine tuning scripts get stale quickly, especially now with a lot of different variations of lora.

[dnakov]

On the other hand, it'd be good if we trained and maintained an r2 model and provided the weights. I have some AWS credits i don't mind burning for this but would need help preparing the dataset

[trufae]

Agree on that. What we should focus is on documenting and providing ways to generate all this training data in a way that can be consumed to finetune our own models. ideally the functionary one or the mistral. or even utopia are the ones im using the most and would love to improve.

[trufae]

https://www.kdnuggets.com/mistral-7b-v02-fine-tuning-mistral-new-open-source-llm-with-hugging-face