Code

Add an LLM fine-tuning example#90

Code

Merged

lebrice merged 43 commits intomastermila-iqia/ResearchTemplate:masterfrom add_nlp_examplemila-iqia/ResearchTemplate:add_nlp_exampleCopy head branch name to clipboardNov 15, 2024

+1,319-349

Commits

Commits on Nov 8, 2024

• WIP: Add an LLM finetuning example

  Show description for f103347
  lebrice

  committed
  f103347

  Copy full SHA for f103347

• WIP: add / rename more configs

  Show description for 9b9d698
  lebrice

  committed
  9b9d698

  Copy full SHA for 9b9d698

• Finetuning example seems to be working

  Show description for d0500b4
  lebrice

  committed
  d0500b4

  Copy full SHA for d0500b4

• Making progress, more self-contained example

  Show description for e68a1dc
  lebrice

  committed
  e68a1dc

  Copy full SHA for e68a1dc

• Works! (need to fix the hash used for path though)

  Show description for 59060d9
  lebrice

  committed
  59060d9

  Copy full SHA for 59060d9

• Improve hashing, reduce default block size

  Show description for a678a93
  lebrice

  committed
  a678a93

  Copy full SHA for a678a93

• Fix val_loss logging and add docstring

  Show description for 4c11b99
  lebrice

  committed
  4c11b99

  Copy full SHA for 4c11b99

• Increase the number of dataloader workers

  Show description for 7b9fb19
  lebrice

  committed
  7b9fb19

  Copy full SHA for 7b9fb19

Commits on Nov 11, 2024

• Use smaller model for now

  Show description for cfa3a26
  lebrice

  committed
  cfa3a26

  Copy full SHA for cfa3a26

• Use FSDP in the example

  Show description for 10c07eb
  lebrice

  committed
  10c07eb

  Copy full SHA for 10c07eb

• Fix bug in id generation from config classes

  Show description for 5c4f659
  lebrice

  committed
  5c4f659

  Copy full SHA for 5c4f659

• Tweak config, try to setup mid-epoch checkpointing

  Show description for f1fced9
  lebrice

  committed
  f1fced9

  Copy full SHA for f1fced9

• Rename HFExample -> TextClassificationExample

  Show description for f02e4da
  lebrice

  committed
  f02e4da

  Copy full SHA for f02e4da

• Fix broken links in nav

  Show description for 09a5b74
  lebrice

  committed
  09a5b74

  Copy full SHA for 09a5b74

Commits on Nov 12, 2024

• Remove "huggingface" datamodule config

  Show description for 74161cc
  lebrice

  committed
  74161cc

  Copy full SHA for 74161cc

• Fix issues in config/tests for text_classification

  Show description for ef454af
  lebrice

  committed
  ef454af

  Copy full SHA for ef454af

• Add an entry to test the llm_finetuning_example

  Show description for 504ece9
  lebrice

  committed
  504ece9

  Copy full SHA for 504ece9

• Fix issues in the text classification example

  Show description for 32f4f91
  lebrice

  committed
  32f4f91

  Copy full SHA for 32f4f91

• Fix weird docstring issues with hydra-zen

  Show description for ade8bc0
  lebrice

  committed
  ade8bc0

  Copy full SHA for ade8bc0

• Fix test and config of text_classification_example

  Show description for 5bca6b9
  lebrice

  committed
  5bca6b9

  Copy full SHA for 5bca6b9

• Move test from main_test.py to example_test.py

  Show description for 8749842
  lebrice

  committed
  8749842

  Copy full SHA for 8749842

• forward_pass is a method of LearningAlgorithmTests

  Show description for 4f5e4fb
  lebrice

  committed
  4f5e4fb

  Copy full SHA for 4f5e4fb

• Various type hint fixes and tweaks

  Show description for 107176e
  lebrice

  committed
  107176e

  Copy full SHA for 107176e

• WIP: Adding some tests for LLM finetuning example

  Show description for 3f08a75
  lebrice

  committed
  3f08a75

  Copy full SHA for 3f08a75

• Fix issue in jax.md

  Show description for e0a26b9
  lebrice

  committed
  e0a26b9

  Copy full SHA for e0a26b9

• Add link to the example page in index.md

  Show description for 3077364
  lebrice

  committed
  3077364

  Copy full SHA for 3077364

Commits on Nov 13, 2024

• Fix tests for the llm finetuning example

  Show description for d2834bc
  lebrice

  committed
  d2834bc

  Copy full SHA for d2834bc

• Fix issue with tuples in regression files

  Show description for 254224b
  lebrice

  committed
  254224b

  Copy full SHA for 254224b

• Fix test for get_hash_of

  Show description for b9bc199
  lebrice

  committed
  b9bc199

  Copy full SHA for b9bc199

• Remove unused _field function

  Show description for 6f5e367
  lebrice

  committed
  6f5e367

  Copy full SHA for 6f5e367

• Fix issue with built-in modules in autoref plugin

  Show description for ce61959
  lebrice

  committed
  ce61959

  Copy full SHA for ce61959

• Add a bit of info in the example doc

  Show description for 39f2226
  lebrice

  committed
  39f2226

  Copy full SHA for 39f2226

• Add more links in the doc of the module

  Show description for 2ae7e1f
  lebrice

  committed
  2ae7e1f

  Copy full SHA for 2ae7e1f

• Fix issue with the text classification example

  Show description for 54efae6
  lebrice

  committed
  54efae6

  Copy full SHA for 54efae6

• Add skipif mark for LLM finetuning test

  Show description for 63a530d
  lebrice

  committed
  63a530d

  Copy full SHA for 63a530d

Commits on Nov 14, 2024

• Fix data_dir of text_classification_example

  Show description for 10c52fc
  lebrice

  committed
  10c52fc

  Copy full SHA for 10c52fc

• Use the "auto" strategy for LLM Finetuning tests

  Show description for 7b69c00
  lebrice

  committed
  7b69c00

  Copy full SHA for 7b69c00

• Fix error in fork_rng of LLM finetuning example

  Show description for 59e0673
  lebrice

  committed
  59e0673

  Copy full SHA for 59e0673

• Try a hacky fix for failing test

  Show description for 922cdb7
  lebrice

  committed
  922cdb7

  Copy full SHA for 922cdb7

• Don't run llm finetuning tests on github Cloud CI

  Show description for e36de4a
  lebrice

  committed
  e36de4a

  Copy full SHA for e36de4a

Commits on Nov 15, 2024

• Add missing regression files

  Show description for b06f3bf
  lebrice

  committed
  b06f3bf

  Copy full SHA for b06f3bf

• Rename llm_finetuning_example -> llm_finetuning

  Show description for 04be192
  lebrice

  committed
  04be192

  Copy full SHA for 04be192

• Fix import error

  Show description for 102c51c
  lebrice

  committed
  102c51c

  Copy full SHA for 102c51c