Code

Ensembling over layers#259

Code

Open

lauritowal wants to merge 79 commits intomainEleutherAI/elk:mainfrom ensembling_layerEleutherAI/elk:ensembling_layerCopy head branch name to clipboard

+363-93

Commits

Commits on Apr 26, 2023

• Log ensembled metrics

  norabelrose

  committed
  934cd54

  Copy full SHA for 934cd54

• Fixing pyright version

  norabelrose

  committed
  dff69bf

  Copy full SHA for dff69bf

• Merge remote-tracking branch 'origin/main' into ensembling

  norabelrose

  committed
  b181d3e

  Copy full SHA for b181d3e

Commits on Apr 29, 2023

• experiment with layer ensembling

  lauritowal

  committed
  a493b85

  Copy full SHA for a493b85

Commits on Apr 30, 2023

• add draft example for ensembling datasets

  lauritowal

  committed
  af5def6

  Copy full SHA for af5def6

• add comment

  lauritowal

  committed
  04a2a82

  Copy full SHA for 04a2a82

• Merge branch 'main' into ensembling_layer

  lauritofzi

  committed
  f433885

  Copy full SHA for f433885

• add eval in comments

  lauritowal

  committed
  cda7de7

  Copy full SHA for cda7de7

• Merge branch 'ensembling_layer' of https://github.com/EleutherAI/elk into ensembling_layer

  lauritowal

  committed
  c9f2558

  Copy full SHA for c9f2558

Commits on May 1, 2023

• add different root

  lauritofzi

  committed
  0ceaa3a

  Copy full SHA for 0ceaa3a

• Merge branch 'ensembling_layer' of https://github.com/EleutherAI/elk into ensembling_layer

  lauritofzi

  committed
  86fb1c8

  Copy full SHA for 86fb1c8

Commits on May 24, 2023

• Merge branch 'main' into ensembling_layer

  lauritowal

  committed
  47a3f60

  Copy full SHA for 47a3f60

Commits on May 27, 2023

• add empty list of vals

  lauritowal

  committed
  0bd274f

  Copy full SHA for 0bd274f

Commits on Jun 16, 2023

• Merge branch 'main' into ensembling_layer

  lauritowal

  committed
  04f0b4c

  Copy full SHA for 04f0b4c

Commits on Jun 17, 2023

• add first version of layer ensembling to eval

  lauritowal

  committed
  994af9b

  Copy full SHA for 994af9b

Commits on Jun 19, 2023

• add vals to train

  lauritowal

  committed
  6ca1916

  Copy full SHA for 6ca1916

• refactoring & cleanup of eval and layer ensembling

  lauritowal

  committed
  b0d0f83

  Copy full SHA for b0d0f83

• add annotations

  lauritowal

  committed
  241a03a

  Copy full SHA for 241a03a

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for e8d042a
  pre-commit-ci[bot]

  committed
  e8d042a

  Copy full SHA for e8d042a

• rename vals to layer_outputs

  lauritowal

  committed
  a4ace25

  Copy full SHA for a4ace25

• Merge branch 'ensembling_layer' of https://github.com/EleutherAI/elk into ensembling_layer

  lauritowal

  committed
  b025c71

  Copy full SHA for b025c71

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 2156ad8
  pre-commit-ci[bot]

  committed
  2156ad8

  Copy full SHA for 2156ad8

• fir formatting

  lauritowal

  committed
  e391da6

  Copy full SHA for e391da6

• Merge branch 'ensembling_layer' of https://github.com/EleutherAI/elk into ensembling_layer

  lauritowal

  committed
  449971f

  Copy full SHA for 449971f

Commits on Jun 21, 2023

• make layer ensembling work on multiple gpus

  lauritowal

  committed
  528367d

  Copy full SHA for 528367d

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for d4df517
  pre-commit-ci[bot]

  committed
  d4df517

  Copy full SHA for d4df517

• make sure we use the same device

  lauritowal

  committed
  2661ea1

  Copy full SHA for 2661ea1

• Merge branch 'ensembling_layer' of https://github.com/EleutherAI/elk into ensembling_layer

  lauritowal

  committed
  043aa7a

  Copy full SHA for 043aa7a

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 21cccb7
  pre-commit-ci[bot]

  committed
  21cccb7

  Copy full SHA for 21cccb7

• calc layer ensembling for all prompt ensembling modes

  lauritowal

  committed
  2495c3a

  Copy full SHA for 2495c3a

• Merge branch 'ensembling_layer' of https://github.com/EleutherAI/elk into ensembling_layer

  lauritowal

  committed
  69af43c

  Copy full SHA for 69af43c

Commits on Jun 23, 2023

• implement ensembling enum

  derpyplops

  committed
  908308b

  Copy full SHA for 908308b

• Fix ensembling value writing error

  derpyplops

  committed
  fc980d7

  Copy full SHA for fc980d7

• accidentally a print

  derpyplops

  committed
  5aa30a9

  Copy full SHA for 5aa30a9

• slightly refactor layer stuff and fix tests

  derpyplops

  committed
  d5b8584

  Copy full SHA for d5b8584

• try fixing type hints

  derpyplops

  committed
  6380814

  Copy full SHA for 6380814

• tidy up output

  derpyplops

  committed
  98d19b7

  Copy full SHA for 98d19b7

• accidentally a char

  derpyplops

  committed
  e6914e1

  Copy full SHA for e6914e1

Commits on Jun 24, 2023

• rename to PromptEnsembling

  lauritowal

  committed
  29b1cb8

  Copy full SHA for 29b1cb8

Commits on Jul 9, 2023

• Merge branch 'main' into ensembling_layer

  lauritowal

  committed
  421590c

  Copy full SHA for 421590c

• add annotations and types

  lauritowal

  committed
  bed615a

  Copy full SHA for bed615a

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for bf49e99
  pre-commit-ci[bot]

  committed
  bf49e99

  Copy full SHA for bf49e99

• clearer naming: prompt_ensembling

  lauritowal

  committed
  1f5d8be

  Copy full SHA for 1f5d8be

Commits on Jul 12, 2023

• Merge branch 'ensembling_layer' of https://github.com/EleutherAI/elk into ensembling_layer

  lauritowal

  committed
  03a37d2

  Copy full SHA for 03a37d2

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for ec37716
  pre-commit-ci[bot]

  committed
  ec37716

  Copy full SHA for ec37716

• better name for ensembling

  lauritowal

  committed
  1936624

  Copy full SHA for 1936624

• Merge branch 'ensembling_layer' of https://github.com/EleutherAI/elk into ensembling_layer

  lauritowal

  committed
  b243932

  Copy full SHA for b243932

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 484788e
  pre-commit-ci[bot]

  committed
  484788e

  Copy full SHA for 484788e

• Merge branch 'main' into ensembling_layer

  lauritowal

  committed
  8c34797

  Copy full SHA for 8c34797

• Merge branch 'ensembling_layer' of https://github.com/EleutherAI/elk into ensembling_layer

  lauritowal

  committed
  ea5e9e8

  Copy full SHA for ea5e9e8

Commits on Jul 13, 2023

• Merge branch 'main' into ensembling_layer

  lauritowal

  committed
  c0545aa

  Copy full SHA for c0545aa

• remove pseudo auroc

  lauritowal

  committed
  b6de957

  Copy full SHA for b6de957

• rename to prompt_ensembling

  lauritowal

  committed
  cf32b0c

  Copy full SHA for cf32b0c

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 769676a
  pre-commit-ci[bot]

  committed
  769676a

  Copy full SHA for 769676a

• precomit fixes

  lauritowal

  committed
  e6c9d4c

  Copy full SHA for e6c9d4c

• Merge branch 'ensembling_layer' of https://github.com/EleutherAI/elk into ensembling_layer

  lauritowal

  committed
  8093294

  Copy full SHA for 8093294

Commits on Jul 18, 2023

• fix num_classes

  lauritowal

  committed
  6028152

  Copy full SHA for 6028152

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 6d7d99a
  pre-commit-ci[bot]

  committed
  6d7d99a

  Copy full SHA for 6d7d99a

• fix bug where y_true has a dimension of two

  lauritowal

  committed
  4148857

  Copy full SHA for 4148857

• cleanup

  lauritowal

  committed
  964f03d

  Copy full SHA for 964f03d

• Merge branch 'ensembling_layer' of https://github.com/EleutherAI/elk into ensembling_layer

  lauritowal

  committed
  f7ed262

  Copy full SHA for f7ed262

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 06dad69
  pre-commit-ci[bot]

  committed
  06dad69

  Copy full SHA for 06dad69

• add y_true_initial

  lauritowal

  committed
  0d2545b

  Copy full SHA for 0d2545b

• merge

  lauritowal

  committed
  4a717ce

  Copy full SHA for 4a717ce

• fix test error

  lauritowal

  committed
  5952b4b

  Copy full SHA for 5952b4b

Commits on Jul 22, 2023

• Merge branch 'main' into ensembling_layer

  lauritowal

  authored
  7efe38f

  Copy full SHA for 7efe38f

• replace mode with prompt_ensembling.value

  lauritowal

  authored
  049cd63

  Copy full SHA for 049cd63

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for c8236dd
  pre-commit-ci[bot]

  committed
  c8236dd

  Copy full SHA for c8236dd

Commits on Jul 23, 2023

• remove .value for lm_eval

  lauritowal

  committed
  56d1796

  Copy full SHA for 56d1796

Commits on Jul 27, 2023

• add LayerApplied

  derpyplops

  committed
  4d9c781

  Copy full SHA for 4d9c781

• fix run.py part

  derpyplops

  committed
  8961e95

  Copy full SHA for 8961e95

• multidataset layer ensembling

  derpyplops

  committed
  bd06cd3

  Copy full SHA for bd06cd3

• little refactoring

  Show description for f8882c6
  derpyplops

  committed
  f8882c6

  Copy full SHA for f8882c6

• fix tests

  derpyplops

  committed
  23183bc

  Copy full SHA for 23183bc

Commits on Jul 31, 2023

• add annotation + cleanup

  lauritowal

  committed
  d091f9d

  Copy full SHA for d091f9d

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 776c186
  pre-commit-ci[bot]

  committed
  776c186

  Copy full SHA for 776c186

• Merge pull request #282 from EleutherAI/fix-ensembling-jon

  Show description for 9629ba5
  derpyplops

  authored
  9629ba5

  Copy full SHA for 9629ba5

Commits on Oct 13, 2023

• Merge branch 'main' into ensembling_layer

  derpyplops

  committed
  45b527f

  Copy full SHA for 45b527f

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 64e762a
  derpyplops

  committed
  64e762a

  Copy full SHA for 64e762a