Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve CI #27

Merged
merged 26 commits into from
Feb 6, 2024
Merged

Improve CI #27

merged 26 commits into from
Feb 6, 2024

Commits on Feb 5, 2024

  1. Test all three configs in CI matrix

    @mwaskom
    mwaskom committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    584bebe View commit details
    Browse the repository at this point in the history

  2. Minimal config updates

    @mwaskom
    mwaskom committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    4fa39ee View commit details
    Browse the repository at this point in the history

  3. Run CI workflow on PRs only

    @mwaskom
    mwaskom committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    f904f00 View commit details
    Browse the repository at this point in the history

  4. Fix CI yaml

    @mwaskom
    mwaskom committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    b0876c6 View commit details
    Browse the repository at this point in the history

  5. Write the updated config back out

    @mwaskom
    mwaskom committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    593eeed View commit details
    Browse the repository at this point in the history

  6. Fix config read

    @mwaskom
    mwaskom committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    255b2ec View commit details
    Browse the repository at this point in the history

  7. Temp comment out hugginface secret

    @mwaskom
    mwaskom committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    e16ed1e View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2024

  1. Fix data truncation

    @mwaskom
    mwaskom committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    700b15c View commit details
    Browse the repository at this point in the history

  2. Update modal environment name

    @mwaskom
    mwaskom committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    233f220 View commit details
    Browse the repository at this point in the history

  3. Train on more data

    @mwaskom
    mwaskom committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    9f9e131 View commit details
    Browse the repository at this point in the history

  4. Skip validation in CI

    @mwaskom
    mwaskom committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    61b6a05 View commit details
    Browse the repository at this point in the history

  5. Nevermind, I guess we can't disable evals

    @mwaskom
    mwaskom committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    ef7a35b View commit details
    Browse the repository at this point in the history

  6. Revert changes to train / val sizes

    @mwaskom
    mwaskom committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    5f5d343 View commit details
    Browse the repository at this point in the history

  7. Even more val data?

    @mwaskom
    mwaskom committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    74db375 View commit details
    Browse the repository at this point in the history

  8. Fix default mistral sequence len

    @mwaskom
    mwaskom committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    6a5cfa0 View commit details
    Browse the repository at this point in the history

  9. Set eval steps and batch size for CI

    @mwaskom
    mwaskom committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    f21f60e View commit details
    Browse the repository at this point in the history

  10. Print some cuda diagnostics before training

    @mwaskom
    mwaskom committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    f28f503 View commit details
    Browse the repository at this point in the history

  11. Print some config / data diagnostics

    @mwaskom
    mwaskom committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    9798de1 View commit details
    Browse the repository at this point in the history

  12. Don't cancel other matrix jobs one when fails

    @mwaskom
    mwaskom committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    ef8b1f7 View commit details
    Browse the repository at this point in the history

  13. Fix torch attributes

    @mwaskom
    mwaskom committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    737183f View commit details
    Browse the repository at this point in the history

  14. Try larger mixtral micro batch size?

    @mwaskom
    mwaskom committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    231562f View commit details
    Browse the repository at this point in the history

  15. Force micro batch size

    @mwaskom
    mwaskom committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    ddb94d3 View commit details
    Browse the repository at this point in the history

  16. Try setting eval_steps to implausibly large number

    @mwaskom
    mwaskom committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    9185172 View commit details
    Browse the repository at this point in the history

  17. Don't watch gradients in mistral; set eval_steps to default

    @mwaskom
    mwaskom committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    009849b View commit details
    Browse the repository at this point in the history

  18. Try no flash attention?

    @mwaskom
    mwaskom committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    89ab93e View commit details
    Browse the repository at this point in the history

  19. Smaller batch size

    @mwaskom
    mwaskom committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    8046918 View commit details
    Browse the repository at this point in the history