Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Eugene/train test split #87

Open
wants to merge 13 commits into
base: rwkv-x-h100
Choose a base branch
from
Open

Eugene/train test split #87

wants to merge 13 commits into from

Commits on Mar 6, 2024

  1. tweak to data set processing, to have test dataset split overwrite

    @PicoCreator
    PicoCreator committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    65a5c49 View commit details
    Browse the repository at this point in the history

  2. wip test column overwrite

    @PicoCreator
    PicoCreator committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    2c424d5 View commit details
    Browse the repository at this point in the history

  3. fixing prefix/suffix masking for multi-key dataset, and handling of i… 

    …nt values as str in dataset
    @PicoCreator
    PicoCreator committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    4f2c9e2 View commit details
    Browse the repository at this point in the history

  4. fixing instruct tuning data processing

    @PicoCreator
    PicoCreator committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    bacff8d View commit details
    Browse the repository at this point in the history

  5. better empty string handling

    @PicoCreator
    PicoCreator committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    90faac0 View commit details
    Browse the repository at this point in the history

Commits on Mar 7, 2024

  1. datapacking improvements for multi-turn data & masking

    @PicoCreator
    PicoCreator committed Mar 7, 2024
    Configuration menu
    Copy the full SHA
    a9c0633 View commit details
    Browse the repository at this point in the history

  2. wip runs

    @PicoCreator
    PicoCreator committed Mar 7, 2024
    Configuration menu
    Copy the full SHA
    6561ff0 View commit details
    Browse the repository at this point in the history

  3. wip test split testing

    @PicoCreator
    PicoCreator committed Mar 7, 2024
    Configuration menu
    Copy the full SHA
    f2acb6a View commit details
    Browse the repository at this point in the history

  4. WIP train, and 32k dataset

    @PicoCreator
    PicoCreator committed Mar 7, 2024
    Configuration menu
    Copy the full SHA
    c7ce34d View commit details
    Browse the repository at this point in the history

  5. wip dataprep

    @PicoCreator
    PicoCreator committed Mar 7, 2024
    Configuration menu
    Copy the full SHA
    05eb723 View commit details
    Browse the repository at this point in the history

Commits on Mar 8, 2024

  1. handling data as boolean

    @PicoCreator
    PicoCreator committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    11a8f30 View commit details
    Browse the repository at this point in the history

  2. wip simpleRWKV support

    @PicoCreator
    PicoCreator committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    d92047c View commit details
    Browse the repository at this point in the history

  3. wip data prep

    @PicoCreator
    PicoCreator committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    fb6023e View commit details
    Browse the repository at this point in the history