New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added diff function for TsCorpus[resolves #186] #207

Open

ertugrul-dmr wants to merge 2 commits into develop from feature/ertugrul-dmr_186_tscorpus_diff

Contributor

ertugrul-dmr commented Jan 28, 2021 •

edited by dafajon

Loading

sadedegel/dataset/tscorpus/__main__.py:

Added diff command for sadedegel.dataset.tscorpus CLI.
Add tokenizer and sample_size as options to diff command:
- Valid values are simple and bert for tokenizer. Hashtags are eliminated in case bert is set. (Default = simple)
- sample_size for downsampling full tscorpus which is 302936. (Default = 10)
Log each document's diff to console under document_id

ertugrul-dmr and others added 2 commits

January 28, 2021 16:40


          Added diff function for TsCorpus for error analysis

b5c5ff4


          Add tokenizer as option to diff command.

bce69a9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet