Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update scorer for LMentry tasks #22

Open
vvchernov opened this issue Nov 7, 2023 · 1 comment
Open

Update scorer for LMentry tasks #22

vvchernov opened this issue Nov 7, 2023 · 1 comment
Assignees

Comments

@vvchernov
Copy link

vvchernov commented Nov 7, 2023

Get unchecked tasks from LMentry, analyze answers from model and correctness of their evaluation by scorer. Try to update the scorer if errors were found.

@pa-shk
Copy link

pa-shk commented Nov 8, 2023

Improved patterns for the tasks rhyming_words and homophones

For the task rhyming_words old and new scores respectively:
0.6,47.8,47.6
0.6,48.9,47.6

For the task homophones old and new scores respectively:
40.0,36.2,59.5
40.0,36.5,59.5

#20

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants