Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Patterns improvement #20

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Patterns improvement #20

wants to merge 2 commits into from

Conversation

pa-shk
Copy link

@pa-shk pa-shk commented Nov 5, 2023

I'd like to suggest 3 small improvements to existing patterns:

  1. In homophone_scorer add pattern "Both words have different sounds, but {answer} sounds more {like} {query}". Seems like it might be considered as a right answer, but I'm not absolutely sure
  2. In rhyming_scorer add pattern "{query} and {answer} are a rhyme"
  3. In rhyming_scorer apply a more strict rule for negative scoring. If the answer is not in predictions and there is no negation in the answer, the answer should be considered wrong with certainty 1. With the current rule, the majority of wrong answers have certainty 0

Copy link

@vvchernov vvchernov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @pa-shk! Could you present accuracy values for corresponding tasks before and after scorer update?

@pa-shk
Copy link
Author

pa-shk commented Nov 7, 2023

Yes, shure, though the improvement isn’t significant.

For the task rhyming_words old and new scores respectively:
0.6,47.8,47.6
0.6,48.9,47.6

For the task homophones old and new scores respectively:
40.0,36.2,59.5
40.0,36.5,59.5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants