Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/automate stop words list pg #294

Merged
merged 25 commits into from
Jan 14, 2025
Merged

Conversation

polomarcus
Copy link
Collaborator

@polomarcus polomarcus commented Dec 2, 2024

Goal

Create an auto updated stop words list saved in PG

  • ci/cd scaleway
  • create and init SQL table stop_word
  • fix some typo inside the dictionary
  • test : edge cases with seveveral ads words in the same plaintext
  • other PR Test: lighten the mediatree.json for faster test execution #309
  • other PR : mediatree service use stop_word table instead of hard coded list of stop words
  • doc: How to add / modify manually a context ?

text similarity - other PR

  • use this lib do not add almost duplicated context https://github.com/seatgeek/thefuzz - but i need to see how to compare the similar context (stop words) to the whole plaintext which is much longer to the context

Test

pytest -k stop_word

@polomarcus polomarcus changed the title WIP: Feat/automate stop words list pg Feat/automate stop words list pg Jan 14, 2025
@polomarcus polomarcus merged commit ae0355f into main Jan 14, 2025
8 checks passed
@polomarcus polomarcus deleted the feat/automate-stop-words-list-pg branch January 14, 2025 17:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant