-
-
Notifications
You must be signed in to change notification settings - Fork 121
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
reduce len() calls #412
Merged
maxbachmann
merged 7 commits into
rapidfuzz:main
from
bvandercar-vt:bvandercar/reduce-len-calls
Jan 16, 2025
Merged
reduce len() calls #412
Changes from 2 commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
ba243a8
reduce len() calls
bvandercar-vt 0323bc8
Merge branch 'rapidfuzz:main' into bvandercar/reduce-len-calls
bvandercar-vt 04edb16
Merge branch 'main' into bvandercar/reduce-len-calls
bvandercar-vt a011807
style: revert bench folder changes
bvandercar-vt 74b248d
move len to try block
bvandercar-vt c8cfbca
refactor: move common code
bvandercar-vt 4f41086
implement change
bvandercar-vt File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -1208,6 +1208,7 @@ def extract(query, choices, *, scorer=WRatio, processor=None, limit=5, score_cut | |
cdef RF_Scorer* scorer_context = NULL | ||
cdef RF_ScorerFlags scorer_flags | ||
cdef int64_t c_limit | ||
cdef int64_t choices_len = <int64_t>len(choices) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This will crash when choices is a generator. So this would need to be handled inside the try/except There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That makes sense, nice catch, changed: 74b248d |
||
scorer_kwargs = scorer_kwargs.copy() if scorer_kwargs else {} | ||
|
||
setupPandas() | ||
|
@@ -1216,14 +1217,15 @@ def extract(query, choices, *, scorer=WRatio, processor=None, limit=5, score_cut | |
return [] | ||
|
||
try: | ||
if limit is None or limit > len(choices): | ||
limit = len(choices) | ||
if limit is None or limit > choices_len: | ||
limit = choices_len | ||
except TypeError: | ||
# handle generators. In Theory we could retrieve the length later on while | ||
# preprocessing the choices, but this is good enough for now | ||
choices = list(choices) | ||
if limit is None or limit > len(choices): | ||
limit = len(choices) | ||
choices_len = <int64_t>len(choices) | ||
if limit is None or limit > choices_len: | ||
limit = choices_len | ||
|
||
c_limit = limit | ||
if c_limit == 1: | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The changes in the bench directory aren't relevant, since they are outside the benchmarks. So readability is more important here. Really this part is super slow because of:
anyway
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That makes sense. Reverted changes to the
bench
folder.