Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Numba jit-compilation for intersect1d loop in calculate etd #193

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

cisaacstern
Copy link
Contributor

Based on @atmorling's observations in wildlife-dynamics/ecoscope-workflows#45 (comment), made an attempt to optimize this loop. This code doesn't work yet, but just throwing it up as draft so others can see, and perhaps offer some insights!

Not our highest priority right now, but I was too curious to not try. 😄

@walljcg
Copy link
Collaborator

walljcg commented Jun 29, 2024

I wonder if JAX would help at all (when we go down the rabbit hole) :)

@cisaacstern
Copy link
Contributor Author

Via offline review from @alxmrs:

"If you have a slow Set intersection operation, consider using a bloom filter. JIT on an O(N) algorithm won’t help much; you need a better data structure. A skip list might be a good fit, but I think you’ll really enjoy reading up on and implementing a bloom filter."

Alex, hope it's okay to post that review here, wanted to make sure others on this thread benefitted from your insights!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants