-
Notifications
You must be signed in to change notification settings - Fork 55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor retrieval to make it faster to run in numba mode #47
Conversation
I wonder if it is possible to do invertex indexing here, by creating an array that tracks start and end: Lines 329 to 352 in daf29ce
|
…object is initiatilized with backend="numba"
In retrospective, it seems that |
Unfortuantely tqdm won't work, so we can't add progress bar to retrieve when backend is set to numba |
Will do that in a separate PR |
This a work in progress!
This PR will make numba mode faster by rewriting the entire retrieve process into a numba JIT-able function (see
_retrieve_internal_numba_parallel
)TODO:
retrieve_numba
to make it compatible withretrieve
whenBM25
object is initiatilized withbackend="numba"
selection_backend
inretrieve
so that it happens at the object init time_retrieve_internal_numba_parallel
_retrieve_internal_numba_parallel
selection
andnumba.selection
modules