one indexer for multi-thread query cause blocked?

[wenbotse]

I supply a query service with http, which is a multi-thread application. I found when two thread query at the same time , query() function will be blocked , is there any solution solve this problem or walk around this .

[yurymalkov]

Hi @wenbotse,
I am not sure what is the problem. Are you talking about c++ or python interface?

[wenbotse]

Hi @yurymalkov , I mean the python interface . multi threads use the same indexer.query it will be blocked

[wenbotse]

QPS for multi-thread is much smaller than single-thread query

[yurymalkov]

@wenbotse It should not lock anything during the search - GIL is released. However, there is always a python-glue overhead that locks GIL before and after the search - it may lead to significant performance loss in multi-thread regime. It should not be a problem if single query takes more than 1ms though.
If your search time is significantly less than 1ms consider using the batch mode.

[wenbotse]

@yurymalkov Thanks for your reply. In my application , the dim is 768, batch query is [70, 768]. It will cost 20~30ms when call query_knn / get_items(ids) (by python api with single thread) . Is it right ? but I didn't much cpu costs , although I use 'indexer.knn_query(his_items_vec, k=query_num, num_threads = 32)'

[wenbotse]

@yurymalkov the blocked function should be get_items(ids) ?

[yurymalkov]

@wenbotse Yep, get_items does block the GIL, but it should be very fast.