Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace deleted elements at addition #418

Merged
merged 28 commits into from
Jan 12, 2023
Merged

Replace deleted elements at addition #418

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
19e727d
Replace deleted elements at insertion
dyashuni Sep 20, 2022
812b3a4
Refactoring
dyashuni Sep 20, 2022
c4bedcf
Refactoring
Nov 12, 2022
0f3214c
Add stress test to check multithreading
Nov 12, 2022
b440cbd
Merge branch 'develop' into replace_deleted
dyashuni Nov 12, 2022
34fe7f1
Fix possible multithreading issues
Nov 14, 2022
22cdb46
Refactoring
Nov 14, 2022
1f12fdb
Add C++ multi thread load test
Nov 26, 2022
270b237
Add timeout to jobs in actions
Nov 26, 2022
c2fb574
Add locks by label
Nov 26, 2022
c750df8
Remove previous element update locks as now we have locks by label
Nov 26, 2022
ef7e383
Refactoring
Nov 26, 2022
1741f50
Update addPointToVacantPlace
Nov 27, 2022
2ba0acc
Update load test
Nov 27, 2022
aabd0df
Addressing review comments
Dec 13, 2022
c26a45b
Update python 3.6 version in actions to meet Ubuntu 22.04
Dec 13, 2022
aaf5b5d
Remove python 3.6 tests as it is not available in Ubuntu 22.04
Dec 13, 2022
01bd9d0
Refactoring
Dec 17, 2022
50bac85
Fix code and tests
Dec 18, 2022
2711a61
Multithread test to check udate of deleted elements
Dec 18, 2022
a31bd4e
Refactoring
Dec 18, 2022
61ad825
Fix compile issues
Dec 18, 2022
d36fe80
Fix update of elements
Jan 7, 2023
a188fce
Update test params
Jan 7, 2023
5b67a38
Update CMakeLists.txt
dyashuni Jan 8, 2023
e4e97bf
Add test to check recall with replaced elements
Jan 8, 2023
646bfda
Refactoring of test
Jan 8, 2023
1be2fea
Update readme and refactoring
Jan 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Refactoring
  • Loading branch information
Dmitry Yashunin committed Nov 27, 2022
commit ef7e383dc230ad6410dc23083b0797f4fd570f32
2 changes: 1 addition & 1 deletion hnswlib/hnswalg.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ class HierarchicalNSW : public AlgorithmInterface<dist_t> {
static const unsigned char DELETE_MARK = 0x01;

size_t max_elements_{0};
size_t cur_element_count{0};
mutable std::atomic<size_t> cur_element_count{0}; // current number of elements
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it atomic, shouldn't it be locked all the times by external locks?

size_t size_data_per_element_{0};
size_t size_links_per_element_{0};
mutable std::atomic<size_t> num_deleted_{0}; // number of deleted elements
Expand Down
4 changes: 2 additions & 2 deletions python_bindings/bindings.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -473,7 +473,7 @@ class Index {
return py::dict(
"offset_level0"_a = appr_alg->offsetLevel0_,
"max_elements"_a = appr_alg->max_elements_,
"cur_element_count"_a = appr_alg->cur_element_count,
"cur_element_count"_a = (size_t)appr_alg->cur_element_count,
"size_data_per_element"_a = appr_alg->size_data_per_element_,
"label_offset"_a = appr_alg->label_offset_,
"offset_data"_a = appr_alg->offsetData_,
Expand Down Expand Up @@ -1006,7 +1006,7 @@ PYBIND11_PLUGIN(hnswlib) {
return index.index_inited ? index.appr_alg->max_elements_ : 0;
})
.def_property_readonly("element_count", [](const Index<float> & index) {
return index.index_inited ? index.appr_alg->cur_element_count : 0;
return index.index_inited ? (size_t)index.appr_alg->cur_element_count : 0;
})
.def_property_readonly("ef_construction", [](const Index<float> & index) {
return index.index_inited ? index.appr_alg->ef_construction_ : 0;
Expand Down