Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support DataLevel0BlocksMemory data struct #576

Closed
wants to merge 2 commits into from

Conversation

Axlgrep
Copy link

@Axlgrep Axlgrep commented Jul 11, 2024

support DataLevel0BlocksMemory data struct to solve the realloc huge memory doubling problem

  1. The current data_level0_memory_ continuous memory block design can result in memory doubling during large index resize, which is very unfriendly to memory and performance(the realloc operation may trigger an expensive data copy action),so we redesigned data_level0_memory_ (using multiple small memory blocks linked together).
  2. In addition, we found some obvious _mm_prefetch out-of-bounds access issues and fixed them (there may be other hidden issues, please check them as well).

@yurymalkov Please take a look at this MR, thank you...

@yurymalkov yurymalkov changed the base branch from master to develop July 21, 2024 18:31
@yurymalkov
Copy link
Member

Thank you for the PR!

  1. I wonder if you've tested the performance - that it does not affect search/build speed?
  2. Can you add a few tests (also check backward compatibility)?
  3. Can you update the branch? - the CI was broken

@Axlgrep Axlgrep force-pushed the optimize_memory_allocation branch from 1397a32 to f1cf3f6 Compare July 25, 2024 11:51
@Axlgrep Axlgrep closed this Jul 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants