Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bloomfilter file compatibility between 32 and 64 bit architectures #55

Open
yunzheng opened this issue May 15, 2014 · 2 comments
Open

Comments

@yunzheng
Copy link

Currently the bloomfilter files generated on 32bit machines are not compatible on 64 bit machines, and vice versa.

One of the causes is the pointer size differences that is used in the bloomfilter file.
Is it possible to make it compatible between the two architectures?

@kleptog
Copy link

kleptog commented Jan 21, 2015

A minimal patch to do this is below. It does have the effect of being incompatible with previous versions, which may be an issue. If you want to be able to read old versions you need to be able to detect them first, which is tricky currently, but doable.

But, if we're going to change the file format, there are a number of other suggestions:

  • The "MBITARRAY" at the beginning of the file is nine bytes, so everything else is not aligned in memory. Suggest "MBITARRY" or something like that.
  • Move the version field to the front so it's always in the same place. Easier to make future changes.
diff --git a/src/bloomfilter.h b/src/bloomfilter.h
index 95defab..ba318b1 100644
--- a/src/bloomfilter.h
+++ b/src/bloomfilter.h
@@ -11,11 +11,13 @@ struct _BloomFilter {
     uint32_t num_hashes;
     uint32_t hash_seeds[256];
     /* All of the bit data is already in here. */
-    MBArray * array;
     unsigned char bf_version;
     unsigned char count_correct;
     uint64_t elem_count;
-    uint32_t reserved[32];
+    union {
+        uint32_t reserved[32];
+        MBArray * array;
+    };
 };

 typedef struct {

@AlanKK
Copy link

AlanKK commented Oct 3, 2016

What is the status of this fix? I could really use it. As it is now, I will have to choose another bloom filter package.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants