Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/develop'
Browse files Browse the repository at this point in the history
  • Loading branch information
Hussama Ismail committed Aug 19, 2020
2 parents f187668 + 4953564 commit 982091e
Show file tree
Hide file tree
Showing 25 changed files with 46 additions and 46 deletions.
4 changes: 2 additions & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,15 @@ jobs:
- JAVA_OPTS="-Xmx2048m -Xms512m"
- MAVEN_OPTS="$JAVA_OPTS"
script:
- clang++ -fPIC -std=c++11 -O3 -shared bindings.cpp -I hnswlib -o l hnswlib-jna/src/main/resources/libhnswlib-jna.so
- clang++ -fPIC -std=c++11 -O3 -shared bindings.cpp -I hnswlib -o l hnswlib-jna/src/main/resources/libhnswlib-jna-x86-64.so
- mvn test
####################################
- stage: "Unit tests on macos" #
####################################
os: osx
osx_image: xcode9.3
script:
- clang++ -std=c++11 -O3 -shared bindings.cpp -I hnswlib -o l hnswlib-jna/src/main/resources/libhnswlib-jna.dylib
- clang++ -std=c++11 -O3 -shared bindings.cpp -I hnswlib -o l hnswlib-jna/src/main/resources/libhnswlib-jna-x86-64.dylib
- mvn test
###################################################
- stage: "Unit tests on macos (no compilation)" #
Expand Down
22 changes: 11 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ This project contains a [JNA](https://github.com/java-native-access/jna) (Java N

### __Pre-Generated Shared Library__

The jar file includes some pre-generated libraries for _Windows_, _Debian Linux_ and _MacOS_ which should allow an easy integration and abstract all complexity related to compilation. In the case of operating system issues, a runtime exception will be thrown and the manual compilation will be advised.
The jar file includes some pre-generated libraries for _Windows_, _Debian Linux_ and _MacOS_ (x86-64) which should allow an easy integration and abstract all complexity related to compilation. In the case of operating system issues, a runtime exception will be thrown and the manual compilation will be advised.

__On Windows, the [Build Tools for Visual Studio 2019 (C++ build tools)](https://visualstudio.microsoft.com/downloads/#build-tools-for-visual-studio-2019) is required__.

Expand All @@ -20,23 +20,23 @@ Add the following dependency in your `pom.xml`:
<dependency>
<groupId>com.stepstone.search.hnswlib.jna</groupId>
<artifactId>hnswlib-jna</artifactId>
<version>1.2.1</version>
<version>1.2.2-SNAPSHOT</version>
</dependency>
```

For more information and implementation details, please check [hnswlib-jna-example](./hnswlib-jna-example/).

## __Manual Compilation (Whenever it is advised)__

This section includes more information about how to compile the shared libraries on Windows, Linux and Mac. __If you were able to run the example project on your PC, this section can be ignored.__
This section includes more information about how to compile the shared libraries on Windows, Linux and Mac for different architectures (e.g., `x86-64`, `aarch64`). __If you were able to run the example project on your PC, this section can be ignored.__

### __Compiling the Shared Library__

To generate the shared library required by this project, `binding.cpp` needs to be compiled using a C compiler (e.g., `clang` or `gcc`) with C++11 support, at least. The library can be generated with `clang` via:
```
clang++ -O3 -shared bindings.cpp -I hnswlib -o <project_folder>/lib/libhnswlib-jna.dylib
clang++ -O3 -shared bindings.cpp -I hnswlib -o <project_folder>/lib/libhnswlib-jna-x86-64.dylib
```
__Note:__ The shared library's name must be: **libhnswlib-jna** (`.dylib` is the extension for MacOS, for windows use `.dll`, and linux `.so`).
__Note:__ The shared library's name must be: __libhnswlib-jna-ARCH.EXT__ where `ARCH` is the canonical architecture name (e.g., `x86-64` for `amd64`, or `aarch64` for ARM64) and `EXT` is `dylib` for MacOS, for windows use `dll`, and linux `so`.

#### Instructions for Windows

Expand All @@ -46,9 +46,9 @@ __Note:__ The shared library's name must be: **libhnswlib-jna** (`.dylib` is the
2. Download and install [Build Tools for Visual Studio 2019 (or higher)](https://visualstudio.microsoft.com/downloads/#build-tools-for-visual-studio-2019);
3. Compile the bindings using `clang`:
```
clang++ -O3 -shared bindings.cpp -I hnswlib -o <project_folder>/lib/libhnswlib-jna.dll
clang++ -O3 -shared bindings.cpp -I hnswlib -o <project_folder>/lib/libhnswlib-jna-x86-64.dll
```
This procedure will generate the 3 necessary files: `libhnswlib-jna.dll`, `libhnswlib-jna.exp` and `libhnswlib-jna.lib`.
This procedure will generate the 3 necessary files: `libhnswlib-jna-x86-64.dll`, `libhnswlib-jna-x86-64.exp` and `libhnswlib-jna-x86-64.lib`.

##### Using MinGW64

Expand All @@ -58,18 +58,18 @@ This procedure will generate the 3 necessary files: `libhnswlib-jna.dll`, `libhn
4. Unpack the archive and include MinGW64's bin folder into your PATH as well;
5. Compile the bindings using `clang`:
```
clang++ -O3 -target x86_64-pc-windows-gnu -shared bindings.cpp -I hnswlib -o <project_folder>lib/libhnswlib-jna.dll -lpthread
clang++ -O3 -target x86_64-pc-windows-gnu -shared bindings.cpp -I hnswlib -o <project_folder>lib/libhnswlib-jna-x86-64.dll -lpthread
```
This procedure will generate `libhnswlib-jna.dll`.
This procedure will generate `libhnswlib-jna-x86-64.dll`.

#### Instructions for Linux

1. Download and install `clang` (older versions might trigger compilation issues, so it is better use a recent version);
2. Compile the bindings using `clang`:
```
clang++ -O3 -fPIC -shared -std=c++11 bindings.cpp -I hnswlib -o <project_folder>/lib/libhnswlib-jna.so
clang++ -O3 -fPIC -shared -std=c++11 bindings.cpp -I hnswlib -o <project_folder>/lib/libhnswlib-jna-x86-64.so
```
This procedure will generate `libhnswlib-jna.so`.
This procedure will generate `libhnswlib-jna-x86-64.so`.

### __Reading the Shared Library in Your Project__

Expand Down
1 change: 1 addition & 0 deletions hnswlib-jna-example/lib/libhnswlib-jna-x86-64.dll
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/* this is just a placeholder for the dynamic library for windows */
1 change: 1 addition & 0 deletions hnswlib-jna-example/lib/libhnswlib-jna-x86-64.dylib
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/* this is just a placeholder for the dynamic library for mac */
1 change: 1 addition & 0 deletions hnswlib-jna-example/lib/libhnswlib-jna-x86-64.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/* this is just a placeholder for the dynamic library for windows */
1 change: 1 addition & 0 deletions hnswlib-jna-example/lib/libhnswlib-jna-x86-64.lib
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/* this is just a placeholder for the dynamic library for windows */
1 change: 1 addition & 0 deletions hnswlib-jna-example/lib/libhnswlib-jna-x86-64.so
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/* this is just a placeholder for the dynamic library for linux */
1 change: 0 additions & 1 deletion hnswlib-jna-example/lib/libhnswlib-jna.dll

This file was deleted.

1 change: 0 additions & 1 deletion hnswlib-jna-example/lib/libhnswlib-jna.dylib

This file was deleted.

1 change: 0 additions & 1 deletion hnswlib-jna-example/lib/libhnswlib-jna.exp

This file was deleted.

1 change: 0 additions & 1 deletion hnswlib-jna-example/lib/libhnswlib-jna.lib

This file was deleted.

1 change: 0 additions & 1 deletion hnswlib-jna-example/lib/libhnswlib-jna.so

This file was deleted.

4 changes: 2 additions & 2 deletions hnswlib-jna-example/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<parent>
<groupId>com.stepstone.search.hnswlib.jna</groupId>
<artifactId>hnswlib-jna-parent</artifactId>
<version>1.2.1</version>
<version>1.2.2-SNAPSHOT</version>
<relativePath>..</relativePath>
</parent>

Expand All @@ -32,7 +32,7 @@
<dependency>
<groupId>com.stepstone.search.hnswlib.jna</groupId>
<artifactId>hnswlib-jna</artifactId>
<version>1.2.1</version>
<version>1.2.2-SNAPSHOT</version>
</dependency>
</dependencies>

Expand Down
6 changes: 3 additions & 3 deletions hnswlib-jna-legacy/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

<name>hnswlib-jna-legacy</name>
<artifactId>hnswlib-jna-legacy</artifactId>
<version>1.2.1</version>
<version>1.2.2-SNAPSHOT</version>

<licenses>
<license>
Expand All @@ -18,12 +18,12 @@
<parent>
<groupId>com.stepstone.search.hnswlib.jna</groupId>
<artifactId>hnswlib-jna-parent</artifactId>
<version>1.2.1</version>
<version>1.2.2-SNAPSHOT</version>
<relativePath>..</relativePath>
</parent>

<properties>
<hnswlib.jna.version>1.2.1</hnswlib.jna.version>
<hnswlib.jna.version>1.2.2-SNAPSHOT</hnswlib.jna.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
*/
public final class HnswlibFactory {

private static final String LIBRARY_NAME = "hnswlib-jna";
private static final String LIBRARY_NAME = "hnswlib-jna-" + Platform.ARCH;
private static final String JNA_LIBRARY_PATH_PROPERTY = "jna.library.path";

private static Hnswlib instance;
Expand Down Expand Up @@ -42,15 +42,15 @@ static synchronized Hnswlib getInstance() {
}

private static String getLibraryFileName(){
String library;
String extension;
if (Platform.isLinux()) {
library = "libhnswlib-jna.so";
extension = "so";
} else if (Platform.isWindows()) {
library = "libhnswlib-jna.dll";
extension = "dll";
} else {
library = "libhnswlib-jna.dylib";
extension = "dylib";
}
return library;
return String.format("libhnswlib-jna-%s.%s", Platform.ARCH, extension);
}

private static void copyPreGeneratedLibraryFiles(Path folder, String fileName) throws IOException {
Expand All @@ -65,8 +65,8 @@ private static void checkIfLibraryProvidedNeedsToBeLoadedIntoSO() throws IOExcep
Path libraryFolder = Files.createTempDirectory(LIBRARY_NAME);
copyPreGeneratedLibraryFiles(libraryFolder, getLibraryFileName());
if (Platform.isWindows()) {
copyPreGeneratedLibraryFiles(libraryFolder, "libhnswlib-jna.exp");
copyPreGeneratedLibraryFiles(libraryFolder, "libhnswlib-jna.libw");
copyPreGeneratedLibraryFiles(libraryFolder, "libhnswlib-jna-x86-64.exp");
copyPreGeneratedLibraryFiles(libraryFolder, "libhnswlib-jna-x86-64.libw");
}
System.setProperty(JNA_LIBRARY_PATH_PROPERTY, libraryFolder.toString());
libraryFolder.toFile().deleteOnExit();
Expand Down
4 changes: 2 additions & 2 deletions hnswlib-jna/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

<name>hnswlib-jna</name>
<artifactId>hnswlib-jna</artifactId>
<version>1.2.1</version>
<version>1.2.2-SNAPSHOT</version>

<licenses>
<license>
Expand All @@ -18,7 +18,7 @@
<parent>
<groupId>com.stepstone.search.hnswlib.jna</groupId>
<artifactId>hnswlib-jna-parent</artifactId>
<version>1.2.1</version>
<version>1.2.2-SNAPSHOT</version>
<relativePath>..</relativePath>
</parent>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
*/
final class HnswlibFactory {

private static final String LIBRARY_NAME = "hnswlib-jna";
private static final String LIBRARY_NAME = "hnswlib-jna-" + Platform.ARCH;
private static final String JNA_LIBRARY_PATH_PROPERTY = "jna.library.path";

private static Hnswlib instance;
Expand All @@ -27,7 +27,7 @@ private HnswlibFactory() {
*
* @return hnswlib JNA instance.
*/
synchronized static Hnswlib getInstance() {
static synchronized Hnswlib getInstance() {
if (instance == null) {
try {
checkIfLibraryProvidedNeedsToBeLoadedIntoSO();
Expand All @@ -42,15 +42,15 @@ synchronized static Hnswlib getInstance() {
}

private static String getLibraryFileName(){
String library;
String extension;
if (Platform.isLinux()) {
library = "libhnswlib-jna.so";
extension = "so";
} else if (Platform.isWindows()) {
library = "libhnswlib-jna.dll";
extension = "dll";
} else {
library = "libhnswlib-jna.dylib";
extension = "dylib";
}
return library;
return String.format("libhnswlib-jna-%s.%s", Platform.ARCH, extension);
}

private static void copyPreGeneratedLibraryFiles(Path folder, String fileName) throws IOException {
Expand All @@ -65,8 +65,8 @@ private static void checkIfLibraryProvidedNeedsToBeLoadedIntoSO() throws IOExcep
Path libraryFolder = Files.createTempDirectory(LIBRARY_NAME);
copyPreGeneratedLibraryFiles(libraryFolder, getLibraryFileName());
if (Platform.isWindows()) {
copyPreGeneratedLibraryFiles(libraryFolder, "libhnswlib-jna.exp");
copyPreGeneratedLibraryFiles(libraryFolder, "libhnswlib-jna.libw");
copyPreGeneratedLibraryFiles(libraryFolder, "libhnswlib-jna-x86-64.exp");
copyPreGeneratedLibraryFiles(libraryFolder, "libhnswlib-jna-x86-64.libw");
}
System.setProperty(JNA_LIBRARY_PATH_PROPERTY, libraryFolder.toString());
libraryFolder.toFile().deleteOnExit();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,9 +72,9 @@ public void initialize(int maxNumberOfElements) {
/**
* Initialize the index to be used.
*
* @param maxNumberOfElements;
* @param m;
* @param efConstruction;
* @param maxNumberOfElements ;
* @param m ;
* @param efConstruction ;
* @param randomSeed .
*
* @throws IndexAlreadyInitializedException when a index reference was initialized before.
Expand Down
Binary file not shown.
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
<name>hnswlib</name>
<groupId>com.stepstone.search.hnswlib.jna</groupId>
<artifactId>hnswlib-jna-parent</artifactId>
<version>1.2.1</version>
<version>1.2.2-SNAPSHOT</version>
<packaging>pom</packaging>

<description>This project contains a JNA (Java Native Access) implementation built on top of the native Hnswlib (Hierarchical Navigable Small World Graph) which offers a fast approximate nearest neighbor search. It includes some modifications and simplifications in order to provide Hnswlib features with native like performance to applications written in Java. Differently from the original Python implementation, the multi-thread support is not included in the bindings itself but it can be easily implemented on the Java side.</description>
Expand Down

0 comments on commit 982091e

Please sign in to comment.