DiskANN Disk Index Crate

This crate provides disk-based indexing capabilities for DiskANN.

Overview

The disk-index crate contains all the components specifically needed for building and searching disk-based indices:

Structure

src/
├── build/             # Disk index building pipeline
│   ├── builder/       # Core disk index builder and logic
│   ├── chunking/      # Checkpointing and continuation handling
│   └── configuration/ # Build parameters and quantization configuration
├── search/            # Disk index search infrastructure
│   ├── provider/      # Disk vertex providers and caching implementations
│   └── traits/        # Core traits for vertex providers and factories
├── data_model/        # Core data structures for disk indices
├── storage/           # Disk I/O operations and quantization
└── utils/             # Disk-specific utilities

Implementation Status

This crate has been populated with the core disk index functionality from the main diskann crate. The refactor is complete with the following modules implemented:

Build Module

• Builder: Core disk index builder, quantizer, and build operations
• Chunking: Checkpoint and continuation handling for large builds
• Configuration: Disk build parameters, filter parameters, and quantization types

Search Module

• Provider: Disk vertex providers, caching implementations, and factory patterns
• Traits: Core traits for vertex providers and provider factories

Data Model Module

• Graph headers, metadata, layout versioning, and caching structures

Storage Module

• Disk I/O operations with reader and writer APIs
• Quantization compression and generation utilities

Utils Module

• Disk-specific partitioning utilities

Dependencies

This crate depends on:

• diskann: Core types and utilities
• diskann-providers: Main DiskANN library (including storage abstractions)
• diskann-utils: Utility functions
• diskann-vector: Vector operations
• diskann-linalg: Linear algebra operations
• diskann-quantization: Vector quantization