Lib.rs

arrow

Apache Arrow

arrow-cast

Cast kernel and utilities for Apache Arrow

arrow-schema

Defines the logical types for arrow arrays

parquet

Apache Parquet implementation in Rust

polars

DataFrame library based on Apache Arrow

arrow-buffer

Buffer abstractions for Apache Arrow

datafusion

in-memory query engine that uses Apache Arrow as the memory model

pyo3-arrow

Arrow integration for pyo3

lance

A columnar data format that is 100x faster than Parquet for random access

arrow-ipc

Support for the Arrow IPC format

arrow-select

Selection kernels for arrow arrays

vortex

file format with all builtin codecs and a sampling compressor

orc-rust

Apache ORC file format using Apache Arrow in-memory format

arrow-odbc

Read/Write Apache Arrow arrays from/to ODBC data sources

arrow-flight

Apache Arrow Flight

geoarrow-array

GeoArrow array definitions

arrow-array

Array abstractions for Apache Arrow

arrow-data

Array data abstractions for Apache Arrow

arrow-csv

Support for parsing CSV format to and from the Arrow format

datafusion-cli

Command Line Client for DataFusion query engine

ballista

Distributed Compute

arrow-json

Support for parsing JSON format to and from the Arrow format

arrow-digest

Stable hashes for Apache Arrow

datafusion-python

Apache DataFusion DataFrame and SQL Query Engine

arrow-ord

Ordering kernels for arrow arrays

polars-arrow-format

Unofficial flatbuffers and tonic code of Apache Arrow spec

narrow

Apache Arrow

arrow-arith

Arrow arithmetic kernels

datafusion-sqllogictest

DataFusion sqllogictest driver

lance-bitpacking

Vendored copy of https://github.com/spiraldb/fastlanes for use in Lance

sea-clickhouse

ClickHouse Client with SeaQL integration

arrow-string

String kernels for arrow arrays

datafusion-physical-expr

Physical expression implementation for DataFusion query engine

polars-arrow

Minimal implementation of the Arrow specification forked from arrow2

tsdb_timon

Efficient local storage and Amazon S3-compatible data synchronization for time-series data, leveraging Parquet for storage and DataFusion for querying, all wrapped in a simple and intuitive API

polars-parquet

Apache Parquet I/O operations for Polars

arrow-avro

Support for parsing Avro format into the Arrow format

pyo3-introspection

Introspect dynamic libraries built with PyO3 to get metadata about the exported Python types

ballista-scheduler

Ballista Distributed Compute - Scheduler

geodatafusion

Spatial extensions for Apache DataFusion

dora-record

dora goal is to be a low latency, composable, and distributed data flow

lance-datafusion

Internal utilities used by other lance modules to simplify working with datafusion

fsst

FSST string compression for Lance

lance-datagen

A columnar data format that is 100x faster than Parquet for random access

ballista-executor

Ballista Distributed Compute - Executor

arrow-row

Arrow row format

datafusion-physical-plan

Physical (ExecutionPlan) implementations for DataFusion query engine

connector_arrow

Load data from databases to Apache Arrow, the fastest way

ballista-cli

Command Line Client for Ballista distributed query engine

vortex-scalar

Vortex Scalars

lightstream

Composable, zero-copy Arrow IPC and native data streaming for Rust with SIMD-aligned I/O, async support, and memory-mapping

vortex-alp

Vortex ALP array

vortex-sparse

Vortex Sparse array

datafusion-substrait

DataFusion Substrait Producer and Consumer

ar_row

Row-oriented access to Arrow arrays

vortex-datetime-parts

Vortex physical encoding that compresses temporal components individually

geoarrow

amalgamation crate

xml2arrow

Efficiently convert XML data to Apache Arrow format for high-performance data processing

datafusion-physical-expr-common

Common functionality of physical expression for DataFusion query engine

lance-geo

Lance's geospatial extension providing geospatial UDFs

vortex-zigzag

Vortex zig zag array

vortex-datafusion

Apache Datafusion integration for Vortex

arrow-udf

User-defined function framework for arrow-rs

datafusion-catalog

arrow-integration-test

Support for the Apache Arrow JSON test data format

datafusion-datasource-parquet

vortex-io

Core async and blocking IO traits and implementations, used by IPC and file format

datafusion-physical-optimizer

DataFusion Physical Optimizer

lance-index

Lance indices implementation

lance-namespace-impls

Lance Namespace Implementations

stringtape

A tape class for strings arrays compatible with Apache Arrow

datafusion-datasource

datafusion-spark

DataFusion expressions that emulate Apache Spark's behavior

vortex-datetime-dtype

Vortex datetime extension dtype

expman

Core logic and storage engine for expman

datafusion-pruning

DataFusion Pruning Logic

rusqlite-gpkg

GeoPackage reader/writer built on top of rusqlite

datafusion-datasource-json

datafusion-catalog-listing

lance-encoding

Encoders and decoders for the Lance file format

minarrow

Apache Arrow-compatible, Rust-first columnar data library for high-performance computing, native streaming, and embedded workloads. Minimal dependencies, ultra-low-latency access, automatic 64-byte SIMD alignment…

lance-file

Lance file format

csv2arrow

Convert CSV files to Arrow

lance-linalg

A columnar data format that is 100x faster than Parquet for random access

transmcap

CLI for converting MCAP messages into JSONL, CSV, and Parquet via Arrow

datafusion-datasource-csv

geoarrow-expr-geo

GeoArrow

lance-table

Lance table format

ptars

Fast conversion from protobuf to Apache Arrow and back

lance-io

I/O utilities for Lance

flarrow-layout

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

datafusion-session

vortex-roaring

Vortex roaring bitmap arrays

datafusion-datasource-avro

riskless

A pure Rust implementation of Diskless Topics

lance-arrow

Arrow Extension for Lance

vortex-runend-bool

Vortex run end encoded boolean array, strictly better than runend for bool arrays

lance-namespace

Lance Namespace Core APIs

flarrow-builtins

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

arrow-format

Unofficial flatbuffers and tonic code of Apache Arrow spec

lance-jni

JNI bindings for Lance Columnar format

lance-testing

A columnar data format that is 100x faster than Parquet for random access

lance-encoding-datafusion

Encoders and decoders for the Lance file format that rely on datafusion

re_chunk_store

A storage engine for Rerun's Chunks

arrow_extendr

Enables the use of arrow-rs in R using extendr and nanoarrow

tpchgen-arrow

TPC-H data generator into Apache Arrow format

vortex-dict

Vortex dictionary array

flarrow-runtime

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

mcap2arrow-ros2-common

Shared ROS 2 type system and CDR decoding helpers for mcap2arrow

lance-examples

Lance examples in Rust

flarrow-api

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

arrow-udf-js

JavaScript runtime for Arrow UDFs

zarr-datafusion

Extending DataFusion to do SQL queries on Zarr data

mcap2arrow

MCAP to Arrow conversion library with protobuf and ROS 2 decoders

pgpq

Encode Apache Arrow RecordBatches to Postgres’ native binary format

polars-cli

CLI interface for running SQL queries with Polars as backend

perspective

A data visualization and analytics component, especially well-suited for large and/or streaming datasets

datafusion-datasource-arrow

liquid-cache-datafusion-client

10x lower latency for cloud-native DataFusion

geoparquet

reader and writer

json2arrow

Convert JSON files to Arrow

arrow-udf-python

Python runtime for Arrow UDFs

llkv-column-map

Column mapping utilities for the LLKV toolkit

adbc_datafusion

ADBC driver for Apache DataFusion

pyspark-arrow-rs

Derive macros to be used to add some helper functions to Rust structs to make them useable in Pyspark's mapInArrow

csv2parquet

Convert CSV files to Parquet

arrow-tools

packages

iceberg-file-catalog

Filesystem catalog for the unofficial Iceberg table format implementation

json2parquet

Convert JSON files to Parquet

spatialbench-arrow

SpatialBench data generator into Apache Arrow format

lance-namespace-datafusion

Lance namespace integration with Apache DataFusion catalogs and schemas

tpctools

generating and converting TPC-H and TPC-DS data sets

flarrow-message

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

dremio-rs

Dremio Rust client

geoarrow-cast

Functions for converting from one GeoArrow geometry type to another

strawboat

A native storage format based on Apache Arrow

liquid-cache-server

10x lower latency for cloud-native DataFusion

flarrow-url-scheme

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

flarrow-url

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

bodkin

Proc macro to simplify the integration of Arrow Data

liquid-cache-client

10x lower latency for cloud-native DataFusion

lance-tools

Tools for interacting with Lance files and tables

geoarrow-expr-geos

GeoArrow

lance-core

Lance Columnar Format -- Core Library

geoarrow-test

Test data for GeoArrow data

vortex-schema

Vortex file schema abstraction

llkv-threading

Thread pooling utilities for the LLKV toolkit

evolution-slicer

Data slicing components for evolution

mcap2arrow-ros2msg

ROS 2 .msg schema decoder for mcap2arrow CDR payloads

mcap2arrow-protobuf

Protobuf decoder for mcap2arrow using FileDescriptorSet schemas

flarrow-file-ext

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

foreign_vec

Unofficial implementation of Apache Arrow spec in safe Rust

geodatafusion-geoparquet

GeoParquet TableProvider for DataFusion

llkv-types

Common data types for the LLKV toolkit

evolution-parser

Data parsing functionality for evolution

convergence-arrow

Utils for bridging Apache Arrow and PostgreSQL's wire protocol

flarrow-flows

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

mcap2arrow-ros2idl

ROS 2 IDL schema decoder for mcap2arrow CDR payloads

minarrow-pyo3

PyO3 bindings for MinArrow - zero-copy Arrow interop with Python via PyArrow

geoarrow-csv

CSV reader and writer for GeoArrow

evolution-target

Output targets for evolution

adbc-driver-flightsql

ADBC FlightSQL driver native library distribution for Rust

llkv-compute

Compute kernels and math operations for LLKV

hyprstream

High-performance metrics storage and query service using Arrow Flight SQL

lance-arrow-scalar

Arrow scalar type with Ord, Hash, and Eq support

flarrow-url-default

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

mcap2arrow-arrow

Arrow conversion utilities built on top of mcap2arrow-core schemas and values

duckdb-server

DuckDB Server for Mosaic

geoarrow-flatgeobuf

Reader and writer for FlatGeobuf files to GeoArrow memory

parquet_derive

Derive macros for the Rust implementation of Apache Parquet

datafusion-row

Row backed by raw bytes for DataFusion query engine

evolution-common

Common util components of evolution

kapot

Distributed Compute

re_arrow_store

An in-memory time series database for Rerun log data, based on Apache Arrow

evolution-schema

Schema implementations for evolution

arrow-flightsql-odbc

An Apache Arrow Flight SQL server which proxies requests over ODBC

liquid-cache-datafusion-server

10x lower latency for cloud-native DataFusion

ballista-cache

Ballista Cache

llkv-scan

Shared scan interfaces and streaming helpers for the LLKV toolkit

ballista-core

Ballista Distributed Compute

re_data_store

An in-memory time series database for Rerun log data, based on Apache Arrow

datafusion-data-access

General data access layer currently mainly based on the object store interfaces

range-reader

Converts low-level APIs to read ranges of bytes to Read + Seek

geopolars

Geospatial extensions for Polars

arrowmax

High-performance Arrow data stack: columnar storage, zero-copy streaming, and schema codegen

polars_arrow_rvsry99dx

Apache Arrow

mongodb-arrow-connector

MongoDB connector that reads and writes data to/from Apache Arrow

mcap2arrow-core

Core schema and value types for decoding MCAP payloads into structured data

lance-test-macros

A columnar data format that is 100x faster than Parquet for random access

csv2pq

CSV to Apache parquet converter

gandiva_rust_udf

gandiva rust udfs

alloy-rs

Static Rust library for working with the Apache Arrow ffi using any C supported language

tiders-core

Core library for tiders blockchain data framework

re_arrow2

Unofficial implementation of Apache Arrow spec in safe Rust

flarrow-runtime-core

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

valu3-parquet

Parquet and Arrow encoding and decoding for valu3

vercel_blob

client for the Vercel Blob Storage API

tiders-rpc-client

A tiders RPC client for fetching EVM blockchain data from any standard JSON-RPC provider

katniss-pb2arrow

WIP

iron-arrow

Apache Arrow

narrow-derive

Apache Arrow