An Open Source Machine Learning Framework for Everyone

LLM inference in C/C++

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

Port of OpenAI's Whisper model in C/C++

DeepSpeech is an open source embedded (offline, on-device) speech-to-text engine which can run in real time on devices ranging from a Raspberry Pi 4 to high power GPU servers.

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

Parsing gigabytes of JSON per second : used by Facebook/Meta Velox, the Node.js runtime, ClickHouse, WatermelonDB, Apache Doris, Milvus, StarRocks

MLX: An array framework for Apple silicon

Tensor library for machine learning

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.

A microbenchmark support library

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

A C++ standalone library for machine learning

Speech-to-text, text-to-speech, speaker diarization, and VAD using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Android, iOS, HarmonyOS, Raspberry Pi, RISC…

Header-only C++/python library for fast approximate nearest neighbors

Turbopilot is an open source large-language-model based code completion engine that runs locally on CPU

Intermediate Graphics Library (IGL) is a cross-platform library that commands the GPU. It provides a single low-level cross-platform interface on top of various graphics APIs (e.g. OpenGL, Metal an…

General purpose GPU compute framework built on Vulkan to support 1000s of cross vendor graphics cards (AMD, Qualcomm, NVIDIA & friends). Blazing fast, mobile-enabled, asynchronous and optimized for…

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

Large-scale LLM inference engine

FB (Facebook) + GEMM (General Matrix-Matrix Multiplication) - https://code.fb.com/ml-applications/fbgemm/

Real-time speech recognition and voice activity detection (VAD) using next-gen Kaldi with ncnn without Internet connection. Support iOS, Android, Linux, macOS, Windows, Raspberry Pi, VisionFive2, L…

Generalized Random Forests

The Fastest Deep Reinforcement Learning Library

BanditPAM C++ implementation and Python package

ggml implementation of BERT

Runtime Audio Importer plugin for Unreal Engine. Importing audio of various formats at runtime.

Pybind11 bindings for Whisper.cpp

Overlaybd: a block based remote image format. The storage backend of containerd/accelerated-container-image.

A stand-alone implementation of several NumPy dtype extensions used in machine learning.