Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of language models (https://arxiv.org/abs/2211.09110). This framework is also used to evaluate text-to-image …

A framework for few-shot evaluation of language models.

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

A high-throughput and memory-efficient inference and serving engine for LLMs

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

An Apache 2.0 fork of HuggingFace's Large Language Model Text Generation Inference

Large-scale LLM inference engine

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]

Build, evaluate, understand, and fix LLM-based apps

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

Modeling, training, eval, and inference code for OLMo

Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.

Machine Learning Engineering Open Book

AI orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data…

Automate the analysis of GitHub repositories for LLMs with RepoToTextForLLMs. Fetch READMEs, structure, and non-binary files efficiently. Outputs include analysis prompts to aid in comprehensive re…

DSPy: The framework for programming—not prompting—language models

Wrapper to easily generate the chat template for Llama2

LlamaIndex is the leading framework for building LLM-powered agents over your data.

The simplest, fastest repository for training/finetuning medium-sized GPTs.

Easily use and train state of the art late-interaction retrieval methods (ColBERT) in any RAG pipeline. Designed for modularity and ease-of-use, backed by research.

Structured Text Generation

Private chat with local GPT with document, images, video, etc. 100% private, Apache 2.0. Supports oLLaMa, Mixtral, llama.cpp, and more. Demo: https://gpt.h2o.ai/ https://gpt-docs.h2o.ai/

LLM powered development for VSCode

SGLang is a fast serving framework for large language models and vision language models.

Commit0: Library Generation from Scratch

Code for Paper: Training Software Engineering Agents and Verifiers with SWE-Gym

aider is AI pair programming in your terminal