Open-source evaluation toolkit of large vision-language models (LVLMs), support 160+ VLMs, 50+ benchmarks

A reading list of video generation

Cosmos is a world model development platform that consists of world foundation models, tokenizers and video processing pipeline to accelerate the development of Physical AI at Robotics & AV labs. C…

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

Bringing BERT into modernity via both architecture changes and scaling

Accelerating the development of large multimodal models (LMMs) with lmms-eval

HunyuanVideo: A Systematic Framework For Large Video Generation Model

Apollo is a family of LMMs designed for video understanding

A course on aligning smol models.

Code for BLT research paper

Learning Flow Fields in Attention for Controllable Person Image Generation

Normalized Transformer (nGPT)

Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.

Code of Pyramidal Flow Matching for Efficient Video Generative Modeling

Visual self-questioning for large vision-language assistant.

Qwen2-VL is the multimodal large language model series developed by Qwen team, Alibaba Cloud.

Reasoning in Large Language Models: Papers and Resources, including Chain-of-Thought and OpenAI Strawberry o1.

A collection of LLM papers, blogs, and projects, with a focus on OpenAI o1 and reasoning techniques.

Recipes for shrinking, optimizing, customizing cutting edge vision models. 💜

MobileLLM Optimizing Sub-billion Parameter Language Models for On-Device Use Cases. In ICML 2024.

Offcial code for the ECCV2024 paper "Self-Adapting Large Visual-Language Models to Edge Devices across Visual Modalities"

Get up and running with Llama 3, Mistral, Gemma, and other large language models.

🧑‍🏫 60 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gan…

Inspect: A framework for large language model evaluations

Implementation for MatMul-free LM.