Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,15 @@ packages in that they are made to be:
the different PyTorch build configurations (various CUDA versions
and C++ ABIs). Furthermore, older C library versions must be supported.

## Components

- You can load kernels from the Hub using the [`kernels`](kernels/) Python package.
- If you are a kernel author, you can build your kernels with [kernel-builder](builder/).
- Hugging Face maintains a set of kernels in [kernels-community](https://huggingface.co/kernels-community).

## 🚀 Quick Start

Install the `kernels` package with `pip` (requires `torch>=2.5` and CUDA):
Install the `kernels` Python package with `pip` (requires `torch>=2.5` and CUDA):

```bash
pip install kernels
Expand Down
50 changes: 50 additions & 0 deletions kernels/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# kernels

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't put the logo image here, since it takes up a lot of valuable space on PyPI and we it's slow to load (as long as we don't have an SVG).

The Kernel Hub allows Python libraries and applications to load compute
kernels directly from the [Hub](https://hf.co/). To support this kind
of dynamic loading, Hub kernels differ from traditional Python kernel
packages in that they are made to be:

- Portable: a kernel can be loaded from paths outside `PYTHONPATH`.
- Unique: multiple versions of the same kernel can be loaded in the
same Python process.
- Compatible: kernels must support all recent versions of Python and
the different PyTorch build configurations (various CUDA versions
and C++ ABIs). Furthermore, older C library versions must be supported.

The `kernels` Python package is used to load kernels from the Hub.

## 🚀 Quick Start

Install the `kernels` package with `pip` (requires `torch>=2.5` and CUDA):

```bash
pip install kernels
```

Here is how you would use the [activation](https://huggingface.co/kernels-community/activation) kernels from the Hugging Face Hub:

```python
import torch

from kernels import get_kernel

# Download optimized kernels from the Hugging Face hub
activation = get_kernel("kernels-community/activation")

# Random tensor
x = torch.randn((10, 10), dtype=torch.float16, device="cuda")

# Run the kernel
y = torch.empty_like(x)
activation.gelu_fast(y, x)

print(y)
```

You can [search for kernels](https://huggingface.co/models?other=kernels) on
the Hub.

## 📚 Documentation

Read the [documentation of kernels](https://huggingface.co/docs/kernels/).
Loading