diff --git a/README.md b/README.md index ae223fc2..2e157f5a 100644 --- a/README.md +++ b/README.md @@ -23,9 +23,15 @@ packages in that they are made to be: the different PyTorch build configurations (various CUDA versions and C++ ABIs). Furthermore, older C library versions must be supported. +## Components + +- You can load kernels from the Hub using the [`kernels`](kernels/) Python package. +- If you are a kernel author, you can build your kernels with [kernel-builder](builder/). +- Hugging Face maintains a set of kernels in [kernels-community](https://huggingface.co/kernels-community). + ## 🚀 Quick Start -Install the `kernels` package with `pip` (requires `torch>=2.5` and CUDA): +Install the `kernels` Python package with `pip` (requires `torch>=2.5` and CUDA): ```bash pip install kernels diff --git a/kernels/README.md b/kernels/README.md new file mode 100644 index 00000000..b4034864 --- /dev/null +++ b/kernels/README.md @@ -0,0 +1,50 @@ +# kernels + +The Kernel Hub allows Python libraries and applications to load compute +kernels directly from the [Hub](https://hf.co/). To support this kind +of dynamic loading, Hub kernels differ from traditional Python kernel +packages in that they are made to be: + +- Portable: a kernel can be loaded from paths outside `PYTHONPATH`. +- Unique: multiple versions of the same kernel can be loaded in the + same Python process. +- Compatible: kernels must support all recent versions of Python and + the different PyTorch build configurations (various CUDA versions + and C++ ABIs). Furthermore, older C library versions must be supported. + +The `kernels` Python package is used to load kernels from the Hub. + +## 🚀 Quick Start + +Install the `kernels` package with `pip` (requires `torch>=2.5` and CUDA): + +```bash +pip install kernels +``` + +Here is how you would use the [activation](https://huggingface.co/kernels-community/activation) kernels from the Hugging Face Hub: + +```python +import torch + +from kernels import get_kernel + +# Download optimized kernels from the Hugging Face hub +activation = get_kernel("kernels-community/activation") + +# Random tensor +x = torch.randn((10, 10), dtype=torch.float16, device="cuda") + +# Run the kernel +y = torch.empty_like(x) +activation.gelu_fast(y, x) + +print(y) +``` + +You can [search for kernels](https://huggingface.co/models?other=kernels) on +the Hub. + +## 📚 Documentation + +Read the [documentation of kernels](https://huggingface.co/docs/kernels/).