Files

.github
assets
benchmark
clients
docs
integration-tests
launcher
load_tests
proto
router
server
- custom_kernels
- exllama_kernels
- exllamav2_kernels
- tests
  - models
  - utils
  - conftest.py
- text_generation_server
- .gitignore
- Makefile
- Makefile-awq
- Makefile-eetq
- Makefile-flash-att
- Makefile-flash-att-v2
- Makefile-selective-scan
- Makefile-vllm
- README.md
- poetry.lock
- pyproject.toml
- requirements_cuda.txt
- requirements_rocm.txt
.dockerignore
.gitignore
.pre-commit-config.yaml
Cargo.lock
Cargo.toml
Dockerfile
Dockerfile_amd
LICENSE
Makefile
README.md
rust-toolchain.toml
sagemaker-entrypoint.sh
update_doc.py

tests

Feb 8, 2024

09b7c26 · Feb 8, 2024

This branch is 708 commits behind huggingface/text-generation-inference:main.

Name		Name	Last commit message	Last commit date
parent directory ..
models		models	feat: add more latency metrics in forward (huggingface#1346 )	Dec 14, 2023
utils		utils	feat(server): add frequency penalty (huggingface#1541 )	Feb 8, 2024
conftest.py		conftest.py	feat: support typical sampling (huggingface#114 )	Mar 9, 2023