Generic Gateway API Manager for Kubernetes
Caution
This project is still very unstable and not ready for use in production environments.
Kubvernor is a Rust implementation of Kubernetes Gateway APIs. The aim of the project is to be as generic as possible so Kubvernor could be used to manage/deploy different gateways (Envoy, Nginx, HAProxy, etc.)
-
Install Rust, Docker and Kind, Helm ...
-
Deploy your cluster
curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/refs/heads/main/hack/implementations/common/create-cluster.sh | bash- Install required CRDs for Gateway API
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.0/standard-install.yaml- Install Kubvernor CRDs
kubectl apply -f kubernetes/kubvernor-crds.yaml- Configure Kubvernor
kubectl apply -f kubernetes/kubvernor-config.yamlNote
4. (Optionally) install CRDs for Gateway API Inference Extension
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extension/releases/download/v1.1.0/manifests.yaml- Deploy Kubvernor
kubectl apply -f kubernetes/kubvernor-deployment.yaml- All is well if you see a pod in running state
kubectl get pod -n kubvernor- Deploy hello world... two gateways, two http routes, one backend
kubectl apply -f kubernetes/kubvernor-hello-world.yaml- Check that all is well
kubectl get gateway- Make some requests
curl -vki -H 'Host: service-one.com' http://GATEWAY_ADDRESS:1080/data- Using more or less the steps documented here
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api-inference-extension/refs/tags/v1.1.0/config/manifests/vllm/sim-deployment.yaml
helm install vllm-llama3-8b-instruct --set inferencePool.modelServers.matchLabels.app=vllm-llama3-8b-instruct --set inferencePool.image.pullPolicy=IfNotPresent --set inferenceExtension.image.pullPolicy=IfNotPresent --version v1.1.0 oci://registry.k8s.io/gateway-api-inference-extension/charts/inferencepool- Deploy Gateway API Inference Extension Routes
kubectl apply -f kubernetes/kubvernor-hello-inference-world.yaml- Make some requests
curl -vki http://GATEWAY_ADDRESS:1080/v1/chat/completions --header 'Host: www.inference-one.com' -H "Content-Type: application/json" -d '{"model":"meta-llama/Llama-3.1-8B-Instruct", "messages": [{"role":"user", "content":"What is the story?"}]}'kubectl delete -f kubernetes/kubvernor-hello-inference-world.yaml
kubectl delete -f kubernetes/kubvernor-hello-world.yaml
kubectl delete -f kubernetes/kubvernor-deployment.yaml
helm uninstall vllm-llama3-8b-instruct
kubectl delete -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api-inference-extension/refs/tags/v1.1.0/config/manifests/vllm/sim-deployment.yamlNote
- Run Gateway API Conformance suite
./run_conformance_tests.sh- Run Gateway API Inference Extension Conformance tests
git clone --depth 1 --branch v1.1.0 https://github.com/kubernetes-sigs/gateway-api-inference-extension.git
cd gateway-api-inference-extension
go test -v -count=1 -timeout=3h ./conformance --debug -run TestConformance --report-output="../kubvernor-inference-conformance-output.yaml" --organization=kubvernor --project=kubvernor --url=https://github.com/kubvernor/kubvernor --version=0.1.0 --allow-crds-mismatch