Expand description
Nexus - Distributed LLM model serving orchestrator
This library provides the core functionality for managing and routing requests to heterogeneous LLM inference backends.
Modulesยง
- api
- Core API Gateway
- cli
- CLI module for Nexus
- config
- Configuration module for Nexus
- dashboard
- Web dashboard module for real-time monitoring
- discovery
- mDNS service discovery module
- health
- Health checking module for backend health monitoring.
- logging
- Structured logging module for request tracing
- metrics
- Metrics Collection Module
- registry
- Backend Registry module.
- routing
- Intelligent routing system for selecting optimal backends