Add Containerized Benchmarking Support for GuideLLM

[wangchen615]

Add Containerized Benchmarking Support for GuideLLM

Description

This PR adds containerization support for GuideLLM benchmarking, making it easier to run benchmarks in containerized environments. The changes include:

1. A new Dockerfile that:

   • Uses Python 3.11 slim as the base image
   • Installs necessary dependencies, including git and curl
   • Sets up a non-root user for security
   • Includes healthcheck functionality
   • Properly configures the benchmark script

2. A new run_benchmark.sh script that:

   • Provides a flexible way to run benchmarks with configurable parameters
   • Supports environment variable overrides for customization
   • Handles different output formats (json, yaml)
   • Includes proper error handling and logging

Testing

The container can be built and tested using:

docker build -t <container-image-repo>/guidellm-benchmark:latest .
docker run -e TARGET="http://localhost:8000" -e MODEL="neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w4a16" <container-image-repo>/guidellm-benchmark:latest

Configuration Options

The container supports the following environment variables:

• TARGET: The target endpoint for benchmarking (default: http://localhost:8000)
• MODEL: The model to benchmark (default: neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w4a16)
• RATE_TYPE: The rate type for benchmarking (default: sweep)
• DATA: The data configuration (default: prompt_tokens=256,output_tokens=128)
• MAX_REQUESTS: Maximum number of requests (default: 100)
• MAX_SECONDS: Maximum duration in seconds
• OUTPUT_PATH: Path for output files (default: /results/guidellm_benchmark_results)
• OUTPUT_FORMAT: Output format (json, yaml, or yml)

Security Considerations

• Uses a non-root user for running the benchmark
• Includes proper file permissions
• Implements healthcheck for container monitoring
• Follows container security best practices

Related Issues

• Closes Add Docker and Kubernetes Support for GuideLLM Benchmarking #119
• Closes [Nice to Have] Container builds #111

Checklist

• Added Dockerfile with proper security configurations
• Added run_benchmark.sh script with environment variable support
• Added documentation for configuration options
• Tested container build and run
• Followed security best practices

[sjmonson]

Could you put these files under a /build/ directory? Also why python 3.11 and not 3.13 since its the latest version we offer support for?

[markurtz]

Let some comments throughout. Additionally, I know we had a request to move it to build/, but this is a commonly named folder that python and potential future builds will use.

I've generally seen the parent folder be docker/ at the root and then different python versions could be either nested folders under that with a Dockerfile or multiple Dockerfiles a the root with extensions of the python version such as Dockerfile.py39

[markurtz]

NIT: can we remove the benchmark from this so we keep it flexible towards future goals of evals and things like that?

[sjmonson]

I think benchmark is a neutral term for both performance and evaluations.

[markurtz]

See note on the entrypoint for the Dockerfile, I'd like to see if we can remove this script fully or make it optional to package in

[sjmonson]

In latest version, the entrypoint will fall back to executing unwrapped guidellm if anything is set in CMD. The entrypoint is now mostly a stopgap until scenarios with env support land.

https://github.com/wangchen615/guidellm/blob/51c6508a1a117417a897d7b0907bad7d325cac0f/build/entrypoint.sh#L8-L11

[Copilot]

Pull Request Overview

This pull request adds containerized benchmarking support for GuideLLM by introducing a new Dockerfile (Containerfile) and an entrypoint script to run benchmarks with configurable parameters in a containerized environment.

• Added a container entrypoint shell script to parse GUIDELLM_ environment variables and build the benchmark command.
• Created a Containerfile that sets up a multi-stage build with a virtual environment, installs the GuideLLM package, and configures a non-root user with proper security settings.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
build/entrypoint.sh	Introduces the shell script to execute the benchmarking command with GUIDELLM_ parameters.
build/Containerfile	Provides the multi-stage Docker build configuration and security setup for the benchmark.

Comments suppressed due to low confidence (1)

build/Containerfile:1

• The Containerfile specifies Python 3.13 while the PR description mentions Python 3.11 slim. Consider aligning the Python version in the Dockerfile with the documentation to avoid confusion.

ARG PYTHON=3.13