Run locally
Run evaluation code locally to create an experiment in Braintrust and return summary metrics, including a direct link to your experiment. See Interpret results for how to read it.- TypeScript
- Python
- Go
- Ruby
- Java
- C#
Install the SDK and dependencies:Create the eval code:Run your evaluation with the Use Benefits of using the CLI:
bt eval CLI:--watch to re-run automatically when files change:- Automatic
.envloading — reads.env.development.local,.env.local,.env.development, and.env - Multi-file support — pass multiple files or directories:
bt eval [file or directory] .... Runningbt evalwith no arguments runs all eval files in the current directory. - TypeScript transpilation — no build step required; the CLI handles it
Configure experiments
Customize experiment behavior with options:maxConcurrency (max_concurrency in Python) is unset by default, so all of an eval’s tasks can run at once. Set it to avoid provider rate limit errors (429) and to limit memory and open connections on large datasets.
In the TypeScript, Python, and Ruby SDKs, you can set each inline eval case’s origin field to link it back to its source dataset row. This populates the dataset performance view, which only shows experiments whose eval traces set origin.
Next steps
- Interpret results from your experiments
- Compare experiments to measure improvements
- Advanced eval techniques like trials, hill climbing, attachments, and tracing
- Run experiments in the UI without writing code
- Run experiments in CI/CD to catch regressions automatically