AutoEval documentation
AutoEval turns agent behaviors into versioned evaluations that block regressions in CI. It is an open-source, terminal-first tool for defining, running, and gating LLM evaluations — from your terminal, in CI, or from an agentic harness.
The full documentation lives in the open-source repository, where it stays in sync with every release.
Start here
- Quickstart — install, authenticate, and run your first evaluation in a few minutes.
- Command guide — every command in task order.
- CLI reference — exact options, human and JSON output, and exit codes per command.
- README and examples — browse the repository, examples, and contribution guide.