Add evals to your AI application
Have ZeroTwo convert the behavior you expect into an automated eval suite.
Point ZeroTwo at your AI application, have it pinpoint the behavior worth evaluating, and let it add an eval suite you can run.
Related links
Best for
- AI apps with prompts, tools, or retrieval but no repeatable evals.
- Teams wanting regression tests before changing a model, prompt, or agent.
- Manual quality checks worth committing as cases.
Starter prompt
Introduction
Evals test scenarios systematically so regressions surface before release. A runner executes them, and ZeroTwo helps write and maintain the cases.
How to use
Name the runner, the app path to exercise, and one concrete behavior. Expect reviewable output: a config file, an evals/ folder with cases, any adapter, and a command such as npm run evals. Turn on Plan mode (/plan-mode) if you want a read-only plan before any file is written.
Choose what to evaluate
Pick one user-visible promise, such as correctness, grounding, tool use, format and business rules, or migration safety. Draw cases from requirements, bugs, or sanitized support examples.
Ask for an eval plan
Ask ZeroTwo to inspect first and name the app path under test, seed cases, assertions, files, command, and required secrets. Question it if it targets the raw model instead of the real path.
Implement, run, and iterate
Implement the plan plainly, then run a baseline before changing behavior; tune brittle assertions but keep real failures visible.
evals/ promptfooconfig.yaml tests/cases.yaml
Add cases as bugs appear and wire the command into CI.
Related use cases
Upgrade your API integration
Move your integration over to the newest supported models.
Remediate a vulnerability backlog
Convert findings that were already reviewed into small fixes backed by regression proof.
Fix a finding from your security scan
Pick a finding from your scan, inspect and apply a patch, then confirm the fix works.


