← All use cases

Add evals to your AI application

Have ZeroTwo convert the behavior you expect into an automated eval suite.

Point ZeroTwo at your AI application, have it pinpoint the behavior worth evaluating, and let it add an eval suite you can run.

Intermediate
1h

Best for

  • AI apps with prompts, tools, or retrieval but no repeatable evals.
  • Teams wanting regression tests before changing a model, prompt, or agent.
  • Manual quality checks worth committing as cases.

Starter prompt

Add an eval suite to this app using [eval runner, e.g. Promptfoo]. Behavior to evaluate: [behavior]. Propose a plan first, change no production behavior until a baseline runs, and leave a local eval command.

Introduction

Evals test scenarios systematically so regressions surface before release. A runner executes them, and ZeroTwo helps write and maintain the cases.

How to use

Name the runner, the app path to exercise, and one concrete behavior. Expect reviewable output: a config file, an evals/ folder with cases, any adapter, and a command such as npm run evals. Turn on Plan mode (/plan-mode) if you want a read-only plan before any file is written.

Choose what to evaluate

Pick one user-visible promise, such as correctness, grounding, tool use, format and business rules, or migration safety. Draw cases from requirements, bugs, or sanitized support examples.

Ask for an eval plan

Ask ZeroTwo to inspect first and name the app path under test, seed cases, assertions, files, command, and required secrets. Question it if it targets the raw model instead of the real path.

Implement, run, and iterate

Implement the plan plainly, then run a baseline before changing behavior; tune brittle assertions but keep real failures visible.

evals/
  promptfooconfig.yaml
  tests/cases.yaml

Add cases as bugs appear and wire the command into CI.

Related use cases