Skip to main content
An eval checks how your agent replies. You write a chat in order. At the points you mark, the real agent answers, and a judge grades that answer as pass or fail. Run your evals before you publish a change, so you catch a bad reply before a caller hears it.
Evals test text only. They show what the agent says and which tools it calls. They do not test how the agent sounds or how it takes turns. To review real calls after they end, use QA.

A simple example

You run a restaurant agent. You want to be sure it offers a table for two when a caller asks.
  1. Add a User turn: “Hi, can I book a table for two tonight?”
  2. Add an Assistant turn and turn on Checkpoint.
  3. Set the Judge to AI with a prompt such as: Context: {{messages}}. Pass if the agent asks what time the caller wants. Fail if it does not. Respond only with pass or fail.
  4. Click Run eval.
The agent writes its own reply at the checkpoint. The judge reads it and returns Passed or Failed.

Open the evals

  • From an agent: open the menu in the agent editor header and click Evals.
  • From the sidebar: open Evals and pick an agent. The list shows how many evals each agent has and how its last run went.

Create an eval

  1. Click New eval.
  2. Fill in Eval Name. Description is optional.
  3. Pick a Start node if the chat should begin somewhere other than the start of the call.
  4. Under Conversation Turns, add the turns in order.
You can move a turn up or down, or delete it.

Judges

Each checkpoint has a Judge: You can also list the tool calls you expect, with their arguments.

More options on a checkpoint

  • Stop the eval if this fails – the run ends at that checkpoint.
  • Replace the reply with – the rest of the chat sees your text in place of the agent’s reply. Leave it empty to keep what the agent said.
  • Replace the tool calls with – the same, for tool calls.

Run and read results

  • Click Run eval to run one eval. Select several in the list and click Run selected to run them together.
  • If your prompts use variables, set their values in Test Runs as a JSON object.
  • Each run shows Running, Passed or Failed. Open a run to read the chat with each checkpoint marked. If a run stops early, it tells you why, for example “Stopped at a failed checkpoint” or “The agent took too long to reply”.
  • Use Past runs to look at earlier runs.
Evals use your account credit. If you are out of credit, a run will not start. See What you are charged for.

Build evals with your AI assistant

Your assistant can write and run evals for you through the MCP server, for example: “Write three evals for Lead Qualifier: a caller who shares a budget, a caller who refuses to, and a caller who asks for a human. Run them, and fix the agent where it fails.”

QA

Review real calls after they end.

Editing a Workflow

Change the agent when an eval fails.