> ## Documentation Index
> Fetch the complete documentation index at: https://docs.bananaflow.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Evals

> Write a short scripted chat, let your agent reply at the points you pick, and see if each reply passes.

An eval checks how your agent replies. You write a chat in order. At the points you mark, the real agent answers, and a judge grades that answer as pass or fail. Run your evals before you publish a change, so you catch a bad reply before a caller hears it.

> [!NOTE]
>
> Evals test text only. They show what the agent says and which tools it calls. They do not test how the agent sounds or how it takes turns. To review real calls after they end, use [QA](https://docs.bananaflow.ai/voice-agent/qa.md).

## A simple example

You run a restaurant agent. You want to be sure it offers a table for two when a caller asks.

1. Add a **User** turn: "Hi, can I book a table for two tonight?"
2. Add an **Assistant** turn and turn on **Checkpoint**.
3. Set the **Judge** to **AI** with a prompt such as: `Context: {{messages}}. Pass if the agent asks what time the caller wants. Fail if it does not. Respond only with pass or fail.`
4. Click **Run eval**.

The agent writes its own reply at the checkpoint. The judge reads it and returns **Passed** or **Failed**.

## Open the evals

* From an agent: open the menu in the agent editor header and click **Evals**.
* From the sidebar: open **Evals** and pick an agent. The list shows how many evals each agent has and how its last run went.

## Create an eval

1. Click **New eval**.
2. Fill in **Eval Name**. **Description** is optional.
3. Pick a **Start node** if the chat should begin somewhere other than the start of the call.
4. Under **Conversation Turns**, add the turns in order.

| Turn | What it is |
| - | - |
| **User** | What the caller says. |
| **Assistant** | What the agent says. Turn on **Checkpoint** to have the agent reply live and be graded. Without a checkpoint, the text you write is used as the agent's reply, and it is not graded. |
| **Tool Response** | What a tool returns. Paste JSON as text. |
| **System** | A note for the agent, such as the date. |

You can move a turn up or down, or delete it.

## Judges

Each checkpoint has a **Judge**:

| Judge | It passes when |
| - | - |
| **Exact** | The reply and tool calls match what you wrote exactly. |
| **Regex** | The pattern matches any part of the reply. |
| **AI** | An AI judge reads the chat and answers pass or fail. Your prompt must include `{{messages}}` for the whole chat or `{{messages[-1]}}` for the last reply. |

You can also list the tool calls you expect, with their arguments.

### More options on a checkpoint

* **Stop the eval if this fails** – the run ends at that checkpoint.
* **Replace the reply with** – the rest of the chat sees your text in place of the agent's reply. Leave it empty to keep what the agent said.
* **Replace the tool calls with** – the same, for tool calls.

## Run and read results

* Click **Run eval** to run one eval. Select several in the list and click **Run selected** to run them together.
* If your prompts use variables, set their values in **Test Runs** as a JSON object.
* Each run shows **Running**, **Passed** or **Failed**. Open a run to read the chat with each checkpoint marked. If a run stops early, it tells you why, for example "Stopped at a failed checkpoint" or "The agent took too long to reply".
* Use **Past runs** to look at earlier runs.

Evals use your account credit. If you are out of credit, a run will not start. See [What you are charged for](https://docs.bananaflow.ai/billing/charges-and-low-balance.md).

## Build evals with your AI assistant

Your assistant can write and run evals for you through the [MCP server](https://docs.bananaflow.ai/integrations/mcp.md), for example: "Write three evals for *Lead Qualifier*: a caller who shares a budget, a caller who refuses to, and a caller who asks for a human. Run them, and fix the agent where it fails."

## Related

> [card] [QA](https://docs.bananaflow.ai/voice-agent/qa.md)
>
> Review real calls after they end.

> [card] [Editing a Workflow](https://docs.bananaflow.ai/voice-agent/editing-a-workflow.md)
>
> Change the agent when an eval fails.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.