A run produces a score for the test as a whole, a score per success criterion, and a score and a transcript for every conversation it held. This article covers finding a run and reading what it says.
Building the test that produced the run is covered in Creating a Test. Turning what a run found into proposed changes is covered in AI Agents Optimization.
Finding a run
Open a test and select the Evaluations tab. Every run of that test is listed, newest first.
| Column | What it holds |
|---|---|
| Run Time | When the run started. This is the link through to the run's detail. |
| Duration | How long the whole run took. |
| Evaluated Conversations | How many conversations the run holds. While a run is in progress this reads as a running count against the total. |
| Success Rate | The run's overall score. |
| Status | Where the run is. |
A run moves through Queued, then Processing, and ends in Success, Error or Cancelled. Cancelling one puts it briefly in Cancelling while the conversations already under way finish.
A cancelled run is still worth opening. It keeps whatever it completed before it stopped, so a run configured for ten conversations may hold eight and score them normally.
Two things to know about the count. It counts conversations the run held, not conversations the judge managed to score, so it does not drop when an evaluation fails. And a run that produced no conversations at all has no detail page: its row carries no link, which is the signal that nothing was recorded rather than that the link is broken.
The run detail
The date link opens the run.
Success Criteria Rate is the run's overall performance as a single percentage. It is built from the criteria rather than from the conversations: each criterion's own pass rate is worked out first, and the overall figure is the average of those. That is why it does not always match the average of the conversation scores in the table below, and why one criterion that everything fails pulls the whole figure down no matter how well the rest went.
Token Consumption reports what the run cost, split into Input, Cached Input and Output. Simulated runs are not cheap, because every turn carries the whole conversation so far. A historical run of the same size costs a fraction of that, since the conversations are read rather than held.
Conversation summarises the run's shape: the number of conversations, the average number of agent messages and the average duration. AI Agent sits beside it with the per-session view of the same thing: input and output tokens per session, and the average response time.
Under the cards, each success criterion is drawn as its own bar with its pass rate. This is the part to read first. It says which criterion is failing, which is a different question from how the run scored, and it is what points at the change worth making.
The conversations
The table under the breakdown lists every conversation in the run.
| Column | What it holds |
|---|---|
| Score | The conversation's overall score, out of 10. |
| Duration | How long that conversation took. |
| Agent Messages | How many messages the agent sent in it. |
| One column per criterion | That conversation's score for that criterion, under the criterion's name. |
Hovering a score shows the judge's written reasoning for it, which is the fastest way to find out whether a low score is the agent's fault or the criterion's wording.
A cell reading Irrelevant is not a failure and not an error. It is the judge saying the criterion did not apply to that conversation, which happens honestly: a criterion about identifying the customer has nothing to judge in a conversation where nobody was asked to identify themselves. Irrelevant cells are left out of the arithmetic rather than counted as zero, so a run can read 100% with several of them in the table.
The transcript
Selecting a row opens the conversation beside the table.
The full exchange is shown as a chat, the agent on one side and the simulated customer on the other, with handovers between AI Agents and tool calls marked where they happened. A conversation that scores well overall but zero on one criterion is the one to open: the score tells you something went wrong, the transcript tells you where.
What a run does not tell you
Two habits to avoid.
Do not read the message count or the duration as quality. A simulated conversation tends to run to the message cap, because the two models keep exchanging closing pleasantries after the work is done. Conversations that all end at exactly the cap are normal and mean nothing on their own.
Do not read one run as a fixed number. The conversations are generated fresh each time and the scoring is a model's judgment, so the same test run twice gives a different figure. A change of a point or two between runs is noise. What is worth acting on is a criterion that fails consistently, or a figure that moves and stays moved across several runs.