Only the Balio team sees this page.
Evals shows how well each agent design answers real past tickets, against what the company's own team replied, and what it costs. Nothing is sent to anyone: tests replay old tickets.
Where to find it
In the Balio sidebar, click Evals. The page has six tabs: Compare, Tests, Test sets, Review, Calibrate and Model choice.
The tabs
- Compare: pick a test set at the top. One sentence gives the verdict: the best test against today's setup. Score against cost draws each test as a dot. Higher is better, further left is cheaper. The table below lists each measure, the overall score, cost per ticket and speed.
- Tests: every test with its state, design, models, set and cost. Run a test picks a set, a design, a model for each role and a cost at which to stop.
- Test sets: each set of old tickets with its size, period and category mix. Build a new set makes one from a workspace's past tickets.
- Review: one answer at a time. Read the customer's message, the agent's reply and the team's reply, then press Good, Not good, Should have handed off or Skip.
- Calibrate: short cards that teach the AI judges what a good answer is.
- Model choice: for each step of a duty, whether a cheaper model held up on the old tickets. Apply makes the switch. It is never applied automatically.
What the labels mean
- Designs: One agent, Writer + checker, and Researcher + fact-checker + writer + checker.
- Today's setup: the design the agent runs on now. Other tests are compared with it.
- still learning: that measure's judge does not yet agree often enough with people's reviews. Its column is dimmed.
- These judges only inform: the workspace holds keys from one model family only, so a judge would grade its own family. Add a second provider key to fix it.
- within noise: the gap to today's setup is too small to be a real difference.
- "may know the future": a help article changed after the ticket was written, so the test may have an unfair advantage on it.
Good to know
- Every Balio seat can read Evals. Building sets, running tests, reviewing and applying a model choice need the operate right.
- Each test runs real model calls and stops at the cost you set.