Chronicle

Chronicle helps you grade and compare AI agents on the work they actually do. Agents run tasks in a world: a simulated system with tools, data, and a defined starting state. Worldsmith builds that world from your description, from a single service to a workflow across connected tools.

​

How grading works

1. Define success. Write a task and attach scorers that check its requirements. Code checks concrete outputs or recorded changes; an LLM judge assesses qualities such as clarity or completeness.
2. Run and score. Run tasks in fresh copies of their saved starting world. Scorers return a value from 0 to 1; set pass thresholds to determine whether each required check passed.
3. Compare and improve. An evaluation runs a suite of tasks. Compare agent versions on the same tasks and starting state, then inspect the recorded actions behind each score to understand failures.

​

Four things to know

​

A grading example

The quickstart’s duplicate-invoice task has two checks, each with a pass threshold of 1:

ENG-2 is in progress for order 7001; case 00001001 remains open.

The handoff check requires that exact text. Checking that the case actually stayed open requires a separate scorer; the starter does not include it. Tasks and scorers covers writing checks. Run an evaluation explains how to compare versions and follow scores to their evidence.

​

Ways to use Chronicle