AgeBridge
AgeBridge Guided Project · AdvancedAbout 12 hoursPro

Agent evaluation harness

Forty cases with an agreed right answer, a script that runs them against a build you already made, a pass rate, and a GitHub Action that repeats it on every change.

What you'll have at the end

A GitHub repository holding 40 test cases, a runner that calls your lead scorer for each one, a scoring script that prints a pass rate and names every failure, a workflow that runs on every push, and a results sheet a client can read.

What it proves

test-set designregression runscontinuous checks

Accounts to create

Claude CodeBuilds and edits code from the terminalDocsClaude APIThe model behind the steps; pay per useDocsFree offerMakeVisual scenarios, free plan with 1,000 operations a monthDocsFree offerGitHubWhere your code lives and deploys fromDocsGoogle SheetsYour database for the first buildsDocs

What it costs: Free apart from model usage: a full run costs a few cents, and Claude Code needs a Claude subscription or an API key with credit.

Have this ready

  • The lead-scoring build from the start of the ladder, still working
  • A terminal with Node 20 or newer, git, and Claude Code installed
  • A GitHub account where you may create a repository

How it fits together

cases.csv40 inputs with the band they should score inrun.tsone call per caseThe target's webhookthe build being testedresults.csvwhat actually came backscore.tsbands, plus Claude as a judge on the reasonsPass rateprinted, and scored.csv beside itGitHub Actionthe same run on every push
  1. cases.csv: 40 inputs with the band they should score in
  2. run.ts: one call per case
  3. The target's webhook: the build being tested
  4. results.csv: what actually came back
  5. score.ts: bands, plus Claude as a judge on the reasons
  6. Pass rate: printed, and scored.csv beside it
  7. GitHub Action: the same run on every push

The steps

  1. 1. Give the target a URL you can call

    The build you are testing needs one address that takes a lead and answers with JSON. Everything after this step is just calling that address 40 times.

  2. 2. A repository and forty cases

    The test set, which is the real work of this build. Forty leads with a band each: the range of scores you are willing to call correct, agreed before you see what the model does.

  3. 3. A runner that calls the target 40 times

    One command that turns cases.csv into results.csv. It makes no judgements at all: it only records what the target said and how long it took.

  4. 4. The score that decides pass or fail

    One number a client understands, plus a named reason for every failure. The band does most of the work; Claude is used only to judge whether the written reason says what it should.

  5. 5. The same run on every push

    The harness stops being something you remember to run. From here, every change to the repository runs all 40 cases and turns the commit red or green.

  6. 6. A sheet for the client, and a regression caught

    The two things that make this a deliverable: a results sheet somebody outside your terminal can read, and proof that the harness catches a change nobody meant to make.