add-runner-eval

Topics in the source

• Add a Runner Eval

Use this skill for the Runner Evals family: a real Runner/provider session against a seeded mock control plane. Product browser/server/database/Daytona coverage belongs in add-product-e2e-eval. Locate the Paperclip checkout using PAPERCLIPROOT when supplied, or git rev-parse --show-toplevel from a checkout. From outside Git, inspect the workspace roots (for example /paperclipai/paperclip) and verify that the selected root contains packages/paperclip-runner and tests/runner-e2e. Locate paperclip-evals using PAPERCLIPEVALSROOT or a discovered sibling; a worktree's parent directory need not contain that repository. Read doc/evals.md and packages/paperclip-runner/docs/runner-protocol-live-evals.md, then inspect the nearest existing case, roster, schema, and report test before editing. Definitions and authored cases belong in the sibling paperclip-evals/evals/paperclip-runner; Runner integration, aggregation, viewer, and publication behavior belongs in packages/paperclip-runner. Keep the control-plane boundary explicit in names and documentation. The sibling eval README is the concrete map: cases live under cases/, company fixtures under fixtures/, runtime/model settings under configs/,

This is an excerpt. Refer to the complete source and its current requirements.