eval

Topics in the source

• /agenthub:eval — Evaluate Agent Results
• Usage
• What It Does
• Metric Mode (eval command configured)
• LLM Judge Mode (no eval command, or --judge flag)
• Hybrid Mode
• After Eval

Rank all agent results for a session. Supports metric-based evaluation (run a command), LLM judge (compare diffs), or hybrid. Run the evaluation command in each agent's worktree: Output: For each agent: 1. Get the diff: git diff {basebranch}...{agentbranch} 2. Read the agent's result post from .agenthub/board/results/agent-{i}-result.md 3. Compare all diffs and rank by: Correctness — Does it solve the task? Simplicity — Fewer lines changed is better (when equal correctness) Quality — Clean execution, good structure, no regressions Present rankings with justification. Example LLM judge output for a content task: 1. Run metric evaluation first 2. If top agents are within 10% of each other, use LLM judge to break ties 3. Present both metric and qualitative rankings 1. Update session state: 2. Tell the user: Ranked results with winner highlighted Next step: /agenthub:merge to merge the winner Or /agenthub:merge {session-id} --agent {winner} to be explicit