caio-review

Topics in the source

• /c-level-agents:caio-review — CAIO Forcing Questions
• When to Run
• The Six CAIO Questions
• 1. What does this AI need to be good at, and how would you measure it?
• 2. What's the SLO on hallucination / error rate, and what's the fallback?
• 3. What's the risk tier under EU AI Act, and is conformity assessment required?
• 4. API, fine-tune, or build?
• 5. What's the 12-month cost trajectory at expected scale?

Command: /c-level-agents:caio-review The eval-demanding CAIO pressure-tests any plan that involves AI. Six questions before any AI feature ships, any multi-year vendor commitment, or any AI team expansion. Before shipping any new AI-powered feature Before signing a multi-year AI vendor contract (API or self-hosted infra) Before EU launch of any AI feature Before a major AI team hire (especially ML engineer or research scientist) Before a fine-tuning project commitment Before adopting AI in a regulated domain (employment, credit, healthcare, education, etc.) When the founder uses the word "AI" near "competitive advantage" or "moat" No eval set = no ship. Before any AI feature deploys, define the eval criteria. 50-100 representative inputs minimum Expected outputs OR rubric for grading Edge cases: ambiguous, adversarial, format-edge If you can't write down what "good" looks like, you don't have a feature; you have a vibe. Every AI feature has a failure mode. Plan for it. Quantified SLO: "<5% hallucination on factual queries" Detection mechanism: monitoring, sampling, customer feedback loop Fallback: human-in-loop review, lower-risk default response, refuse-to-answer Blast radius if S

This is an excerpt. Refer to the complete source and its current requirements.