ai-regression-testing

Topics in the source

• AI Regression Testing
• When to Activate
• The Core Problem
• Sandbox-Mode API Testing
• Setup (Vitest + Next.js App Router)
• Test Helper for Next.js API Routes
• Writing Regression Tests
• Testing Sandbox/Production Parity

Testing patterns specifically designed for AI-assisted development, where the same model writes code and reviews it — creating systematic blind spots that only automated tests can catch. AI agent (Claude Code, Cursor, Codex) has modified API routes or backend logic A bug was found and fixed — need to prevent re-introduction Project has a sandbox/mock mode that can be leveraged for DB-free testing Running /bug-check or similar review commands after code changes Multiple code paths exist (sandbox vs production, feature flags, etc.) When an AI writes code and then reviews its own work, it carries the same assumptions into both steps. This creates a predictable failure pattern: Real-world example (observed in production): The pattern: sandbox/production path inconsistency is the #1 AI-introduced regression. Most projects with AI-friendly architecture have a sandbox/mock mode. This is the key to fast, DB-free API testing. The key principle: write tests for bugs that were found, not for code that works. The most common AI regression: fixing production path but forgetting sandbox path (or vice versa). Frequency: Most common (observed in 3 out of 4 regressions) Test to catch it: Frequency:

This is an excerpt. Refer to the complete source and its current requirements.