m365-agent-evaluator

Topics in the source

• M365 Agent Evaluator
• Always use this CLI invocation
• Activation workflow
• Current dataset contract
• Public evaluator names
• Common commands
• Version and PATH safety
• File conventions

Use this skill to help users evaluate Microsoft 365 Copilot declarative agents with @microsoft/m365-copilot-eval. The skill designs schema-compatible eval datasets, runs the public preview CLI, analyzes results, and recommends targeted fixes. Default to Microsoft 365 Agents Toolkit (ATK) projects when detected, but do not hard-stop solely because the current directory is not ATK. The CLI can also evaluate deployed agents with an explicit M365AGENTID or --m365-agent-id. Do not recommend the old private aka.ms installer, global installs, bare runevals, bare npx runevals, --input, or --html. 1. Identify the user goal: setup, dataset authoring, running evals, analyzing results, or updating an existing eval suite. 2. Load only the reference needed for the current goal: references/workflow.md for the end-to-end operator workflow and CLI commands. references/azure-setup.md for prerequisites, env files, and secret handling. references/eval-templates.md when creating or editing eval datasets. references/pra-framework.md when deciding what scenarios to generate. references/result-analysis.md after JSON/CSV/HTML results exist. references/guardrails.md before writing files, handling secrets, c

This is an excerpt. Refer to the complete source and its current requirements.