llm-evaluation

Topics in the source

• LLM Evaluation
• Do not use this skill when
• Instructions
• Use this skill when
• Core Evaluation Types
• 1. Automated Metrics
• 2. Human Evaluation
• 3. LLM-as-Judge

Master comprehensive evaluation strategies for LLM applications, from automated metrics to human evaluation and A/B testing. The task is unrelated to llm evaluation You need a different domain or tool outside this scope Clarify goals, constraints, and required inputs. Apply relevant best practices and validate outcomes. Provide actionable steps and verification. If detailed examples are required, open resources/implementation-playbook.md. Measuring LLM application performance systematically Comparing different models or prompts Detecting performance regressions before deployment Validating improvements from prompt changes Building confidence in production systems Establishing baselines and tracking progress over time Debugging unexpected model behavior Fast, repeatable, scalable evaluation using computed scores. Text Generation: BLEU: N-gram overlap (translation) ROUGE: Recall-oriented (summarization) METEOR: Semantic similarity BERTScore: Embedding-based similarity Perplexity: Language model confidence Classification: Accuracy: Percentage correct Precision/Recall/F1: Class-specific performance Confusion Matrix: Error patterns AUC-ROC: Ranking quality Retrieval (RAG): MRR: Mean Rec

This is an excerpt. Refer to the complete source and its current requirements.