safety-alignment-nemo-guardrails

原文涵盖的主题

• NeMo Guardrails - Programmable Safety for LLMs
• Quick start
• Common workflows
• Workflow 1: Jailbreak detection
• Workflow 2: Self-check input/output
• Workflow 3: Fact-checking with retrieval
• Workflow 4: PII detection with Presidio
• Workflow 5: LlamaGuard integration

NeMo Guardrails adds programmable safety rails to LLM applications at runtime. Installation: Basic example (input validation): Detect prompt injection attempts: Validate both input and output: Verify factual claims: Filter sensitive information: Use Meta's moderation model: Use NeMo Guardrails when: Need runtime safety checks Want programmable safety rules Need multiple safety mechanisms (jailbreak, hallucination, PII) Building production LLM applications Need low-latency filtering (runs on T4) Safety mechanisms: Jailbreak detection: Pattern matching + LLM Self-check I/O: LLM-based validation Fact-checking: Retrieval + verification Hallucination detection: Consistency checking PII filtering: Presidio integration Toxicity detection: ActiveFence integration Use alternatives instead: LlamaGuard: Standalone moderation model OpenAI Moderation API: Simple API-based filtering Perspective API: Google's toxicity detection Constitutional AI: Training-time safety Issue: False positives blocking valid queries Adjust threshold: Issue: High latency from multiple checks Parallelize checks: Issue: Hallucination detection misses errors Use stronger verification: Colang 2.0 DSL: See references/colan

这里只展示节选，请以完整原文和当前要求为准。