prompt-caching

Topics in the source

• Prompt Caching
• Quick Start
• When to Use
• Best Practices
• Layout
• Cache-Control
• Design for Caching
• Measurement

Cutting LLM cost and latency by caching repeated prompt prefixes. Chat with long system prompts or tool definitions Agents with large context that repeats across turns RAG apps where system instructions are stable High-volume endpoints where cost matters Put stable content first: system prompt, tools, static context Put dynamic content last: user turns, retrieved chunks Keep a stable prefix across all requests Minimum cacheable size varies by provider (e.g., 1024+ tokens) Mark cacheable segments with cache-control headers Set a TTL that matches your access pattern Don't cache secrets or per-user data Verify cache hits via usage metadata (cachedtokens) Keep the system prompt identical across calls Separate static tool definitions from dynamic input Reuse the same prefix for the whole conversation Batch ephemeral data at the end of the context Track cached vs uncached tokens in usage Measure latency improvement per request Monitor cache hit rate over time Compare cost before and after optimization 1. Identify the stable prefix in your prompts (system, tools, static). 2. Move dynamic content to the end of the context. 3. Mark the prefix cacheable with the provider's mechanism. 4. Veri

This is an excerpt. Refer to the complete source and its current requirements.