observability-langsmith

Topics in the source

• LangSmith - LLM Observability Platform
• When to use LangSmith
• Quick start
• Installation
• Basic tracing with @traceable
• OpenAI wrapper (automatic tracing)
• Core concepts
• Runs and traces

Development platform for debugging, evaluating, and monitoring language models and AI applications. Use LangSmith when: Debugging LLM application issues (prompts, chains, agents) Evaluating model outputs systematically against datasets Monitoring production LLM systems Building regression testing for AI features Analyzing latency, token usage, and costs Collaborating on prompt engineering Key features: Tracing: Capture inputs, outputs, latency for all LLM calls Evaluation: Systematic testing with built-in and custom evaluators Datasets: Create test sets from production traces or manually Monitoring: Track metrics, errors, and costs in production Integrations: Works with OpenAI, Anthropic, LangChain, LlamaIndex Use alternatives instead: Weights & Biases: Deep learning experiment tracking, model training MLflow: General ML lifecycle, model registry focus Arize/WhyLabs: ML monitoring, data drift detection A run is a single execution unit (LLM call, chain, tool). Runs form hierarchical traces showing the full execution flow. Projects organize related runs. Set via environment or code: 1. Structured naming - Use consistent project/run naming conventions 2. Add metadata - Include version

This is an excerpt. Refer to the complete source and its current requirements.