mechanistic-interpretability-saelens

原文涵盖的主题

• SAELens: Sparse Autoencoders for Mechanistic Interpretability
• The Problem: Polysemanticity & Superposition
• When to Use SAELens
• Installation
• Core Concepts
• What SAEs Learn
• Key Validation (Anthropic Research)
• Workflow 1: Loading and Analyzing Pre-trained SAEs

SAELens is the primary library for training and analyzing Sparse Autoencoders (SAEs) - a technique for decomposing polysemantic neural network activations into sparse, interpretable features. Based on Anthropic's groundbreaking research on monosemanticity. GitHub: jbloomAus/SAELens (1,100+ stars) Individual neurons in neural networks are polysemantic - they activate in multiple, semantically distinct contexts. This happens because models use superposition to represent more features than they have neurons, making interpretability difficult. SAEs solve this by decomposing dense activations into sparse, monosemantic features - typically only a small number of features activate for any given input, and each feature corresponds to an interpretable concept. Use SAELens when you need to: Discover interpretable features in model activations Understand what concepts a model has learned Study superposition and feature geometry Perform feature-based steering or ablation Analyze safety-relevant features (deception, bias, harmful content) Consider alternatives when: You need basic activation analysis → Use TransformerLens directly You want causal intervention experiments → Use pyvene or Transfo

这里只展示节选，请以完整原文和当前要求为准。