mechanistic-interpretability-transformer-lens

Topics in the source

• TransformerLens: Mechanistic Interpretability for Transformers
• When to Use TransformerLens
• Installation
• Core Concepts
• HookedTransformer
• Supported Models (50+)
• Activation Caching
• ActivationCache Keys

TransformerLens is the de facto standard library for mechanistic interpretability research on GPT-style language models. Created by Neel Nanda and maintained by Bryce Meyer, it provides clean interfaces to inspect and manipulate model internals via HookPoints on every activation. GitHub: TransformerLensOrg/TransformerLens (2,900+ stars) Use TransformerLens when you need to: Reverse-engineer algorithms learned during training Perform activation patching / causal tracing experiments Study attention patterns and information flow Analyze circuits (e.g., induction heads, IOI circuit) Cache and inspect intermediate activations Apply direct logit attribution Consider alternatives when: You need to work with non-transformer architectures → Use nnsight or pyvene You want to train/analyze Sparse Autoencoders → Use SAELens You need remote execution on massive models → Use nnsight with NDIF You want higher-level causal intervention abstractions → Use pyvene For development version: The main class that wraps transformer models with HookPoints on every activation: | Family | Models | |--------|--------| | GPT-2 | gpt2, gpt2-medium, gpt2-large, gpt2-xl | | LLaMA | llama-7b, llama-13b, llama-2-7b,

This is an excerpt. Refer to the complete source and its current requirements.