model-architecture-mamba

Topics in the source

• Mamba - Selective State Space Models
• Quick start
• Common workflows
• Workflow 1: Language model with Mamba-2
• Workflow 2: Use pretrained Mamba models
• Workflow 3: Mamba-1 vs Mamba-2
• Workflow 4: Benchmark vs Transformers
• When to use vs alternatives

Mamba is a state-space model architecture achieving O(n) linear complexity for sequence modeling. Installation: Prerequisites: Linux, NVIDIA GPU, PyTorch 1.12+, CUDA 11.6+ Basic usage (Mamba block): Complete LM with generation: Load from HuggingFace: Available models: state-spaces/mamba-130m state-spaces/mamba-370m state-spaces/mamba-790m state-spaces/mamba-1.4b state-spaces/mamba-2.8b Mamba-1 (smaller state): Mamba-2 (multi-head, larger state): Key differences: State size: Mamba-1 (dstate=16) vs Mamba-2 (dstate=128) Architecture: Mamba-2 has multi-head structure Normalization: Mamba-2 uses RMSNorm Distributed: Mamba-2 supports tensor parallelism Generation speed comparison: Expected results: Mamba: 5× faster inference Memory: No KV cache needed Scaling: Linear with sequence length Use Mamba when: Need long sequences (100K+ tokens) Want faster inference than Transformers Memory-constrained (no KV cache) Building streaming applications Linear scaling important Advantages: O(n) complexity: Linear vs quadratic 5× faster inference: No attention overhead No KV cache: Lower memory usage Million-token sequences: Hardware-efficient Streaming: Constant memory per token Use alternatives inst

This is an excerpt. Refer to the complete source and its current requirements.