emerging-techniques-model-pruning

Topics in the source

• Model Pruning: Compressing LLMs
• When to Use This Skill
• Installation
• Quick Start
• Wanda Pruning (One-Shot, No Retraining)
• SparseGPT (Second-Order Pruning)
• N:M Structured Pruning (Hardware Accelerator)
• Core Concepts

Use Model Pruning when you need to: Reduce model size by 40-60% with <1% accuracy loss Accelerate inference using hardware-friendly sparsity (2-4× speedup) Deploy on constrained hardware (mobile, edge devices) Compress without retraining using one-shot methods Enable efficient serving with reduced memory footprint Key Techniques: Wanda (weights × activations), SparseGPT (second-order), structured pruning, N:M sparsity Papers: Wanda ICLR 2024 (arXiv 2306.11695), SparseGPT (arXiv 2301.00774) Source: ICLR 2024 (arXiv 2306.11695) Source: arXiv 2301.00774 Magnitude Pruning (baseline): Wanda (weights × activations): SparseGPT (second-order): Unstructured (fine-grained): Prune individual weights Higher quality (better accuracy) No hardware speedup (irregular sparsity) Structured (coarse-grained): Prune entire neurons, heads, or layers Lower quality (more accuracy loss) Hardware speedup (regular sparsity) Semi-structured (N:M): Best of both worlds 50% sparsity (2:4) → 2× speedup on NVIDIA GPUs Minimal accuracy loss Pruning methods at 50% sparsity (LLaMA-7B): | Method | Accuracy Loss | Speed | Memory | Retraining Needed | |--------|---------------|-------|---------|-------------------| | Ma

This is an excerpt. Refer to the complete source and its current requirements.