fine-tuning-axolotl

Topics in the source

• Axolotl Skill
• When to Use This Skill
• Quick Reference
• Common Patterns
• Example Code Patterns
• Reference Files
• Working with This Skill
• For Beginners

Comprehensive assistance with axolotl development, generated from official documentation. This skill should be triggered when: Working with axolotl Asking about axolotl features or APIs Implementing axolotl solutions Debugging axolotl code Learning axolotl best practices Pattern 1: To validate that acceptable data transfer speeds exist for your training job, running NCCL Tests can help pinpoint bottlenecks, for example: Pattern 2: Configure your model to use FSDP in the Axolotl yaml. For example: Pattern 3: The contextparallelsize should be a divisor of the total number of GPUs. For example: Pattern 4: For example: - With 8 GPUs and no sequence parallelism: 8 different batches processed per step - With 8 GPUs and contextparallelsize=4: Only 2 different batches processed per step (each split across 4 GPUs) - If your per-GPU microbatchsize is 2, the global batch size decreases from 16 to 4 Pattern 5: Setting savecompressed: true in your configuration enables saving models in a compressed format, which: - Reduces disk space usage by approximately 40% - Maintains compatibility with vLLM for accelerated inference - Maintains compatibility with llmcompressor for further optimization (exa

This is an excerpt. Refer to the complete source and its current requirements.