post-training-miles

Topics in the source

• miles: Enterprise-Grade RL for Large-Scale Model Training
• When to Use miles
• Key Features
• Low-Precision Training
• Performance Optimizations
• Train-Inference Alignment
• Installation
• Quick Start

miles is a high-performance, enterprise-ready RL framework optimized for large-scale model post-training. Built as a production fork of slime, it addresses critical challenges in MoE training stability, low-precision training, and train-inference alignment. Choose miles when you need: Training 1TB+ MoE models (DeepSeek V3, Qwen3-MoE) FP8 or INT4 quantization-aware training Bit-wise identical train-inference alignment Speculative RL for maximum throughput Production stability with enterprise support Consider alternatives when: You want the research-grade original → use slime You need flexible backend swapping → use verl You want PyTorch-native abstractions → use torchforge Unified FP8: End-to-end FP8 for both inference and training INT4 QAT: 1TB models on single-machine VRAM (H200) Rollout Routing Replay (R3): Bit-wise expert alignment for MoE Speculative RL: 25%+ rollout speedup with online SFT draft models Zero-Copy Weight Sync: CUDA IPC zero-copy mapping Partial Rollout: Recycle half-finished trajectories TIS/MIS: Truncated/Masked Importance Sampling for off-policy correction Kernel-level optimization: FlashAttention-3, DeepGEMM integration miles inherits slime's configuration sy

This is an excerpt. Refer to the complete source and its current requirements.