post-training-verl

Topics in the source

• verl: Volcano Engine Reinforcement Learning for LLMs
• When to Use verl
• Key Features
• Installation
• Quick Start: GRPO Training
• Core Architecture
• Workflow 1: Math Reasoning with GRPO
• Prerequisites Checklist

verl is a flexible, efficient, and production-ready RL training library for large language models from ByteDance's Seed team. It implements the HybridFlow framework (EuroSys 2025) and powers models like Doubao-1.5-pro achieving O1-level performance on math benchmarks. Choose verl when you need: Production-ready RL training at scale (tested up to 671B parameters) Flexibility to swap backends (FSDP ↔ Megatron-LM ↔ vLLM ↔ SGLang) Support for multiple RL algorithms (PPO, GRPO, RLOO, REINFORCE++, DAPO) Multi-turn rollout with tool calling for agentic workflows Vision-language model RL training Consider alternatives when: You need Megatron-native training → use slime or miles You want PyTorch-native abstractions with Monarch → use torchforge You only need simple SFT/DPO → use TRL or Axolotl Training backends: FSDP, FSDP2, Megatron-LM Rollout engines: vLLM, SGLang, HuggingFace Transformers Algorithms: PPO, GRPO, DAPO, RLOO, ReMax, REINFORCE++, SPIN, SPPO Models: Qwen-3, Llama-3.1, DeepSeek, Gemma-2 (0.5B to 671B) Advanced: LoRA RL, sequence parallelism, expert parallelism, multi-turn tools verl uses a HybridFlow programming model separating control flow from computation: -- Use this workf

This is an excerpt. Refer to the complete source and its current requirements.