post-training-torchforge

Topics in the source

• torchforge: PyTorch-Native Agentic RL Library
• When to Use torchforge
• Key Features
• Architecture Overview
• Installation
• ROCm Installation
• Quick Start
• SFT Training (2+ GPUs)

torchforge is Meta's PyTorch-native RL library that separates infrastructure concerns from algorithm concerns. It enables rapid RL research by letting you focus on algorithms while handling distributed training, inference, and weight sync automatically. Choose torchforge when you need: Clean separation between RL algorithms and infrastructure PyTorch-native abstractions (no Ray dependency) Easy algorithm experimentation (GRPO, DAPO, SAPO in 100 lines) Scalable training with Monarch actor system Integration with TorchTitan for model parallelism Consider alternatives when: You need production-ready stability → use miles or verl You want Megatron-native training → use slime torchforge is experimental and APIs may change Algorithm isolation: Implement RL algorithms without touching infrastructure Scalability: From single GPU to thousands via Monarch Modern stack: TorchTitan (training), vLLM (inference), TorchStore (sync) Loss functions: GRPO, DAPO, CISPO, GSPO, SAPO built-in -- Use this workflow for training reasoning models with group-relative advantages. [ ] 3+ GPUs (GPU0: trainer, GPU1: refmodel, GPU2: generator) [ ] Model from HuggingFace Hub [ ] Training dataset (GSM8K, MATH, etc.

This is an excerpt. Refer to the complete source and its current requirements.