trl-training
Post-train LLMs with TRL (Transformers Reinforcement Learning) — SFT, DPO, GRPO, KTO, and reward-model training. Use when writing or debugging training code with the TRL Python API or the trl CLI.
Target: Software development & testing. Automatically collected · unverified.
Excerpt from the author’s documentation
Extracted from a fixed version of SKILL.md in its original language. This is the author’s documentation; installation, compatibility and runtime results have not been verified by this site.
Read the original English text
Topics in the source
- TRL
- Dataset formats
- SFT: the fields that matter
- GRPO: online RL
- CLI
Each method pairs a Trainer class with a Config dataclass. Configs extend transformers.TrainingArguments, so all of its arguments work in any trainer config. | Trainer | Dataset type | |---|---| | SFTTrainer | language modeling or prompt-completion | | DPOTrainer | preference (chosen/rejected pairs) | | GRPOTrainer | prompt-only + reward function(s) | | DistillationTrainer | prompt-only + a teacher model (on-policy distillation) | | KTOTrainer | unpaired preference (per-sample bool label) | | RewardTrainer | preference (chosen/rejected pairs); trains a scalar reward model, not a policy | Many more trainers (OnlineDPO, ORPO, CPO, GKD, …) live in trl.experimental with unstable APIs: https://huggingface.co/docs/trl/experimentaloverview Pass model as a string and route loading kwargs through modelinitkwargs (e.g. {"dtype": "bfloat16", "attnimplementation": "kernels-community/flash-attn2"}) instead of calling frompretrained yourself. The tokenizer/processor is inferred from the model; pass processingclass only when it differs. For LoRA, pass peftconfig=LoraConfig(...). Conversational: {"messages": [{"role": ..., "content": ...}]} (language modeling) or {"prompt": [...], "completion": [.
This is an excerpt. Refer to the complete source and its current requirements.
- Source repository
- huggingface/skills
- Source file
- skills/trl-training/SKILL.md
- Fixed revision
- 25910b3d0ba31a2ba102f901cdc9e008c278a69c
Sources and evidence
This record was collected automatically. Its documentation, suitability and runtime behavior have not been manually verified.
Author: huggingface/skills · original repository metadata. Documentation review: Not reviewed. Last source check: 2026-11-09T16:01:02.774Z.
Categories are inferred from names, descriptions and paths. They do not establish compatibility.
License and attribution
Apache-2.0. Third-party content and copyright remain with their authors. Refer to the original license and terms.
Original license ↗Synchronization, copying and user-reported success do not mark a resource as runtime-tested.
Limitations and notes
- Collecting metadata does not mean this site has installed, adapted, runtime-tested or security-reviewed the resource.