torchtune
PyTorch's native LLM post-training library (BSD-3-Clause) — readable, modifiable recipes for full/LoRA/QLoRA SFT, DPO, PPO, GRPO, distillation, and QAT. No longer actively developed: wind-down announced July 15, 2025; final release v0.6.1 (April 7, 2025).
Summary
PyTorch's native LLM post-training library (BSD-3-Clause) — readable, modifiable recipes for full/LoRA/QLoRA SFT, DPO, PPO, GRPO, distillation, and QAT. No longer actively developed: wind-down announced July 15, 2025; final release v0.6.1 (April 7, 2025).
It was the code-first pole of the fine-tuning tool spectrum — where Axolotl, LLaMA-Factory, and TRL wrap the Hugging Face stack in configs or Trainer classes, torchtune shipped plain PyTorch recipes with "minimal abstraction" that users forked and edited. Benchmarks in the README cover consumer (RTX 4090) through datacenter (A100, A6000) GPUs, with NVIDIA, Intel XPU, AMD ROCm, Apple MPS, and Ascend NPU support. Datasets come from HF Hub, local files, or remote URLs into instruct/chat/preference/text-completion/multimodal pipelines.
- It is not merely "less active" — maintenance formally ended, and the promised critical-fix window (through 2025) has expired as of 2026. Pinning it in a new stack means adopting unmaintained code.
- torchforge is the spiritual successor but not a drop-in one: it targets RL post-training at cluster scale (announcement demos on 512 H100s), not torchtune's single-device LoRA/QLoRA niche.
- No native object-storage/S3 dataset or checkpoint path is documented — remote loading means HF Hub or plain URLs.
alternative_toHugging Face TRL — the still-maintained library covering the same SFT/DPO/PPO/GRPO ground on the HF stackalternative_toAxolotl / LLaMA-Factory — config-driven counterparts that remain in active developmentimplementsParameter-Efficient Fine-Tuning (LoRA / QLoRA) — LoRA/QLoRA recipes, including multimodal QLoRA and QAT+LoRAimplementsDirect Preference Optimization (DPO) — including distributed DPO from v0.6.0scoped_toAI Runtime Infrastructure — training-side runtime tooling for models you host
Definition
PyTorch's native post-training library (BSD-3-Clause, ~5.8k stars): hackable Python recipes for supervised fine-tuning (full, LoRA, QLoRA), knowledge distillation, DPO, PPO, GRPO, and quantization-aware training, written directly in PyTorch rather than atop a Trainer abstraction. **Development status changed: the maintainers announced on July 15, 2025 that active development stopped, effective immediately.** The final release is v0.6.1 (April 7, 2025).
It served engineers who wanted fine-tuning code they could read and modify end-to-end — "hackable recipes, minimal abstraction," in the maintainers' own words — with memory optimizations (activation offloading, QAT, multimodal QLoRA) benchmarked on consumer hardware like the RTX 4090 as well as A100/A6000-class GPUs. Datasets load from Hugging Face Hub, local files, or remote URLs; the docs mention no S3/object-storage-native path, so any object-storage integration is the user's own plumbing. Its wind-down matters to anyone choosing a fine-tuning stack today.
Recent developments
- Development wound down (Jul 15, 2025). The maintainers stated "we are stopping active development on torchtune, effective immediately," with critical bug fixes and security patches promised only through 2025 — a window that has now lapsed. Per GitHub issue #2883.
- Final release v0.6.1 (Apr 7, 2025). Patch on v0.6.0 (Mar 25, 2025), which had brought tensor-parallel training, multinode support, and distributed DPO. Per PyPI.
- Successor direction: torchforge (announced Oct 22, 2025). Meta's PyTorch team launched torchforge, a PyTorch-native library for scalable RL post-training and agentic development, demonstrated on up to 512 H100s. The wind-down notice promised a new scale-first post-training product; the torchforge announcement itself does not mention torchtune, and torchforge targets large-cluster RL rather than single-GPU fine-tuning. Per PyTorch blog.
Connections 8
Outbound 6
scoped_to1alternative_to3Inbound 2
alternative_to2Resources 4
The official wind-down announcement — the single most decision-relevant fact about torchtune in 2026.
Primary repo (now under meta-pytorch): recipes, methods, hardware benchmarks, license.
Dataset pipeline reference — HF Hub/local/URL loading, five dataset types.
Where Meta's post-training effort went after torchtune — torchforge, RL-at-scale.