Technology

LLaMA-Factory

A WebUI + CLI fine-tuning framework (Apache-2.0, ~73.7k stars) covering 100+ LLMs/VLMs with full, LoRA/QLoRA, and preference-optimization training; latest release v0.9.5 (May 30, 2026).

7 connections 4 resources 1 post

Summary

What it is

A WebUI + CLI fine-tuning framework (Apache-2.0, ~73.7k stars) covering 100+ LLMs/VLMs with full, LoRA/QLoRA, and preference-optimization training; latest release v0.9.5 (May 30, 2026).

Where it fits

The broad-coverage, low-code entry point to fine-tuning on your own data: register a dataset in `dataset_info.json` (local file, HF Hub, ModelScope, custom script, or `cloud_file_name` pointing at S3/GCS object storage), pick a method, train from the LLaMA Board web UI, and serve via an OpenAI-compatible API. Its README's VRAM table puts a 7B model at roughly 60GB for full bf16 training, ~16GB for LoRA, ~6GB for 4-bit QLoRA — so consumer GPUs handle the PEFT path.

Misconceptions / Traps

It is not a competitor to Unsloth's kernels so much as a consumer of them — the README lists Unsloth and Liger Kernel as optional acceleration integrations inside LLaMA-Factory runs.
Datasets must match one of two schemas (Alpaca or ShareGPT/OpenAI); arbitrary JSON won't load without column mapping in dataset_info.json. Allowed file types are json, jsonl, csv, parquet, arrow.
The web UI is the headline feature, but production-scale paths (Megatron-core backend, Ray Trainer, Docker images for CUDA/ROCm/Ascend) are CLI/config territory.

Key Connections

implements Parameter-Efficient Fine-Tuning (LoRA / QLoRA) — LoRA/QLoRA (plus DoRA, OFT, PiSSA) are its primary consumer-GPU training modes
implements Direct Preference Optimization (DPO) — DPO is one of its built-in preference-training stages, alongside PPO/KTO/ORPO/SimPO
alternative_to Axolotl — both are config-driven wrappers over the HF ecosystem; LLaMA-Factory adds the web UI and wider VLM coverage
alternative_to torchtune — the code-first PyTorch-native path, now wound down, vs LLaMA-Factory's config/UI-first path
scoped_to Object Storage for AI Data Pipelines — cloud_file_name loads training data directly from S3/GCS

Definition

What it is

An open-source framework (Apache-2.0) for fine-tuning 100+ large language and vision-language models — Llama, Qwen, DeepSeek, Gemma, Mistral, GLM, Phi, and multimodal families — through either a Gradio web UI ("LLaMA Board") or a CLI. It unifies full fine-tuning, parameter-efficient methods (LoRA, QLoRA, DoRA, OFT, PiSSA), and preference/RL methods (DPO, PPO, KTO, ORPO, SimPO, reward modeling) behind one configuration surface, and was published at ACL 2024. The repo reports ~73.7k GitHub stars and remains actively developed.

Why it exists

It targets the engineer who wants to fine-tune on their own data without writing a training loop — pick a model, register a dataset, choose a method, train, then serve via an OpenAI-compatible API with vLLM or SGLang backends. The trade-off is a large configuration-driven surface rather than hackable code. The object-storage tie is real: `dataset_info.json` accepts a `cloud_file_name` field for datasets in S3/GCS cloud storage, alongside Hugging Face Hub, ModelScope, local files, and custom loading scripts.

Recent developments

Latest signals

v0.9.5 released (May 30, 2026). Latest stable release on PyPI, adding support for Qwen3.5/3.6 and Gemma 4 model families and compatibility with Transformers v5. Per PyPI and GitHub releases.
v0.9.4 (Dec 31, 2025). Added Orthogonal Fine-Tuning (OFT), Megatron-LM training support, and a KTransformers backend. Per GitHub releases.
Megatron-core backend integration (Oct 2025). The README's changelog notes Megatron-core-based training for large-scale runs, extending the framework beyond single-node accelerator setups. Per GitHub README.