Architecture

Parameter-Efficient Fine-Tuning (LoRA / QLoRA)

Fine-tuning by training small low-rank matrices against a frozen base model (LoRA), optionally with the base quantized to 4-bit NF4 (QLoRA) — the technique that brings 7B–70B fine-tuning onto single owned GPUs.

13 connections 4 resources

Summary

What it is

Where it fits

It is the central method node for the fine-tune-own-data need: every major self-hosted trainer (TRL, Axolotl, Unsloth, LLaMA-Factory, torchtune) exposes it, and vLLM serves the resulting adapters many-at-a-time on one base model. The economics per Unsloth's docs: QLoRA needs ~5 GB VRAM at 7B, ~8.5 GB at 14B, ~26 GB at 32B, ~41 GB at 70B; 16-bit LoRA needs ~19 GB at 7B and ~164 GB at 70B. Adapters are small, portable artifacts — mergeable into the base (`merge_and_unload`), composable across tasks (TIES/DARE weighted merging), and cheap to store and version per-task.

Misconceptions / Traps

LoRA is not free-lunch equivalence with full fine-tuning: Biderman et al. (TMLR 2024) measured it learning less on target domains (code, math) at standard ranks — full fine-tuning's weight perturbations have 10–100x higher rank.
QLoRA's 4-bit quantization applies to the frozen base during training; the trained adapter itself is higher precision, and merged deployment quality depends on how you re-quantize.
An adapter is only portable to the exact base model (and revision) it was trained against; merging methods like cat sum adapter ranks and can OOM on large models (per PEFT docs).

Key Connections

scoped_to AI Runtime Infrastructure — the fine-tuning method layer of the self-hosted AI stack.
augments General-Purpose LLM — adapters specialize a frozen general base without touching its weights.
used_by Hugging Face TRL — PEFT is the adapter substrate TRL trains through (SFT/DPO with LoRA configs).
used_by Axolotl — config-driven LoRA/QLoRA fine-tuning wraps PEFT.
used_by vLLM — multi-LoRA serving hosts many adapters on one base model, hot-loadable at runtime.
solves Catastrophic Forgetting — partially: LoRA is documented (TMLR 2024) to forget less of source-domain capability than full fine-tuning, more than weight decay or dropout do.

Definition

What it is

LoRA (Low-Rank Adaptation, Hu et al., June 2021) fine-tunes a model by freezing the pretrained weights and injecting small trainable rank-decomposition matrices (a pair A·B of rank r) into each transformer layer — reducing trainable parameters by up to 10,000x versus full fine-tuning of GPT-3 175B, cutting GPU memory ~3x, and adding no inference latency once the matrices are merged back into the base weights. QLoRA (Dettmers et al., May 2023) extends this by quantizing the frozen base model to 4-bit NormalFloat (NF4), double-quantizing the quantization constants, and using paged optimizers — enough to fine-tune a 65B model on a single 48GB GPU while matching 16-bit fine-tuning quality. The reference implementation is Hugging Face's PEFT library (Apache-2.0, v0.20.0 as of July 2025).

Why it exists

It is the method that makes "fine-tune on my own data, on hardware I own" economically real: per Unsloth's requirements docs, QLoRA 4-bit fine-tuning needs roughly 5 GB VRAM for a 7B model, 8.5 GB for 14B, 26 GB for 32B, and 41 GB for 70B — versus 19 GB (7B) and 164 GB (70B) for 16-bit LoRA — putting 7B–32B in reach of a single consumer GPU. The trade-off is documented, not free: Biderman et al. (TMLR 2024) found LoRA underperforms full fine-tuning on the target domain at common ranks, while forgetting less of the base model's other capabilities. The output artifact is a small adapter rather than a full checkpoint, which makes per-task adapters cheap to store, version, and ship through an object-storage artifact lake, and lets one served base model host many adapters (vLLM multi-LoRA).

Recent developments

Latest signals

PEFT v0.20.0 shipped nine new adapter methods (July 28, 2025). HiRA, GLoRA, BEFT, MonteCLoRA, VeLoRA, Uni-LoRA, FRoD, MiCA, and DEFT, plus automatic LoRA target-module selection (KappaTuneSelector picks modules by weight-matrix condition number) and broader quantization support. Per PEFT releases.
Red Hat's Training Hub adopted Unsloth as a LoRA/QLoRA backend (April 1, 2026). The article reports roughly 70% less VRAM than full fine-tuning, ~2x faster training than standard LoRA pipelines, and 7B fine-tuning on a single 24GB GPU with 4-bit NF4 — an "algorithm-driven rather than framework-driven" packaging of the method. Per Red Hat Developer.
vLLM serves many LoRA adapters concurrently on one base model, with runtime hot-loading (current docs, fetched August 2026). --enable-lora plus --lora-modules registers adapters selected per-request via the model field; max_loras/max_lora_rank/max_cpu_loras bound concurrency, and /v1/load_lora_adapter / /v1/unload_lora_adapter endpoints (gated by VLLM_ALLOW_RUNTIME_LORA_UPDATING) swap adapters without restarting the server. Per vLLM docs.
PEFT documents adapter merging and composition as first-class operations (current docs, fetched August 2026). merge_and_unload() folds an adapter into the base weights for latency-free inference; add_weighted_adapter() combines multiple adapters with linear, svd, ties, and dare_ties methods; batched inference can mix adapters per-row via adapter_names. Per PEFT LoRA developer guide.