Fine-Tuning on Your Own Data — the Decision Path

The question this guide answers: you have data you will not ship to a third party, hardware you control (or rent on your terms), and a model you want to specialize. What method, what framework, what data pipeline, and what do you have to protect against?

• Topics: Object Storage
• Technologies: Unsloth, Axolotl, LLaMA-Factory, Hugging Face TRL, torchtune, lakeFS, vLLM
• Standards: WebDataset
• Architectures: Parameter-Efficient Fine-Tuning (LoRA / QLoRA), Direct Preference Optimization (DPO), Synthetic Training Data Generation, Training Data Streaming from Object Storage, Checkpoint/Artifact Lake on Object Storage
• Pain Points: Catastrophic Forgetting, GPU Starvation, Data Loading Bottleneck
• Model Classes: DeepSeek-R1

1. Know what fine-tuning costs before you start: the model forgets. Fine-tuning measurably degrades capabilities outside the target domain — knowledge, reasoning, and reading comprehension all drift, and the effect was worse on larger models in the 1B–7B range studied (Luo et al.). The single most consequential method finding: LoRA learns less and forgets less (Biderman et al., TMLR 2024) — adapters underperform full fine-tuning on the target task at common ranks but preserve far more of the base model. If your use case tolerates "very good on my domain" in exchange for "still competent everywhere else," the adapter path is the default. See Catastrophic Forgetting.

2. Method: QLoRA is the on-your-own-hardware default. Per the Unsloth requirements table: 4-bit QLoRA fine-tunes a 7B model in ~5 GB VRAM, 14B in ~8.5 GB, 32B in ~26 GB, 70B in ~41 GB — versus 19 GB (7B) and 164 GB (70B) for 16-bit LoRA. One consumer GPU covers 7B–14B; one workstation card covers 32B; a single 48 GB card fine-tuned 65B in the original QLoRA paper. Full fine-tuning is for when you have cluster budget AND accept the forgetting bill from step 1. See Parameter-Efficient Fine-Tuning (LoRA / QLoRA).

3. Framework: pick by hardware shape, not by feature list.

   • One consumer GPU → Unsloth: custom Triton kernels, the published VRAM floor above, 500+ supported models. Multi-GPU is unofficial; the Studio UI is AGPL-3.0 while the core is Apache-2.0.
   • Multi-GPU / repeatable pipelines → Axolotl: YAML-config runs, FSDP1/2 + DeepSpeed, and experimental native dataset loading from S3/GCS/Azure — the only one of the four with a real object-storage ingest path in the config layer.
   • Broadest model coverage + a WebUI → LLaMA-Factory: LoRA/QLoRA/DoRA plus DPO/PPO/KTO/ORPO/SimPO, dataset_info.json can pull training files from S3/GCS, and its published VRAM table matches the QLoRA math above.
   • Library-level control → Hugging Face TRL: SFT/DPO/GRPO/KTO trainers you compose in Python; the other three build on pieces of it.
   • Do not start new work on torchtune — PyTorch wound its development down in mid-2025 (final release v0.6.1); it remains a readable reference, not a foundation.

4. Alignment on your own preference data: DPO, not RLHF infrastructure. Direct Preference Optimization trains directly on prompt / chosen / rejected pairs — no separate reward model, no PPO loop, nothing leaves your network. TRL ships it with 15 documented loss variants; GRPO (the DeepSeekMath-lineage method behind DeepSeek-R1) is the reasoning-flavored sibling when you can score outputs programmatically.

5. Generating training data with a teacher model: check the license BEFORE you distill. Teacher-model output terms differ materially: DeepSeek-R1 is MIT and explicitly permits distillation; Qwen3 is Apache-2.0; Llama 4's community license imposes naming/attribution conditions on derivatives; Gemma treats models trained on its synthetic output as Model Derivatives subject to its terms; and the Gemini API prohibits using outputs to train competing models. Tooling: distilabel (Apache-2.0) and NVIDIA NeMo Curator cover generation + curation. See Synthetic Training Data Generation.

6. Data pipeline: shard formats when you stream, simple files when you don't. A corpus that fits on local NVMe can stay JSONL/parquet. Streaming from object storage at training time is what WebDataset's tar-shard convention (pipe: URLs straight from S3) and MosaicML Streaming's MDS format exist for — sequential shard reads instead of per-sample requests, which is the difference between feeding the GPU and starving it. See Training Data Streaming from Object Storage.

7. Artifacts: adapters are small — version everything. A LoRA adapter is megabytes against a base model's gigabytes; keep per-task adapters in your checkpoint/artifact lake, version datasets with lakeFS-style branching, and serve many adapters off one base with vLLM's multi-LoRA runtime loading. The whole loop — data, shards, checkpoints, adapters — lives on storage you own.