Technology

Axolotl

Apache-2.0 YAML-driven LLM fine-tuning framework (v0.18.0, July 2026) covering LoRA/QLoRA, full fine-tuning, QAT, preference tuning (DPO/IPO/KTO/ORPO), and RL (GRPO/GDPO) with FSDP1/2, DeepSpeed, and expert-parallel distributed training.

12 connections 4 resources 1 post

Summary

What it is

Where it fits

The config-file end of the self-hosted fine-tuning spectrum: one YAML reused across preprocessing, training, evaluation, quantization, and inference, which suits reproducible multi-GPU jobs on hardware you control. It is also the framework in this group with documented native object-storage dataset loading — `s3://`, `gs://`, `abfs://`, `oci://` paths resolve directly in the dataset config.

Misconceptions / Traps

Cloud paths work for loading datasets, not for checkpoints — the checkpoint docs describe only local output_dir saving (with save_steps and a trigger-file dynamic_checkpoint mechanism); cloud platforms typically mount a bucket instead.
Remote-filesystem dataset support is flagged "currently experimental" in the docs.
No official minimum-VRAM table is published; the stated floor is an NVIDIA GPU (Ampere or newer for bf16/Flash Attention) or AMD GPU, Python >=3.11, PyTorch >=2.11.

Key Connections

scoped_to Object Storage for AI Data Pipelines — native S3/GCS/Azure/OCI dataset loading via storage_options
implements Parameter-Efficient Fine-Tuning (LoRA / QLoRA) — core adapter methods alongside full fine-tuning and QAT
implements Direct Preference Optimization (DPO) — DPO plus IPO/KTO/ORPO preference tuning
alternative_to Unsloth — breadth + multi-GPU YAML pipelines vs. Unsloth's single-GPU kernel-speed focus

Definition

What it is

Axolotl is an open-source (Apache 2.0) fine-tuning framework for large language models, driven by a single YAML configuration file that covers the full pipeline: dataset preprocessing, training, evaluation, quantization, and inference. It wraps the Hugging Face ecosystem and adds a wide post-training method matrix — LoRA/QLoRA adapters, full fine-tuning, quantization-aware training (int8/int4/FP8/NVFP4/MXFP4), preference tuning (DPO, IPO, KTO, ORPO), and RL methods (GRPO, GDPO) — across text, vision-language, and audio models (GPT-OSS, LLaMA, Mistral, Mixtral, Qwen2-VL, LLaVA, and others).

Why it exists

It serves engineers who want to fine-tune on data they control without writing training code: declare the run in YAML, keep the config in version control, reproduce it anywhere. Its distributed-training breadth (FSDP1/FSDP2, DeepSpeed, expert parallelism for MoE models) targets multi-GPU and cluster runs more than single consumer cards. The object-storage angle is real on the input side: datasets load directly from S3 (`s3://`), GCS (`gs://`), Azure (`abfs://`/`adl://`), and OCI (`oci://`) paths with credentials from standard environment chains — though the docs mark remote-filesystem support "currently experimental," and checkpoints are documented as local-`output_dir` only.

Recent developments

Latest signals

v0.18.0 released (July 17, 2026). Adds NVFP4 4-bit expert-LoRA for MoE kernels (ScatterMoE/SonicMoE), GLM-5.2 fine-tuning with 2D expert parallelism over FSDP2, and hidden-states activation offloading enabling full-parameter training at 128k-token context by offloading per-layer checkpoints to CPU. Per v0.18.0 release.
v0.17.0 (June 2026) added distributed MoE training via DeepSeek's DeepEP kernels, BitNet 1.58-bit full fine-tuning, and remote training against a Tinker/Hatchery API endpoint instead of local hardware. Per releases.
Cloud dataset loading is documented for S3, GCS, Azure, and OCI via path-protocol prefixes and storage_options, with credentials from env vars, ~/.aws/credentials, or instance metadata; the docs label remote filesystems "currently experimental." Per dataset loading docs.