LoRA vs QLoRA vs Full Fine-Tuning: Which Should You Use?

Full fine-tuning, LoRA and QLoRA differ mainly in what they train, what they keep in memory, and what you store afterwards. Here is how to choose between them.

By ForGen AI · Updated October 23, 2026

At a glance

GPU figures are rough guides for typical sequence lengths and batch sizes, not guarantees.

Choose full fine-tuning when…

• You need one model, not many, and have the hardware budget.
• The model must take in a large amount of new domain knowledge or a new language.
• You are building a base model that others will adapt later.

Choose LoRA when…

• You want task, style or format adaptation at a small fraction of the cost.
• You need several variants of the same model and want to swap them at serving time.
• You have enough GPU memory for the 16-bit base model. See the LoRA guide.

Choose QLoRA when…

• GPU memory is the constraint and you still want a large base model.
• You are fine-tuning on a single workstation or a single cloud GPU. See the QLoRA guide.

And when you need hundreds of adapters

Low-rank methods make each adapter cheap. At fleet scale, the remaining cost is running the base model over overlapping datasets again and again. Flora removes that duplication by running each shared prompt through the base model once. Read more in multi-adapter training.

FAQ

Is LoRA as good as full fine-tuning?

For most instruction, style and task-adaptation work, LoRA reaches quality comparable to full fine-tuning. Full fine-tuning can still win when the model must absorb large amounts of new knowledge.

Which method uses the least GPU memory?

QLoRA. It stores the frozen base model in 4-bit NF4 and trains only small adapters.

Which is best for many customer-specific models?

LoRA or QLoRA. Each adapter is a few megabytes, many adapters can be served from one base model, and shared-pass training can remove duplicated compute across adapters.