QLoRA Explained: 4-bit NF4, Double Quantization and Paged Optimizers

QLoRA combines LoRA with a 4-bit quantized base model, which brings fine-tuning of very large models within reach of a single GPU. Here are the three ideas behind it and how to configure them.

By ForGen AI · Updated October 23, 2026

What is QLoRA?

QLoRA (Dettmers, Pagnoni, Holtzman and Zettlemoyer, 2023, arXiv:2305.14314) backpropagates through a frozen, 4-bit quantized base model into LoRA adapters. The base weights are stored in 4 bits and dequantized on the fly to a 16-bit compute type (usually bfloat16) for each matrix multiply. Gradients and optimizer state exist only for the adapters.

The three ideas behind QLoRA

1. 4-bit NormalFloat (NF4)

Pretrained weights are roughly normally distributed. NF4 is a 4-bit data type whose 16 levels sit at quantiles of a normal distribution, so each level is used about equally often. The paper shows it beats standard 4-bit integer and float formats for this kind of data.

2. Double quantization

Block-wise quantization stores one scaling constant per block of weights, and those constants take up memory too. Double quantization quantizes the constants themselves, saving about 0.37 bits per parameter, or roughly 3 GB on a 65B model.

3. Paged optimizers

Memory spikes on long sequences can cause out-of-memory crashes partway through training. Paged optimizers use NVIDIA unified memory to move optimizer state to CPU RAM when the GPU fills up, then page it back when needed.

Configuration with BitsAndBytes

import torch
from transformers import AutoModelForCausalLM, BitsAndBytesConfig

bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
)
model = AutoModelForCausalLM.from_pretrained(MODEL_ID, quantization_config=bnb)

# then attach LoRA adapters as usual, and train with
# optim="paged_adamw_8bit" in TrainingArguments

Approximate memory for the frozen base model

Activations, adapters and optimizer state come on top of these numbers and depend on batch size and sequence length.

FAQ

What is the difference between LoRA and QLoRA?

LoRA trains low-rank adapters on top of a frozen base model stored in 16-bit precision. QLoRA does the same, but stores the frozen base model in 4-bit NormalFloat (NF4), which cuts its memory footprint roughly 4x.

Does QLoRA reduce quality?

The QLoRA paper (Dettmers et al., 2023) reports that 4-bit QLoRA matches 16-bit full fine-tuning and 16-bit LoRA on the benchmarks it tested.

Can QLoRA run on a single GPU?

Yes. The QLoRA paper fine-tunes a 65B-parameter model on one 48 GB GPU. 7B–13B models fit comfortably on consumer GPUs.