LoRA vs QLoRA in Llama Factory: Which Should You Use?

LoRA and QLoRA are the two methods most people use with Llama Factory. Both freeze the base model and train small adapter matrices. The difference is how the base model is stored while training.

LoRA in Llama Factory

With LoRA the base weights stay in 16-bit. Only the adapters receive gradients. Memory is roughly 2 GB per billion parameters, so a 7B model needs about 16 GB. Training is fast because there is no dequantization step.

finetuning_type: lora
lora_rank: 8
lora_alpha: 16
lora_target: all

QLoRA in Llama Factory

QLoRA loads the base model in 4-bit (or 8-bit) using bitsandbytes, then trains LoRA adapters on top. A 7B model fits in about 6 GB at 4-bit. The trade-off is a slower step time and a small quality cost that is usually acceptable.

finetuning_type: lora
quantization_bit: 4
quantization_method: bnb

Other quantization backends are supported too: AQLM, AWQ, GPTQ, HQQ and EETQ. Models already published in GPTQ or AWQ form can be loaded directly.

Which one to pick in Llama Factory

• Pick LoRA if the 16-bit model fits in memory. It trains faster and the result is a little better.
• Pick QLoRA if memory is tight or you want to train a larger model than your card normally allows.
• For experiments, start with QLoRA at 4-bit on a small model, then scale up once the data and config look right.

Tuning the adapter in Llama Factory

lora_rank sets the adapter size. 8 is a sensible default; 16 to 64 helps on harder tasks. lora_alpha is often set to twice the rank. lora_target: all applies adapters to every linear layer, which gives better results than attention-only in most reports.

Variants such as DoRA (use_dora: true), rsLoRA, LoRA+, PiSSA and LoftQ are available through flags in the same config.

After training in Llama Factory

Both methods produce an adapter folder. You can load it at inference with adapter_name_or_path or merge it into the base weights with the export command. Note that a QLoRA adapter should be merged onto the 16-bit base model, not the quantized one.