LoRA made fine-tuning large models practical on ordinary hardware by training a tiny correction instead of the whole model.
How it works
Hu and colleagues observed that the change fine-tuning makes to a weight matrix tends to have a low rank. So LoRA keeps the original matrix W₀ frozen and learns the change as the product of two thin matrices:
W = W₀ + BA, where B is d × r, A is r × k, and r is much smaller than d or k.
B starts at zero, so training begins from the unchanged model. The paper mostly applied LoRA to the attention query and value projections.
The parameter savings are easy to compute. For one 4,096 × 4,096 matrix:
| Parameters | |
|---|---|
| Full update | 4,096 × 4,096 = 16,777,216 |
| LoRA, r = 8 | 8 × (4,096 + 4,096) = 65,536 |
That is about 0.4% of the full count. On GPT-3 175B the paper reports 10,000 times fewer trainable parameters, a 3× cut in GPU memory, and checkpoints shrinking from about 350 GB to about 35 MB.
Why it matters
- Cheaper training. Fewer trainable weights means less memory for optimiser state. Combined with quantization of the frozen base, larger models fit on smaller GPUs.
- No extra latency. Hugging Face’s PEFT guide notes adapters can be merged with
merge_and_unload(), so serving costs the same as the base model. - Many tasks, one base. Because the base stays frozen, you can keep one copy of an open-weight model and swap small adapters per customer or task.
- Limits. LoRA changes behaviour and style well. It is still fine-tuning, so it needs good data and an eval.
On BroadNet Inference
BroadNet’s API reference does not document LoRA, adapter uploads or per-request adapter selection. You call fixed model IDs such as broadnet/qwen3.8-27b. Private endpoints and BroadNet’s Arabic fine-tunes are available on request, but the docs do not say how they were trained. If you plan to train adapters yourself, compare that against API prices with the post on self-hosting vs an API, and if you run both, a router such as LiteLLM can send traffic to either.