---
title: "What is model distillation? Teacher and student LLMs"
description: "Distillation trains a small student model to imitate a larger teacher, so it is cheaper to run. How soft targets work and how distilled reasoning models are made."
url: https://broadrouter.com/glossary/distillation/
lang: en
updated: 2026-11-16
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Distillation (knowledge distillation)

Updated 2026-11-16

Distillation, or knowledge distillation, is a training method in which a smaller student model learns to reproduce the outputs of a larger teacher model, such as its probability distributions or generated answers, so the student keeps much of the teacher's quality at a lower cost to run.

Also called: knowledge distillation, model distillation, distilled model, teacher-student training

Big models are expensive to serve. Distillation moves what a big model knows into a small one.

## How it works

Hinton, Vinyals and Dean described the modern form in 2015. The teacher’s output probabilities become “soft targets” for the student. A plain label says only “this is a 7”. A teacher’s probabilities also say it looks a little like a 1 and not at all like a 4, and that extra signal helps the student learn.

To expose it, they raise the softmax [temperature](https://broadrouter.com/glossary/temperature/). The paper notes that a higher T “produces a softer probability distribution over classes”. Their MNIST results:

| Model | Test errors |
| --- | --- |
| Large teacher net | 67 |
| Small net, trained on labels only | 146 |
| Same small net, distilled at T = 20 | 74 |

With LLMs, distillation often uses generated text instead of probabilities. The teacher writes answers, and the student is [fine-tuned](https://broadrouter.com/glossary/fine-tuning/) on them. The DeepSeek-R1 paper reports that the reasoning patterns of large models can guide smaller ones. Its model card lists six distilled models from 1.5B to 70B parameters, built on Qwen2.5 and Llama bases and fine-tuned on reasoning data generated by DeepSeek-R1.

## Why it matters

-   **Cost and speed.** A smaller model needs less GPU memory and generates faster, which shows up in per-token prices and latency.
-   **Narrower skill.** A student is strong where it was trained and weaker elsewhere. Test it on your own tasks rather than trusting a teacher’s benchmark.
-   **Stacking.** Distillation pairs with [quantization](https://broadrouter.com/glossary/quantization/) and [LoRA](https://broadrouter.com/glossary/lora/) in self-hosted setups.
-   **Licensing.** The teacher’s licence or API terms decide whether you may train on its outputs.

## On BroadNet Inference

BroadNet’s [API reference](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-distillation&utm_content=body#models) does not say whether any of its models were distilled, so do not assume it. Price differences between models do show why small models matter: BroadNet Gemma 4 VL 26B lists $0.22 per 1M output tokens against $3.46 for GLM 5.3. Whether the smaller model is good enough is a question for your own eval. The post on the [best open model for coding agents](https://broadrouter.com/blog/best-open-model-for-coding-agents/) compares the options by task.

## Questions

### What is the difference between distillation and quantization?

Distillation trains a new, smaller model. Quantization keeps the same model and stores its weights at lower precision. Both make inference cheaper, and they can be combined.

### Is a distilled model as good as the teacher?

Usually not on every task, but it can come close on the tasks it was distilled for. In the MNIST test by Hinton, Vinyals and Dean, the distilled small net made 74 errors against the teacher's 67.

### Is distilling another provider's model allowed?

Read the teacher model's licence or API terms before you train on its outputs. They differ on this point, and some restrict it.

## See also

-   [Fine-tuning](https://broadrouter.com/glossary/fine-tuning/) (term)
-   [LoRA (low-rank adaptation)](https://broadrouter.com/glossary/lora/) (term)
-   [Quantization](https://broadrouter.com/glossary/quantization/) (term)
-   [Temperature](https://broadrouter.com/glossary/temperature/) (term)
-   [Open-weight model](https://broadrouter.com/glossary/open-weight-model/) (term)
-   [Why a reasoning model returns an empty answer](https://broadrouter.com/guides/reasoning-tokens-empty-response/) (guide)
-   [Open-weight vs closed AI models for business: cost, control, risk](https://broadrouter.com/blog/open-weight-vs-closed-models/) (article)
-   [Best open model for coding agents: pick by tools, context and cost](https://broadrouter.com/blog/best-open-model-for-coding-agents/) (article)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-distillation&utm_content=page-cta) ·

**Sources**

-   [Hinton, Vinyals and Dean (2015), Distilling the Knowledge in a Neural Network](https://arxiv.org/abs/1503.02531)
-   [DeepSeek-AI (2025), DeepSeek-R1 paper](https://arxiv.org/abs/2501.12948)
-   [DeepSeek-R1 model card on Hugging Face (distilled models)](https://huggingface.co/deepseek-ai/DeepSeek-R1)
