AI Model Distillation, Explained

How a “teacher” model trains a tiny “student”, and why it wiped $589 billion off one company in a day

Interactive companion to the inphronesys.com explainer · Prices labelled early-2025 · Data: DeepSeek V3/R1 papers, Bloomberg, CNBC

447×
smaller than the teacher
1.5B student vs 671B R1
27×
cheaper output tokens
R1 $2.19 vs o1 $60.00 / M (early-2025)
$589B
wiped off Nvidia in one day
27 Jan 2025 · −17% · biggest ever
The story, in order
Click any point on the timeline for a plain-language recap and its source.
Tiny students, real scores
Model size (billions of parameters, log scale) vs AIME 2024 score
Selected student Other students o1-mini reference
R1-Distill-Qwen-1.5B
Distilled from DeepSeek-R1 (671B) · open weights
    The price shock that started the panic
    API price per million tokens, DeepSeek R1 vs OpenAI o1. Both reasoning models, early-2025 launch prices
    OpenAI o1 DeepSeek R1
    The same reasoning job cost about 27× less on R1, roughly 96% cheaper. That, more than any single benchmark, is what rattled the markets.
    What distillation actually is (and isn’t)
    Distillation is compression, not magic. A big, expensive “teacher” model trains a small, cheap “student” to imitate its answers. The student ends up nearly as good at a specific job at a fraction of the size and cost. The idea comes from a 2015 Google paper by Hinton, Vinyals & Dean. Think of a master chef writing down a recipe so a line cook reproduces the dish without 20 years of training.

    The bit the headlines blurred

    • Classic distillation needs the teacher’s internal “soft” outputs: you need access to the model’s guts.
    • The DeepSeek row was mostly about the cheaper cousin: training on another model’s API text outputs (“synthetic data”, or less kindly, “copying homework”). This is what allegedly breached OpenAI’s Terms of Service.
    • OpenAI and Microsoft said they found evidence and opened a probe; investor David Sacks cited “substantial evidence.” No public, independently verifiable proof was released, so this stays an allegation.

    The $5.6 million myth

    The famous $5.6M was real, but it was the electricity-and-rental bill for the final training run only (2.788M H800 GPU-hours × $2/hour = $5.576M). DeepSeek’s paper explicitly excludes prior research, failed experiments, data, salaries, and the chips themselves. Analysts (SemiAnalysis) put DeepSeek’s total server capital expenditure near $1.3 billion, a different thing entirely. It was never the price of building the company.

    The takeaway

    Distillation is a normal, legitimate, industry-wide technique: every “mini / flash / haiku” tier is effectively a distilled descendant of a flagship. The fight is about consent and contracts (whose outputs, against whose Terms of Service), not about whether the technique is fair game. For a business, the upshot is simple: capable models are getting radically cheaper, and that is mostly the distillation dividend.

    Sources

    Origin paper: Hinton, Vinyals & Dean, “Distilling the Knowledge in a Neural Network” (arXiv:1503.02531) · DeepSeek-V3 and R1 technical reports (671B/37B MoE; R1-Distill Table 5; $5.6M final-run figure) · Nvidia one-day loss: CNBC, Bloomberg · R1 pricing: gate.ai · $1.3B capex estimate: SemiAnalysis via Stratechery · Accusation & probe: TechCrunch, Fortune · Feb 2026 escalation: CNBC. Distilled checkpoints are released open-weight under their respective base-model (Qwen2.5 / Llama-3) licences; the R1 repo itself is MIT.