The $589 billion Monday

On Monday, 27 January 2025, Nvidia lost $589 billion in market value. In one day. That’s the biggest one-day loss for any single company in US stock-market history, and it wasn’t caused by a factory fire or a fraud scandal.

It was caused by a chatbot.

A Chinese AI app had just climbed to number one on the US App Store, powered by a model that reportedly cost a rounding error to train. Wall Street looked at that number, looked at the hundreds of billions it had bet on expensive chips, and blinked. The word at the center of the whole panic was one most people had never heard: distillation.

Here’s the twist. Distillation isn’t a hack, a leak, or a Chinese invention. It’s a ten-year-old idea from a Google research lab. And by the time the dust settled, it turned out to be one of the most useful, most misunderstood, and most quietly universal techniques in modern AI. Let me walk you through what it actually is, what really happened, and why the number everyone quoted was wrong.

So what is distillation, really?

Picture a master chef with 20 years of training. She can taste a sauce and know it needs more acid. She’s brilliant, she’s slow, and she costs a fortune to keep on staff.

Now picture her writing down a recipe so precise that a line cook can reproduce the dish on a Tuesday night without any of the 20 years. The line cook won’t match her on every exotic dish. But on the ten plates the restaurant actually sells? Nobody can tell the difference.

That’s distillation. A big, expensive "teacher" model trains a small, cheap "student" model to imitate its answers. The student ends up nearly as good at a specific job, at a fraction of the size and cost. The idea comes from a 2015 paper by Geoffrey Hinton, Oriol Vinyals and Jeff Dean, titled "Distilling the Knowledge in a Neural Network." Hinton, for context, is one of the people who won a Nobel Prize for the foundations of this stuff. This is not a fringe technique.

The picture above is the whole idea in one frame. The big grey circle is the teacher. The tiny blue dot is the student. The student is 447 times smaller, measured in parameters (the adjustable dials a model tunes during training, and a rough proxy for its size), and it still scores 83.9% on a standard math exam. Compression, not magic.

The distinction every headline skipped

Now for the part that got lost in the noise. There are two different things hiding under the word "distillation," and the news story mostly concerned the second one.

Classic distillation needs the teacher’s internal "soft" outputs. Think of it as watching the chef’s hands, not just tasting the plate. You need real access to the model’s guts, which usually means you own the teacher.

Training on another model’s API text outputs is the cheaper cousin. You send a rival’s model thousands of questions, collect its answers, and train your own model on that pile of answers. Some people call this "synthetic data." Others call it "copying homework." You don’t need the teacher’s guts, just its front door.

These are not the same thing, and the difference matters for the whole controversy. Watching a chef you hired is fine. Photographing a competitor’s menu through the window and reselling their dishes is where the lawyers show up. The DeepSeek row was mostly about that second flavor, and specifically about whether it broke OpenAI’s Terms of Service.

The shock, in order

The timeline is worth slowing down on, because the panic compressed months of work into a single scary Monday.

DeepSeek-V3 arrived on 26 December 2024: a 671-billion-parameter model (with 37 billion active per token, for the technically curious) released with open weights, meaning anyone can download and run the model themselves. Then came DeepSeek-R1 on 20 January 2025, built to compete with the best. R1 is a reasoning model, one that works through a problem step by step before answering. By 27 January the app was number one on the US iOS store, and that same Monday Nvidia dropped roughly 17%, the $589 billion evaporating with it. Within days, OpenAI and Microsoft had opened a probe. A year later, in February 2026, Anthropic joined OpenAI in flagging what they called "industrial-scale" distillation campaigns, coordinating through the industry’s Frontier Model Forum.

What jumped out at me from this sequence is how little of it was about the technique and how much was about the price tag. Which brings us to the number.

About that $5.6 million

This is the single most misreported figure of the entire saga, so I’m going to be precise.

DeepSeek’s V3 technical report said the model’s final training run (what the paper calls its "official training") took 2,788,000 H800 GPU-hours. At an assumed rental rate of $2 per GPU-hour, that’s:

2,788,000 × $2 = $5,576,000, which rounds to about $5.6 million.

Here’s what the paper actually said versus how the media ran with it. The report was explicit that this figure covers the final training run only. It excludes prior research, the failed experiments, the architecture tinkering, the data, the salaries, and (this is the big one) the hardware itself. The GPUs were already owned. The headlines dropped all of that and reported "$5.6 million" as if it were the total price of building a frontier AI lab.

It wasn’t. Analysts at SemiAnalysis (via Stratechery) put DeepSeek’s total server capital expenditure at roughly $1.3 billion. That’s a completely different measure: the cost of owning thousands of chips, not the electricity bill for one run. Comparing the two as if they’re the same thing is how you end up with a wrong story that moves markets.

The honest one-liner: the $5.6 million was real, but it was the rental-and-electricity bill for the final training run, not the price of the company, the chips, or the years of research behind it.

The accusation

So did DeepSeek copy OpenAI’s homework? As of now, that’s an allegation, not a proven fact. Keep that straight, because the internet did not.

Around 28 to 29 January 2025, reports emerged that OpenAI and Microsoft were probing whether DeepSeek had improperly used OpenAI’s API. Microsoft researchers said they’d observed large amounts of data leaving through OpenAI’s API in the autumn of 2024 and had flagged it. David Sacks, the White House AI adviser, put it bluntly: "There’s substantial evidence that what DeepSeek did here is they distilled the knowledge out of OpenAI’s models." OpenAI said it found evidence of distillation and noted that "PRC-based companies, and others, are constantly trying to distill the models of leading U.S. AI companies."

Notice what’s missing from all of that. Public, independently verifiable proof. No court has found DeepSeek did this. Training a competing model on OpenAI’s outputs would breach OpenAI’s Terms of Service, yes. But "would breach if true" and "proven" are different sentences. The correct framing is "OpenAI says," "reportedly," "alleged."

The glass house

And now the irony, which is delicious enough to deserve its own paragraph.

The frontier labs accusing others of training on data they shouldn’t have are fighting that exact battle from the other side. On 27 December 2023, The New York Times sued Microsoft and OpenAI in the Southern District of New York. The core allegation: OpenAI trained its models on millions of Times articles without authorization, and the models can reproduce copyrighted text nearly verbatim. In 2025 a judge denied most of OpenAI’s motions to dismiss, and the majority of the claims are proceeding.

I’m not saying two wrongs make a right. I’m saying the entire industry is arguing about who trained on whose stuff without asking, and the moral high ground is more crowded than the press releases suggest. The provocative version: the DeepSeek fight was never about whether distillation is legitimate. It was about consent and contracts, and almost nobody in this story has clean hands on that question.

Distillation, done in the open

Here’s the part that should reassure anyone worried the technique itself is shady. DeepSeek also published its own distilled models, out in the open, entirely above board.

They took DeepSeek-R1 as the teacher and distilled it into six smaller open student models, built on the already-open Qwen2.5 and Llama-3 families: sizes of 1.5B, 7B, 8B, 14B, 32B and 70B parameters. This was legal and uncontroversial for one simple reason. The teacher was their own model, and the students were built on bases that were already open. No stolen homework required.

Look at what those small models can do. The dashed red line is OpenAI’s o1-mini scoring 63.6% on the AIME 2024 math exam. The 32-billion-parameter distilled student clears it at 72.6%, and it’s about 21 times smaller than its 671-billion-parameter teacher. Even the 1.5-billion-parameter tiddler, small enough to run on a decent laptop, scores 83.9% on the MATH-500 benchmark. A few years ago that would have been science fiction.

The price shock

If the size chart is the "wow," the price chart is the reason Wall Street panicked.

At launch in early 2025, OpenAI’s o1 reasoning model cost $60.00 to generate a million output tokens (tokens are the chunks of text a model reads and writes, and a million of them runs to roughly 750,000 words). DeepSeek’s R1, a comparable reasoning model, cost $2.19 for the same job. That’s about 27 times cheaper, or roughly 96% off. Both are reasoning-tier models, so this is close to apples-to-apples on capability. When a competitor matches your capability tier and charges 4% of your price, the market doesn’t ask polite questions. It sells.

Where it stands now

By 2025 into 2026, distillation stopped being scary and became boring, which is the best thing that can happen to a technology. It’s a normal, legitimate, industry-wide practice. Every major lab’s "small," "fast," or "cheap" tier (the mini, flash and haiku-class models you already use) is widely understood to be a distilled descendant of a bigger flagship. The Chinese open-weight wave keeps rolling: Qwen from Alibaba, Kimi from Moonshot, GLM from Zhipu, and DeepSeek itself.

The controversy narrowed to exactly where it belongs. Not "is distillation cheating," but "whose outputs did you train on, and against whose Terms of Service." That’s a contract question, not a science question.

Why a supply chain reader should care

You might be wondering why a blog for supply chain professionals is spending a Tuesday on AI lab drama. Fair. Here’s the thread.

The tools creeping into your daily work (the demand-forecast assistant, the supplier-email triage bot, the spec-extraction helper) are getting radically cheaper, and distillation is a big reason why. A capable model that cost $60 per million tokens in early 2025 has a near-twin at $2.19. That’s the difference between "we can afford to run this on one category" and "we can run it across the whole spend base." When you evaluate an AI vendor’s pricing next quarter, remember that the underlying cost of intelligence is falling fast, and the "mini" model in their pitch is probably a distilled one that’s good enough for 90% of what you need.

Interactive Dashboard

Explore the distillation story yourself. Move through the timeline, compare teacher and student model sizes, and see how the price and performance numbers stack up.

Your next steps

Five things you can actually do this week, not vague advice.

1. Run the price math on one real task. Take one AI-assisted job you already pay for (email triage, document summaries, forecast commentary) and check what a distilled "mini" model would cost for the same volume. The gap is often larger than 20x.
2. Blind-test a small model against your current one. Pull 50 real examples of a high-volume task. Run them through your current model and an open distilled one. Score both without looking at the labels. Count how often the cheap one is good enough.
3. Read the abstract of the 2015 Hinton paper (arXiv:1503.02531). Ten minutes, no math required, and you’ll understand distillation better than most people quoting it on LinkedIn.
4. Ask your AI vendors one question: "Is the model in this tier distilled, and from what?" You won’t always get a straight answer, but the quality of the dodge tells you a lot.
5. Bookmark the open-weight landscape. If you haven’t read our companion piece on the current open-weight model field, start there, then reassess whether your current AI spend still makes sense at 2026 prices.

Distillation isn’t villainy and it isn’t magic. It’s compression. The fight was always about consent and contracts, and the dividend, cheaper capable models, lands in your budget whether you followed the drama or not.

# =============================================================================
# generate_distill_images.R
# Charts for "What Is AI Model Distillation?" (2026-07-23)
# All values are hardcoded VERBATIM from:
#   Drafts/2026-07-23_AI_Model_Distillation_FINAL_DATASET.md (Section 5)
# Do NOT edit numbers here — this consumes an already-FINAL dataset.
# Run from project root:  Rscript Scripts/generate_distill_images.R
# =============================================================================

source("Scripts/theme_inphronesys.R")

library(ggplot2)
library(dplyr)
library(scales)

# Roles: challenger / open = brand blue (emphasis); frontier = muted grey
col_focus  <- iph_colors$blue
col_grey   <- iph_colors$grey
col_hi     <- iph_colors$red      # the market-shock milestone
col_navy   <- iph_colors$navy

set.seed(42)  # no randomness used, but per house convention

# =============================================================================
# CHART 1 — distill_timeline.png : the story, 2015 -> 2026
# =============================================================================
timeline <- tibble::tribble(
  ~order, ~date_label,   ~event,                                              ~emphasis,
  1,      "Mar 2015",    "Hinton, Vinyals & Dean publish the\ndistillation paper", "normal",
  2,      "Dec 2024",    "DeepSeek-V3 released\n(671B open-weight MoE)",           "normal",
  3,      "Jan 2025",    "DeepSeek-R1 released;\napp hits #1 in the US",           "normal",
  4,      "27 Jan 2025", "Nvidia loses $589B in a day\n(biggest one-day loss ever)", "high",
  5,      "Jan 2025",    "OpenAI/Microsoft open\ndistillation probe",              "normal",
  6,      "Feb 2026",    "Anthropic + OpenAI flag\n“industrial-scale” distillation", "normal"
)

# alternate labels above / below the spine to avoid overlap
timeline <- timeline %>%
  mutate(
    x      = order,
    y_seg  = ifelse(order %% 2 == 1, 1, -1),
    y_txt  = ifelse(order %% 2 == 1, 1.35, -1.35),
    y_date = ifelse(order %% 2 == 1, 0.42, -0.42),
    is_hi  = emphasis == "high",
    pt_col = ifelse(is_hi, col_hi, col_focus)
  )

p1 <- ggplot(timeline) +
  # the spine
  annotate("segment", x = 0.6, xend = 6.4, y = 0, yend = 0,
           color = iph_colors$lightgrey, linewidth = 1.4) +
  # connectors from spine to text
  geom_segment(aes(x = x, xend = x, y = 0, yend = y_seg),
               color = iph_colors$lightgrey, linewidth = 0.5) +
  # milestone dots
  geom_point(aes(x = x, y = 0, color = pt_col), size = 4.4) +
  geom_point(aes(x = x, y = 0), color = "white", size = 1.5) +
  # date labels near the spine
  geom_text(aes(x = x, y = y_date, label = date_label),
            family = "Inter", fontface = "bold", size = 3.1,
            color = iph_colors$dark) +
  # event text
  geom_text(aes(x = x, y = y_txt, label = event, color = pt_col),
            family = "Inter", size = 2.85, lineheight = 0.92) +
  scale_color_identity() +
  scale_x_continuous(limits = c(0.4, 6.6)) +
  scale_y_continuous(limits = c(-2.1, 2.1)) +
  labs(
    title = "How a compression trick from 2015 wiped out $589 billion in a day",
    subtitle = "The distillation story, from a Google research paper to the DeepSeek shock",
    caption = "Sources: arXiv:1503.02531; arXiv:2412.19437; arXiv:2501.12948; Bloomberg & CNBC (27 Jan 2025); CNBC (24 Feb 2026)."
  ) +
  theme_inphronesys(grid = "none") +
  theme(
    axis.text  = element_blank(),
    axis.title = element_blank(),
    panel.grid = element_blank()
  )

ggsave("https://inphronesys.com/wp-content/uploads/2026/07/distill_timeline.png", p1,
       width = 8, height = 4, dpi = 100, bg = "white")

# =============================================================================
# CHART 2 — distill_price_performance.png : the 27x price shock (output tokens)
# =============================================================================
price <- tibble::tribble(
  ~model,          ~price, ~role,
  "OpenAI o1",     60.00,  "frontier",
  "DeepSeek R1",    2.19,  "challenger"
) %>%
  mutate(model = factor(model, levels = c("OpenAI o1", "DeepSeek R1")),
         fill_col = ifelse(role == "challenger", col_focus, col_grey))

p2 <- ggplot(price, aes(x = model, y = price, fill = fill_col)) +
  geom_col(width = 0.55) +
  geom_text(aes(label = dollar(price, accuracy = 0.01)),
            vjust = -0.5, family = "Inter", fontface = "bold",
            size = 4.2, color = iph_colors$dark) +
  # 27x callout arrow between the two bars
  annotate("segment", x = 1, xend = 2, y = 45, yend = 45,
           color = col_focus, linewidth = 0.5,
           arrow = arrow(length = unit(0.02, "npc"), type = "closed")) +
  annotate("text", x = 1.5, y = 49,
           label = "about 27× cheaper",
           family = "Inter", fontface = "bold", size = 4,
           color = col_focus) +
  annotate("text", x = 1.5, y = 40,
           label = "$60.00 ÷ $2.19 = 27.4",
           family = "Inter", fontface = "italic", size = 3,
           color = iph_colors$grey) +
  scale_fill_identity() +
  scale_y_continuous(limits = c(0, 66),
                     labels = dollar_format(accuracy = 1),
                     expand = expansion(mult = c(0, 0.02))) +
  labs(
    title = "Same job, one twenty-seventh of the bill",
    subtitle = "Price to generate one million output tokens, reasoning models, early-2025 API prices",
    x = NULL,
    y = "Output price per million tokens",
    caption = "Sources: OpenAI API pricing (o1); DeepSeek R1 launch pricing (gate.ai). Both are reasoning-tier models."
  ) +
  theme_inphronesys(grid = "y")

ggsave("https://inphronesys.com/wp-content/uploads/2026/07/distill_price_performance.png", p2,
       width = 8, height = 4.5, dpi = 100, bg = "white")

# =============================================================================
# CHART 3 — distill_size_performance.png : tiny students, real scores
# =============================================================================
students <- tibble::tribble(
  ~student,                 ~params, ~aime, ~lab_hjust, ~lab_vjust,
  "1.5B",                   1.5,     28.9,  -0.35,       0.5,
  "7B",                     7,       55.5,  -0.35,       0.5,
  "Llama-8B",               8,       50.4,   1.30,       0.5,
  "14B",                    14,      69.7,  -0.30,      -0.6,
  "32B",                    32,      72.6,  -0.30,       0.5,
  "Llama-70B",              70,      70.0,  -0.30,       1.4
)

o1_mini <- 63.6

p3 <- ggplot(students, aes(x = params, y = aime)) +
  # o1-mini reference line
  geom_hline(yintercept = o1_mini, linetype = "dashed",
             color = iph_colors$red, linewidth = 0.5) +
  annotate("text", x = 1.5, y = o1_mini + 2.4,
           label = "OpenAI o1-mini = 63.6",
           family = "Inter", fontface = "italic", size = 3,
           color = iph_colors$red, hjust = 0) +
  geom_line(color = iph_colors$lightgrey, linewidth = 0.8) +
  geom_point(size = 3.6, color = col_focus) +
  # highlight the 32B point
  geom_point(data = filter(students, student == "32B"),
             size = 5.4, shape = 21, stroke = 1.1,
             color = col_navy, fill = col_focus) +
  geom_text(aes(label = paste0(student, ": ", aime),
                hjust = lab_hjust, vjust = lab_vjust),
            family = "Inter", size = 2.9, color = iph_colors$dark) +
  annotate("text", x = 30, y = 40,
           label = "Teacher = DeepSeek-R1, 671B params\n(off the chart, ~21× the 32B student)",
           family = "Inter", fontface = "italic", size = 3,
           color = iph_colors$grey, hjust = 0.5, lineheight = 0.95) +
  scale_x_log10(breaks = c(1.5, 3, 7, 14, 32, 70),
                labels = c("1.5B", "3B", "7B", "14B", "32B", "70B"),
                limits = c(1.2, 130)) +
  scale_y_continuous(limits = c(20, 82), breaks = seq(30, 80, 10)) +
  labs(
    title = "Distillation, done in the open: small students, grown-up scores",
    subtitle = "DeepSeek-R1-Distill models on the AIME 2024 math exam (pass@1). Bigger student, higher score.",
    x = "Student model size (parameters, log scale)",
    y = "AIME 2024 score (%)",
    caption = "Source: DeepSeek-R1 paper, arXiv:2501.12948 (Table 5). Distilled into open Qwen2.5 / Llama-3 bases."
  ) +
  theme_inphronesys(grid = "y")

ggsave("https://inphronesys.com/wp-content/uploads/2026/07/distill_size_performance.png", p3,
       width = 8, height = 5, dpi = 100, bg = "white")

# =============================================================================
# CHART 4 — distill_teacher_student.png : conceptual big teacher -> small student
# Circle AREA proportional to parameters (radius proportional to sqrt(params))
# =============================================================================
circle <- function(cx, cy, r, n = 120) {
  t <- seq(0, 2 * pi, length.out = n)
  data.frame(x = cx + r * cos(t), y = cy + r * sin(t))
}

teacher_p <- 671
student_p <- 1.5
# radius scaled so teacher is visually big but fits; area ∝ params
r_teacher <- 1.5
r_student <- r_teacher * sqrt(student_p / teacher_p)  # ~0.071

tc <- circle(2.2, 0, r_teacher)
sc <- circle(7.6, 0, r_student)

p4 <- ggplot() +
  # teacher
  geom_polygon(data = tc, aes(x, y), fill = col_grey, alpha = 0.9) +
  annotate("text", x = 2.2, y = 0, label = "TEACHER",
           family = "Inter", fontface = "bold", size = 4.2, color = "white") +
  annotate("text", x = 2.2, y = -0.34, label = "DeepSeek-R1",
           family = "Inter", size = 3.2, color = "white") +
  annotate("text", x = 2.2, y = r_teacher + 0.42, label = "671B parameters",
           family = "Inter", fontface = "bold", size = 3.4, color = iph_colors$dark) +
  # student
  geom_polygon(data = sc, aes(x, y), fill = col_focus) +
  annotate("text", x = 7.6, y = -0.62, label = "STUDENT",
           family = "Inter", fontface = "bold", size = 3.6, color = iph_colors$dark) +
  annotate("text", x = 7.6, y = -0.92, label = "R1-Distill-Qwen-1.5B",
           family = "Inter", size = 3, color = iph_colors$grey) +
  annotate("text", x = 7.6, y = 0.55, label = "1.5B",
           family = "Inter", fontface = "bold", size = 3.4, color = iph_colors$dark) +
  # distillation arrow
  annotate("segment", x = 3.9, xend = 7.0, y = 0, yend = 0,
           color = col_navy, linewidth = 1,
           arrow = arrow(length = unit(0.03, "npc"), type = "closed")) +
  annotate("text", x = 5.45, y = 0.42, label = "distillation",
           family = "Inter", fontface = "bold", size = 3.8, color = col_navy) +
  annotate("text", x = 5.0, y = -0.62,
           label = "the student learns to\nimitate the teacher's answers",
           family = "Inter", fontface = "italic", size = 2.9,
           color = iph_colors$grey, lineheight = 0.95) +
  annotate("text", x = 5.0, y = -1.7,
           label = "447× smaller — and still scores 83.9% on the MATH-500 exam",
           family = "Inter", fontface = "bold", size = 3.3, color = col_focus) +
  coord_fixed(xlim = c(0, 9.5), ylim = c(-2.1, 2.3)) +
  labs(
    title = "The big idea: shrink the model, keep the smarts",
    subtitle = "Distillation compresses a huge “teacher” into a small “student” that mimics its output",
    caption = "Circle area scales with parameter count. Source: DeepSeek-R1 paper, arXiv:2501.12948. 671 / 1.5 = 447."
  ) +
  theme_inphronesys(grid = "none") +
  theme(
    axis.text  = element_blank(),
    axis.title = element_blank(),
    panel.grid = element_blank()
  )

ggsave("https://inphronesys.com/wp-content/uploads/2026/07/distill_teacher_student.png", p4,
       width = 8, height = 4.5, dpi = 100, bg = "white")

cat("Done. Generated 4 PNGs in Images/\n")

References

• Hinton, G., Vinyals, O., & Dean, J. (2015). Distilling the Knowledge in a Neural Network. arXiv:1503.02531. https://arxiv.org/abs/1503.02531
• DeepSeek-AI (2024). DeepSeek-V3 Technical Report. arXiv:2412.19437. https://arxiv.org/abs/2412.19437
• DeepSeek-AI (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948. https://arxiv.org/abs/2501.12948
• Bloomberg (27 Jan 2025). Nvidia Loses $589 Billion as DeepSeek Batters Stock. https://www.bloomberg.com/news/newsletters/2025-01-27/nvidia-loses-589-billion-as-deepseek-batters-stock-evening-briefing-americas
• CNBC (27 Jan 2025). Nvidia sheds almost $600 billion in market cap, biggest drop ever. https://www.cnbc.com/2025/01/27/nvidia-sheds-almost-600-billion-in-market-cap-biggest-drop-ever.html
• Stratechery (2025). DeepSeek FAQ. https://stratechery.com/2025/deepseek-faq/
• TechCrunch (29 Jan 2025). Microsoft probing whether DeepSeek improperly used OpenAI’s API. https://techcrunch.com/2025/01/29/microsoft-probing-whether-deepseek-improperly-used-openais-api/
• Fortune (29 Jan 2025). What is distillation? David Sacks on DeepSeek and OpenAI. https://www.fortune.com/2025/01/29/deepseek-openais-what-is-distillation-david-sacks
• The New York Times Company v. Microsoft Corporation and OpenAI (filed 27 Dec 2023, S.D.N.Y.). https://en.wikipedia.org/wiki/The_New_York_Times_v._Microsoft_and_OpenAI
• CNBC (24 Feb 2026). Anthropic, OpenAI raise alarm on Chinese AI distillation. https://www.cnbc.com/2026/02/24/anthropic-openai-china-firms-distillation-deepseek.html