By Nitesh
- AI distillation trains a smaller model to imitate a larger one by learning from its outputs, not its code or weights.
- It is standard practice. Nearly every lab uses it internally to compress frontier models into cheap, deployable ones.
- Nothing is breached. The API is used as designed, which is precisely why it is hard to prevent.
- US agencies have accused six Chinese AI firms of doing it at scale against American models, and Treasury has threatened sanctions.
AI distillation went from a routine engineering technique to a geopolitical accusation in about eighteen months. Understanding what it actually is explains both why labs rely on it and why it is nearly impossible to stop.
This guide covers the mechanism, why it works, what it costs, and where the legal line now sits.
What is AI distillation, in plain English?
You have a large, expensive, capable model. You want a small, cheap one that behaves similarly. So you ask the large model a great many questions, record its answers, and train the small model on those question-and-answer pairs.
The small model learns to produce what the large one produced. It is imitation by example, the same way a student learns by working through solved problems rather than by reading the teacher’s brain.
The terminology is deliberate: the large model is the teacher, the small one the student. The technique is described in the original 2015 paper by Geoffrey Hinton and colleagues, available via arXiv.
Distillation is a technique where you feed the outputs of a more powerful AI model to a weaker one during training, essentially teaching the weaker model to imitate the stronger model’s behaviour.
Why does AI distillation work at all?
AI distillation works because the teacher’s outputs contain more information than the original training data did.
Raw text tells a model what humans wrote. A teacher model’s answer tells it what a highly capable system concluded, including the reasoning steps, the structure of a good response, and which of several plausible answers to prefer. That is a far denser signal.
Better teachers also produce cleaner data. A frontier model asked to solve ten thousand maths problems will get most right and phrase them consistently. Curating equivalent material from the open internet would take far longer and be noisier.
How much cheaper is it than training from scratch?
Substantially. The cost asymmetry is what makes AI distillation strategically important.
Training a frontier model requires enormous compute, a large research team, and years of accumulated methodology. Distilling one requires API access, a budget for queries, and the engineering to train on the results. The gap between those two costs is not a small discount. It is the difference between a national-scale project and a well-funded startup’s quarterly spend.
This is also why distillation is genuinely useful and not merely parasitic. It is how a capable model becomes affordable enough to run on a phone, and how AI agents become cheap enough to run in the thousands.
What are the limits?
AI distillation has real limits. A student rarely exceeds its teacher, and it inherits the teacher’s mistakes.
Distillation transfers behaviour, including bad behaviour. If the teacher is confidently wrong about something, the student learns to be confidently wrong in the same way. And because the student never saw the underlying reasoning, only the outputs, it tends to be more brittle at the edges of what it was shown.
It also cannot transfer what the teacher was never asked. A distilled model’s capability boundary is drawn by the query set, which is why the design of those millions of prompts matters more than the volume.
Why has AI distillation become a sanctions question?
Because nothing is stolen in a way existing law recognises easily.
No weights are copied. No source code changes hands. No system is breached. The API is used exactly as built, at a volume and for a purpose the seller prohibits in its terms of service but cannot technically prevent.
Terms of service are a contract, not a control. Enforcement means detecting the pattern, and the detection signals are behavioural: query volume, account creation patterns, proxy origins, and whether the prompt distribution looks like a customer using a product or a competitor harvesting a dataset.
That is the substance of the current dispute. As we reported when US agencies accused Moonshot of distilling Anthropic’s Fable 5 into Kimi K3, the allegation rests on multiple accounts and proxy services used to stay under rate limits, which is what turns ordinary usage into alleged evasion.
How would anyone prove it?
With difficulty, and probably not from the model itself. Guidance on model provenance and evaluation is published by bodies including NIST.
Two models trained on overlapping internet-scale data answer similarly on many prompts without either copying the other. Statistical similarity is suggestive, not conclusive.
The stronger evidence sits in the teacher’s API logs, which only the teacher holds. That creates an awkward situation where the accusing party owns the proof and has commercial reasons not to publish it, while the accused cannot disprove a negative.
What does a distillation dataset actually look like?
Concretely: a JSON file with a few million rows, each holding a prompt and the teacher’s full response.
The design of those prompts is the entire craft. A naive approach asks the teacher a million random questions and trains on whatever comes back. That produces a student that is broadly mediocre. A deliberate approach targets the specific capabilities you want to acquire.
If the goal is coding ability, the prompt set concentrates on code generation, debugging, and explanation across many languages and difficulty levels. If the goal is mathematical reasoning, it concentrates on multi-step problems where the teacher shows its working, because the intermediate steps are what teach the student to reason rather than to guess.
This is why US agencies described the alleged campaign as using “carefully designed requests” rather than simply high volume. Volume without design is expensive noise. The accusation is not that Moonshot used the API a lot. It is that the query distribution looks engineered to extract specific competencies.
Why do the teacher’s probabilities matter more than its answers?
Here is the technical detail that separates real distillation from simple imitation, and it explains why API access is less useful than direct access.
When a model generates a word, it does not pick one option. It produces a probability distribution across its whole vocabulary: perhaps 61% for one word, 22% for another, 9% for a third, and a long tail. Then it samples from that distribution.
Those probabilities are called soft labels, and they carry far more information than the chosen word alone. If the teacher assigns 61% to “increase” and 22% to “rise”, it is telling the student that both are reasonable and one is preferred. Training on the full distribution transfers that nuance. Training only on the final word discards it.
Hinton’s original 2015 paper on distillation is built entirely on this insight. The paper also introduced a temperature parameter, which flattens the distribution during training so the student learns from the teacher’s uncertainty as well as its confidence.
The practical consequence: a lab distilling its own model internally has the soft labels and gets a much better student. A lab distilling through someone else’s public API usually gets only the sampled text, which is a weaker signal. That gap is one reason the timing objections to the Moonshot allegation carry weight.
How many queries would a serious campaign need?
Published distillation work typically uses datasets in the range of hundreds of thousands to a few million examples. The exact number depends on how much capability you are trying to transfer and how large the student is.
Assume a mid-range figure of two million completions, each averaging perhaps 500 output tokens. That is roughly one billion output tokens. At frontier API pricing, a campaign on that scale is a meaningful but entirely affordable expense for a funded company: a cost measured in hundreds of thousands of dollars against a frontier training run measured in hundreds of millions.
That ratio, roughly three orders of magnitude, is the number that makes distillation strategically important and impossible to ignore commercially.
The rate-limit problem is what forces the evasion. A single account on a commercial tier cannot process a billion output tokens quickly. Spreading the load across many accounts and proxy origins is how you compress months of allowed usage into weeks, and it is precisely the pattern that shows up in logs.
What signals would a provider see?
Distillation at scale looks nothing like ordinary product usage, and the differences are measurable.
Prompt diversity without user diversity. Real customers ask narrow, repetitive questions related to their business. A harvesting campaign issues enormously varied prompts covering many domains, which is what a training set needs and what a product user never does.
No conversational continuation. Humans follow up, clarify and refine. A harvester sends a prompt, takes the answer, and never returns to that thread.
Uniform request timing. Human usage follows daily and weekly rhythms tied to working hours. Automated harvesting runs flat, around the clock, at whatever rate the limits allow.
Account creation clustering. Many accounts registered close together, with similar payment patterns or shared infrastructure, then all consuming at maximum rate.
None of these is conclusive alone. Together they form a signature that is hard to disguise while still getting the volume you need, and this is the evidence class that would settle the current dispute if any provider chose to publish it.
Can a model be watermarked to prove distillation?
Attempts exist, and none of them work well yet.
The theory is to deliberately seed the teacher’s outputs with a statistical fingerprint: a subtle bias in word choice, detectable in aggregate but invisible in any single response. If a student inherits the fingerprint, that is evidence of training on the teacher’s text.
Three problems break it in practice. The fingerprint has to survive the student’s own training, which averages over millions of examples and tends to wash out weak signals. It has to survive paraphrasing or filtering, which any careful harvester would apply. And it must not degrade the teacher’s own output quality, since customers are paying for the best answers, not marked ones.
There is also a game-theory problem. A watermark that is publicly documented can be stripped. A watermark kept secret cannot be used as evidence without disclosing it, at which point it becomes strippable. This is why the enforcement conversation has moved toward API telemetry and export controls rather than technical proof.
How does distillation differ from fine-tuning and RAG?
These get conflated constantly, and the distinctions matter legally as well as technically.
Fine-tuning takes an existing model and continues training it on your own data. The starting model is yours or licensed; the new data is yours. No third-party model is involved.
Retrieval-augmented generation does not change the model at all. It fetches relevant documents at query time and includes them in the prompt. The model’s weights are untouched.
Distillation creates or improves a model using another model’s outputs as the training signal. That third-party dependency is the whole legal question, and it is why terms of service target this specifically while permitting the other two.
A fourth case sits awkwardly between them: using a frontier model to generate synthetic training data for a task, rather than to copy its general capability. Most providers prohibit this too, though it is far harder to characterise as extraction, and it is where a great deal of ordinary applied AI work currently sits.
What does the law actually say?
Very little that is settled, which is why this is being handled through sanctions rather than courts.
Model weights can be trade secrets, but distillation does not access them. Copyright protects expression, and whether a model’s outputs are copyrightable expression at all remains contested in multiple jurisdictions. Contract law applies cleanly to terms of service, but a breach of contract by a foreign company with no US assets is a remedy on paper.
That leaves export controls and sanctions, which do not require proving theft. They require a policy determination that a transfer of capability occurred contrary to national interest. The evidentiary bar is lower and the decision is political rather than judicial, which is exactly why the Treasury route was chosen.
What should you do differently if you build on APIs?
Five things, in order of how much trouble they save.
Keep provenance records. Log what data trained what model, and where it came from. If you are ever asked, the absence of records is itself a problem.
Read the output-use clause specifically. Most provider terms distinguish between using outputs in your product and using them to train a model. The second is usually prohibited outright.
Do not architect around bulk harvesting. If your pipeline requires millions of completions from a third-party API to function, you have built a dependency that a terms change can end overnight.
Prefer open-weight models where you need training freedom. Models released under permissive licences can be distilled and fine-tuned without this exposure. The capability gap is narrowing.
Assume the rules tighten. Any sanctions regime aimed at distillation will need a working definition of acceptable API use for model development, and that definition will apply to everyone rather than only to the firms currently named.
What does this mean if you are building on someone else’s API?
Read the terms, and assume the rules on AI distillation will tighten.
Most providers already prohibit AI distillation from their outputs to train competing models. What is changing is enforcement appetite and the arrival of state-level consequences. A sanctions regime aimed at distillation will require a working definition of acceptable API use for model development, and that definition will apply to everyone, not only to the firms currently named.
For Indian AI companies specifically, much of the applied sector builds on frontier APIs. The practical advice is unglamorous: keep records of what you trained on, avoid architectures that require bulk output harvesting, and do not assume a technique being standard makes it permitted.
What's your reaction?
