LLM distillation: transfer capability without assuming equivalence
Use distillation to transfer a bounded capability into a smaller model, then test robustness and residual losses.
Distillation transfers observed behaviors; it does not guarantee global equivalence with the teacher.
Average scores can hide loss of rare cases, correct refusals or out-of-distribution robustness.
The child is retained only for the capability contract actually preserved under protected evaluation.
What the architecture must make explicit
Cost is reduced at realization level: avoiding calls, shortening context, quantizing, routing or rematerializing a responsibility can matter more than isolated model optimization.
Move from a keyword to an inspectable engineering decision
Profile
Measure actual system costs, including retries, transfers, memory and verification.
Attribute
Determine which responsibilities still require a powerful model.
Transform
Test reduction, quantization, specialization, caching, routing or substitution.
Compare
Keep the same capability contract and publish exact hardware conditions.
A claim is only as strong as its binding
The gate must evaluate the exact output produced by the exact realization. Integrity, scientific validity and authority are related, but never interchangeable.
Questions that change the design
What does llm distillation: transfer capability without assuming equivalence require?
Distillation transfers observed behaviors; it does not guarantee global equivalence with the teacher. Cost is reduced at realization level: avoiding calls, shortening context, quantizing, routing or rematerializing a responsibility can matter more than isolated model optimization.
Which evidence should be retained?
At minimum: exact realization identity, inputs and outputs, protocol version, measurements, failure cases and the decision rule. The child is retained only for the capability contract actually preserved under protected evaluation.
Does AMF prescribe a particular model?
No. AMF treats the model, runtime, data, tools and deterministic components as candidate materials. Their retention depends on measured contribution and declared constraints.